Method and system for three-dimensional imaging - Patents.com

Real-time visual feedback during intraoral scans using projected images corrects registration gaps and ensures data sufficiency, addressing manual inefficiencies and inaccuracies in dental imaging systems.

JP7596274B2Active Publication Date: 2024-12-09DENTSPLY SIRONA INC +1
View PDF 11 Cites 0 Cited by

Patent Information

Application Number
JP2021539038
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-01-30
Filing Date
2020-01-22
Publication Date
2024-12-09
Estimated Expiration
2040-01-22

AI Technical Summary

Technical Problem

Current dental imaging systems require tedious manual processes for ensuring data quality during intraoral scans, leading to user disorientation and potential inaccuracies, with insufficient feedback on data sufficiency and registration success.

Method used

A method and system that provides real-time visual feedback through projected images or light patterns onto the oral cavity, indicating successful or unsuccessful registration and data sufficiency, allowing for dynamic control of the scanning process without constant monitor viewing.

Benefits of technology

Enables efficient, real-time correction of registration gaps and ensures high-quality data acquisition by projecting feedback directly on the oral cavity, reducing user disorientation and improving scanning accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007596274000001
    Figure 0007596274000001
  • Figure 0007596274000002
    Figure 0007596274000002
  • Figure 0007596274000003
    Figure 0007596274000003
Patent Text Reader

Abstract

A method and system for dynamically providing visual feedback on the quality of data collected during intraoral scanning. Images or light patterns are projected onto objects, such as teeth, for 3D measurements and to relay feedback to a user about the quality of the 3D measurements. In this way, unsuccessful registration surface areas of a tooth that have not yet been acquired may be corrected by informing the user to repeat the scan at a corresponding location in the oral cavity.
Need to check novelty before this filing date? Find Prior Art

Description

FIELD OF THEINVENTION

[0001] This application relates generally to methods and systems for three-dimensional (3D) imaging, and more particularly to methods and systems for dynamically providing real-time feedback about the quality of data being collected during an intraoral scan, the real-time feedback being provided at the intraoral scan site.

[0002] There are various options available to treat dental defects, such as by designing restorations or orthodontic aligners. During such treatment, the clinician may scan the patient's oral cavity to create a three-dimensional model of the patient's oral cavity. The collection of high-quality 3D data may be important when reconstructing a three-dimensional image of an object. During the intraoral scan, the clinician may periodically change his / her view between the patient's oral cavity and the display, for example, by orienting the camera in the patient's mouth for 3D data acquisition and controlling the scanning process from the display, to (i) ensure that the data acquisition is ongoing and not interrupted, (ii) check whether the registration of the scanned body or the oral cavity was successful, (iii) check which areas of the oral cavity have already been scanned and which areas have not yet been scanned, and / or (iv) ensure that the acquired 3D data is sufficient for clinical purposes and / or that enough individual optical 3D measurements have been acquired for an accurate reconstruction of virtual teeth.

[0003] The main drawbacks of this manual process are that it is tedious, time consuming, and the user may lose orientation as he or she looks back and forth between the patient's mouth and the screen. Additionally, current systems may not indicate whether the acquired data is sufficient or whether some intraoral sites should be scanned again. Moreover, by simply looking at the display, the clinician may miss inaccuracies in the reconstruction that cannot be detected by the untrained eye.

[0004] US Patent No. 9,838,670 B2 discloses a "point and click" camera, determining whether a current position of the camera is sufficient to capture a second image based on a first position of a first image. The invention provides a method for three-dimensional imaging, including storing a first two-dimensional image of a field of view of a scanning device at a first position where a first three-dimensional view of an object is captured by the scanning device, estimating a location metric of a second two-dimensional image of the field of view of the scanning device at the second position relative to the first image while the scanning device is moved from the first position to the second position, and generating instructions for providing feedback to a user based on the location metric, the feedback being provided to indicate whether the second position is sufficient to capture a second three-dimensional view.

[0005] U.S. Patent No. 7,940,260 B2 discloses a scanning system that can acquire three-dimensional images as an incremental series of matched three-dimensional data sets by testing successful incremental matches in real time and providing various visual user cues and process corrections depending on the relationship of newly acquired data to previously acquired data.

[0006] U.S. Pat. No. 4,837,732 A discloses a method for facilitating acquisition of data defining the three-dimensional shape of prepared teeth and their immediate vicinity, comprising displaying a live image from a scanhead on a video display, manually orienting the scanhead relative to a prepared tooth while observing the image of the tooth on the video display, thereafter generating corresponding depth and contrast images from data generated by the scanhead at a selected orientation, and thereafter processing the depth image based on the contrast image. Summary of the Invention

[0007] The above-related existing limitations as well as other limitations can be overcome by the method of claim 1 and the system of claim 11 for dynamically providing visual feedback on the quality of data collected during intraoral scanning. Images or light patterns may be projected onto objects such as teeth for 3D measurements and to relay feedback to the user on the quality of the 3D measurements. Here, failed registrations and holes / gaps (i.e. surface areas not scanned) can be corrected by informing the user to repeat the scan at the corresponding location in the oral cavity.

[0008] In an aspect herein, the invention provides a method for dynamically providing visual feedback on quality and / or other predetermined characteristics of scan data collected during an intraoral scan, said visual feedback being provided at or near an intraoral scan site, the method comprising the steps of providing a plurality of individual optical 3D measurements of surfaces of the intraoral cavity, determining whether each of the plurality of individual optical 3D measurements and the accumulated plurality of 3D measurements meets a predetermined recording criterion, and based on the determining step, projecting said visual feedback including feedback of successful registration and / or feedback of unsuccessful registration onto surfaces of the intraoral cavity corresponding to the individual optical 3D measurements.

[0009] According to another aspect of the invention, a method is provided comprising a combination of one or more of the following: (i) the successful registration feedback and / or unsuccessful registration feedback is projected as part of a projection image comprising a first region corresponding to a projection light beam for 3D measurement and a second region corresponding to the successful registration feedback and / or unsuccessful registration feedback; (ii) further comprising providing the visual feedback in real time; (iii) further comprising extracting 3D coordinates of each of the plurality of individual optical 3D measurements to determine if the optical 3D measurements have been previously scanned; (iv) each of the plurality of individual optical 3D measurements overlaps with another individual optical 3D measurement to form an overlapping area; and (v) the predetermined recording criteria is selected from the group consisting of: (a) a sufficient size of the overlapping area; (b) a sufficient waviness of the object surface in the overlapping area; (c) a sufficient roughness of the object surface in the overlapping area; (d) a sufficient number of characteristic geometries in the overlapping area; (e) a sufficient image quality / resolution in the overlapping area; and / or (f) an area of ​​the intraoral cavity not included in the accumulated dataset.

[0010] In another aspect of the invention, there is provided a system for dynamically providing visual feedback on the quality of data collected during an intraoral scan, said visual feedback being provided at or near an intraoral scan site, the system comprising at least one processor configured to provide a plurality of individual optical 3D measurements of a plurality of measurement surfaces of the intraoral cavity, determine whether each of the plurality of individual optical 3D measurements meets a predetermined registration criterion, and based on the determining step, project said visual feedback comprising feedback of successful registration and / or feedback of unsuccessful registration onto surfaces of the intraoral cavity corresponding to the individual optical 3D measurements.

[0011] According to another aspect of the present invention, there is provided a system including a combination of one or more of the following: (i) the processor is further configured to project the successful registration feedback and / or the unsuccessful registration feedback as part of a projected image including a first region corresponding to a projected light beam for 3D measurement and a second region corresponding to the successful registration feedback and / or the unsuccessful registration feedback; (ii) the processor is further configured to provide the visual feedback in real time; (iii) the processor is further configured to extract 3D coordinates of each of a plurality of individual optical 3D measurements to determine if the optical 3D measurements have been previously scanned; (iv) the system further includes at least one projector and at least one image sensor in communication with the processor; (v) the projector is housed within the intraoral camera; (vi) the projector is separate from the intraoral camera; and (vii) the projector is selected from the group consisting of a digital light processing projector, a light emitting diode projector, a laser projector, a liquid crystal on silicon projector, and a liquid crystal display projector.

[0012] Further features and advantages, as well as the structure and operation of various embodiments herein, are described in detail below with reference to the accompanying drawings.

[0013] The illustrative embodiments will become more fully understood from the detailed description and accompanying drawings set forth herein below. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a block diagram illustrating a system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating inner and outer rays according to an exemplary embodiment of the invention. [Diagram 3] FIG. 3 is a diagram illustrating a measurement volume according to an exemplary embodiment of the invention. [Figure 4] FIG. 4 is a perspective view illustrating a camera projection according to an exemplary embodiment of the present invention. [Diagram 5] FIG. 5 is a top view illustrating a projected image according to an exemplary embodiment of the present invention. [Figure 6] FIG. 6 is a flow chart illustrating a method according to an exemplary embodiment of the invention. [Figure 7] FIG. 7 is a block diagram illustrating a computer system in accordance with an illustrative embodiment of the invention. Detailed Description of the Invention

[0015] According to exemplary aspects described herein, a method and system for dynamically providing feedback on the quality of data being collected during intraoral scanning and feedback on which areas of the oral cavity have already been acquired, said feedback being provided at or near the intraoral scanning site. However, it may be understood that the inventive concepts disclosed herein may not be limited to such applications and may be usefully used in various imaging applications. For example, the systems and methods described herein may be usefully used in non-oral applications, non-dental applications, or other applications where imaging may be enhanced by real-time feedback or where the generation of an object surface in three dimensions may be based on stitching several 3D views captured at any orientation relative to the object using a scanning device under the free control of a user. All such variations and alternative embodiments as would be apparent to one skilled in the art are intended to fall within the scope of the present invention.

[0016] The present invention can provide a method and system for providing feedback during intraoral scanning by means of images or light patterns projected onto objects such as teeth. Thus, the present invention can enable intraoral scanning that may not require a display / monitoring screen, and failed registration gaps / holes can be corrected by notifying the user to repeat the scan at the corresponding location in the oral cavity.

[0017] A system for modeling and visualizing dental solutions FIG. 1 illustrates a block diagram of a light guidance system 101 comprising at least one projector 10 having a light source 12, at least one image sensor 18 (e.g., a 3D image sensor), and a computer system 100 having at least one computer processor 122 (FIG. 7). The projector 10 and the image sensor 18 can be in communication with the computer system 100. The projector 10 can be a projector housed in a camera, such as an intraoral camera 32. Alternatively, the projector 10 can be a separate projector, such as a digital light projector, that is external to the intraoral camera 32. The projector 10 can operate on the principle of filtering the light source 12 based on the image to be displayed. A lens (not shown) can then be used to transfer the image to a surface on which the image can be displayed. Different types of projectors may be used, including digital light processing (DLP) projectors, which may be based on digital micromirror device (DMD) technology, where an array of micromirrors may be configured to tilt toward or away from the light source 12 in the projector 10 to generate bright or dark pixels on the projection surface. Other types of projectors may include light-emitting diode (LED) projectors, laser projectors, liquid crystal on silicon (LCoS) projectors, and liquid crystal display (LCD) projectors. One or more projectors 10 may be used to project one or more projected images 48 (FIG. 5) onto a surface, such as the tooth surface 28 (FIG. 4), and may be constructed and operated according to at least one exemplary embodiment herein. The projectors may illuminate the 3D measurement field and surfaces outside the measurement field. Here, information about already acquired / scanned surfaces, surfaces not yet acquired, as well as other 3D measurement information (such as scanbody registration) may be visually overlaid by using patterns or colors projected onto surfaces outside the measurement field. In this way, the user can control the scanning process while viewing inside the patient's mouth.The projection may be dynamic, and the illumination beam 20 from the projector 10 may be controlled to generate the one or more projected images 48, preferably at predetermined time intervals, and at least a portion of the illumination beam 20 may be reflected into a monitoring beam 50 for 3D measurement.

[0018] As shown in FIG. 4, each projection image 48 may include a first / inner region 44 corresponding to a projection light beam for 3D measurement and / or a second / outer region 42 corresponding to a projection light beam for providing visual feedback to the user, or vice versa. The shape of the inner region 44 and / or the outer region 42 may be predefined, e.g., square, rectangular, circular, etc. In the exemplary embodiment of the invention shown in FIG. 2, the intraoral camera 32 may be electrically paired with a separate projector 10a, which projects at least a portion of the projection image 48 onto the tooth surface 28 for user feedback. The intraoral camera 32 may itself also have a projector 10 for projecting at least another portion of the projection image 48 for 3D measurement. The pairing may be achieved by using a common controller, such as a computer processor 122, to simultaneously control the illumination of the tooth surface 28 by the illumination beam 20 from the projector 10a and the intraoral camera 32 and the recording of the reflected monitoring beam 50.

[0019] In another exemplary embodiment of the present invention as shown in Figure 3, the projector 10b of the intraoral camera 32 may be configured to project a projection image 48 onto the tooth surface 28 for both 3D measurement and user feedback, where the inner ray 24 of the illumination beam 20 may be configured to generate an inner region 44 of the projection image 48 for 3D measurement and / or the outer ray 22 of the illumination beam 20 may be configured to generate an outer region 42 of the projection image 48 for user feedback, or vice versa.

[0020] Thus, the projector 10b of the intraoral camera 32 can be configured to project an illumination beam 20 that includes both a 3D measurement light beam and a user feedback light beam.

[0021] In a further embodiment, projector 10 may be configured to alternate between projecting a 3D measurement light beam and projecting a user feedback light beam.

[0022] In a further embodiment, the intraoral camera 32 can be configured such that the reflected monitoring beam 50 for 3D measurement includes all or substantially all of the portion of the illumination beam 20 configured for 3D measurement. The further portion of the illumination beam 20 configured for 3D measurement (the inner ray of the illumination beam 24) may be configured to be a structured illumination pattern and / or may be modulated before illumination. By using various structured illumination patterns, the 3D surface profile of the tooth surface 28 can be measured. Furthermore, by modulating at a predetermined frequency, only the signal corresponding to that frequency may be detected by the image sensor 18 for further processing.

[0023] How to model and visualize dental solutions Having described the light guidance system 101 of FIG. 1, a method for dynamically providing feedback on the quality of the data being collected during an intraoral scan will be further described in relation to FIG.

[0024] The invention may include generating and updating a 3D model through a scanning process by a user moving the intraoral camera head 30 over a location (e.g., tooth surface 28). The scan may begin as shown in step S100, and the memory of the computer system 100 may store the volume of the oral cavity 52 as an empty volume that may be later updated as the scan progresses. After placing the intraoral camera head 30 over the tooth surface in step S200, the projector 10 of the intraoral camera 32 may project an initial default projection image onto an area of ​​the oral cavity 52 to indicate that image data for processing has not yet been acquired.

[0025] In the present exemplary embodiment, the projector 10 may be configured to provide a successful registration feedback indicating a successful registration of the current measurement surface 54 and an unsuccessful registration feedback indicating an unsuccessful registration of the current measurement surface 54, for example a blue rectangle or stripe pattern for 3D measurement (inner area 44 of the projected image 48 in FIG. 5, which also corresponds to the current measurement surface 54) may be inside a larger red rectangle (outer area 42 of the projected image 48 corresponding to the other surface 56 of the cavity), the red rectangle indicating an unsuccessful registration of the intra-oral surface on which said feedback is projected. Upon successful registration, the corresponding surface 46 of the successful registration in the outer area 42 may receive a successful registration feedback, for example a green light, indicating that the corresponding surface 46 of the successful registration has been successfully registered. In the present embodiment, the outer area 42 may receive an initial unsuccessful registration feedback indicating that the scan has not started. Of course, other implementations of the shapes and / or colors of the different components of the projected image 48 may be realized without departing from the scope of the present invention.

[0026] Preferably, the inner rays 24 of the illumination beam 20 as shown in FIG. 3 (or rays from the intraoral camera 32 in the light guidance system 101 with separate projector 10a for user feedback as shown in FIG. 2), which may be configured to form the inner region 44 of the projected image 48, may pass through the current measurement volume 26 (i.e., the volume under the intraoral camera head 30) and land on at least a portion of the tooth surface 28 to be measured, said portion of the tooth surface 28 being the current measurement surface 54 (step S300). The outer rays 22 of the illumination beam 20 may coincide on the other surface 56 of the cavity within a predefined boundary 58 within which feedback may be visually relayed to the user as to whether the corresponding point on said other surface 56 has been scanned before. At the start of the scan, default feedback may be relayed to the user to indicate that said other surface 56 has not been scanned before.

[0027] As shown in step S300, the inner ray 24 of the illumination beam may be substantially reflected from the current measurement surface 54 towards the image sensor 18 of the intraoral camera 32 for further processing. Here, the three-dimensional coordinates of the current measurement surface 54 may be extracted in step S400, and it may be determined whether all or a part of the current measurement surface 54 has been previously registered (step S500). If the current measurement surface 54 or any part of the current measurement surface has not been previously acquired, the corresponding new 3D data (e.g., xyz coordinates) of the unregistered part may be stored (step S600) by determining whether the new 3D data and / or the accumulated 3D data are sufficient according to one or more predefined recording criteria / conditions, the predefined recording criteria including, for example, whether the new 3D data and / or the accumulated 3D data have a desired predefined resolution, a predefined noise level, a predefined 3D point density, and / or a mismatch between the individual optical 3D measurements. If the predefined recording criteria are met, the new 3D data may be stored in an image stack for post-processing, as shown in step S800, or the 3D data may be used to create a real-time 3D reconstruction of the intraoral cavity. The outer region 42 of the projection image 48 corresponding to the new 3D data may then be updated in step S1000 to relay to the user that the previous measurement surface has been sufficiently recorded according to the predefined recording criteria when the camera position of the intraoral camera 32 changes to the new measurement surface. Here, the need to constantly look at a monitor to track the progress of the ongoing intraoral scan and make any necessary adjustments may be eliminated or substantially eliminated, as visual feedback to the user may be shown in the vicinity of the intraoral surface being scanned.

[0028] Alternatively, if the current measurement surface 54 has been previously scanned, a next acquisition may be triggered (steps S700, S1200) to continue recording other surfaces.

[0029] Further, if the new data acquired in step S600 is insufficient according to predetermined recording criteria, the default feedback to the user indicating that the current measurement surface 54 has not been scanned / registered may remain unchanged or may be changed in a predetermined manner to indicate that the current measurement surface 54 has not been scanned.

[0030] In an exemplary embodiment of the invention, as shown in step S600, determining whether new data is sufficient may be accomplished as follows: A plurality of three-dimensional optical measurements may be received by the processor 122, each of the plurality of three-dimensional optical measurements corresponding to a plurality of current measurement surfaces 54 of one or more dental objects generated by the intraoral camera 32 according to one or more recording sequences. Each optical measurement / image may be overlapped with another optical measurement to form a plurality of overlapping areas (not shown), each overlapping area may include a later optical image and a previous optical image, the later optical image being generated at a later imaging position in the recording sequence than the previous optical image. Here, for each overlapping area, it may be determined whether the overlapping area satisfies one or more predetermined recording criteria. The predetermined recording criteria may include, for example, (i) sufficient size of the overlap area, (ii) sufficient waviness of the object surface in the overlap area, (iii) sufficient roughness of the object surface in the overlap area, (iv) a sufficient number of characteristic geometries in the overlap area, (v) sufficient image quality / resolution in the overlap area, and / or (vi) areas of the oral cavity not included in the accumulated dataset.

[0031] If a predetermined registration criterion is met, the later optical 3D measurement may be added / registered to an image sequence containing the previous optical 3D measurement, and / or a corresponding global 3D image of the sequence may be reconstructed in real time or at a later time (step 800).Those skilled in the art will appreciate that other registration steps may be used herein without departing from the essential attributes of the invention.

[0032] Based on the relative position of the camera head 30 from the surface being measured, the "x,y,z" coordinates of the images in the sequence, and / or the reconstructed global 3D image, the projector 10 may be configured to project feedback to the user indicating which corresponding portions of the intraoral cavity have been sufficiently recorded and which areas have not been sufficiently recorded (step S1000), for example a green light may be projected on the portions of the intraoral cavity that have been sufficiently recorded and a red light may be projected on the portions of the intraoral cavity that have not been recorded or have not yet been sufficiently recorded. This may be repeated in real time for all other surfaces and recording sequences, as shown in steps S1100, S1300. Thus, the quality of the registration during the scan may be provided to the user in progress, and as such the user may not need to constantly look at the monitor to check the progress of the scanning procedure, as failed registrations may be corrected by repeating the scan at the corresponding location in the intraoral cavity.

[0033] In another workflow, the current measurement surface 54 may be scanned and a corresponding optical image acquired. The corresponding optical image may include information about the 3D point cloud, the position of the intra-oral camera 32 relative to the 3D point cloud. The corresponding optical image may be registered to the accumulated 3D data set from previous measurements and added to the accumulated data set if there is an overlap (e.g., a match according to an Iterative Closest Point (ICP) algorithm). The position information for the intra-oral camera 32 relative to the accumulated data set may be known. If there is no overlap (no match), the intra-oral camera 32 may lose track of its position and the user may move the intra-oral camera 32 to a position where the new measurement may be registered and added to the 3D data set. The accumulated data set may then be analyzed for holes / gaps in the data, low point density and / or inconsistencies. Additionally, the analysis may include or be a determination of whether the temperature of the intra-oral camera 32 exceeds a predefined value. Based on the results of the analysis and the position of the intra-oral camera 32 relative to the accumulated data set, a projected image 48 for the projector 10 may be generated as follows: (i) Areas under the camera where the accumulated data set is not represented may be indicated by a first color (e.g. red) in the projection image 48, (ii) areas like holes / gaps, low density areas and inconsistencies may be indicated by a second color (e.g. blue) in the projection image 48, and / or (iii) areas containing valid data (areas where the accumulated data is represented) may be indicated by a third color (e.g. green). Here, the intra-oral camera 32 is moving and may directly trigger the next acquisition once the current acquisition is processed. In the embodiment here, the intra-oral camera 32 may determine the projection direction of the light ray from a relative transformation of the previous 3D measurements to the accumulated data set. Here, the intra-oral camera 32 may have information about its position relative to the accumulated 3D data set. Based on this information, the intra-oral camera 32 may determine the position (relative to the camera window) of the areas that were not recorded well (red areas in the feedback projection) and project the corresponding light ray to said areas.Also, in an exemplary embodiment, information about the scanning workflow may be projected based on the determined positional information. For example, the visual feedback may include information about the next area of ​​the intraoral cavity 52 for acquisition, fogging of the intraoral camera window based on the determined temperature of the intraoral camera 32, instructions to remove the camera from the intraoral cavity 52 based on said fogging, etc. The temperature indication may be followed by a warning, such as one indicating that the intraoral camera 32 may soon shut down. Additionally, the indication of fogging may be followed by instructions, such as instructions to heat or blow off the camera.

[0034] Furthermore, by using illumination that can alternate between the projection light beam for providing feedback and the projection light beam for 3D measurement, interference between the projection light beam for providing feedback and the projection light beam for 3D measurement can be reduced or eliminated.

[0035] Of course, other methods similar to those described herein for modeling and visualizing dental solutions may be implemented in light of the description provided.

[0036] In view of the foregoing, it will be appreciated that the exemplary embodiments described herein provide a method for dynamically providing feedback on the quality of data being collected during an intraoral scan.

[0037] A computer system for modeling and visualizing dental solutions Having described the light guidance systems 101 and methods, reference is now made to Figure 7, which illustrates a block diagram of a computer system 100 that may be used in accordance with at least some of the example embodiments described herein. Although various embodiments may be described herein with respect to this example computer system 100, it may become apparent to one skilled in the art after reading this description how to implement the present disclosure using other computer systems and / or architectures.

[0038] In embodiments herein, computer system 100 may form part of light guidance system 101. In other embodiments herein, computer system may be separate from light guidance system 101. Although various embodiments are described herein with respect to this exemplary computer system 100, after reading this description, it may become apparent to one of ordinary skill in the art how to implement the present disclosure using other computer systems and / or architectures.

[0039] The computer system 100 may include one or more processors 122. The computer processor 122 may include, for example, a central processing unit, a multi-processing unit, an application specific integrated circuit ("ASIC"), a field programmable gate array ("FPGA"), or the like. The processor 122 may be connected to a communication infrastructure 124 (e.g., a communication bus or network). In the present embodiment, the processor 122 includes a CPU that acquires an image stack or image sequence from a signal pre-processing unit of the intraoral camera 32. The stack may be temporarily stored in memory and then analyzed to determine whether the images meet predetermined recording criteria, as described in the methods discussed herein. The processor 122 may then operate the projector 10 and / or the light source 12 to control the illumination beam 20.

[0040] Computer system 100 may optionally include a display interface (or other output interface) 126, which may transfer video graphics, text, and other data from communications infrastructure 124 (or from a frame buffer, not shown) for display on display unit 128 (which in one exemplary embodiment may form or be included in display unit 128 of FIG. 1), although display interface 126 and / or display unit 128 may not be required.

[0041] Computer system 100 may also include an input unit 130 that may be used by a user of computer system 100 to send information to computer processor 122. Input unit 130 may include a keyboard and / or a trackball or other input device, such as a touch screen monitor.

[0042] In one example, the display unit 128, the input unit 130, and the computer processor 122 may collectively form a user interface.

[0043] One or more steps for generating the 3D image and controlling the system 101 to generate feedback to the user may be stored in a non-transitory storage device in the form of computer readable program instructions. To execute a procedure, the processor 122 loads the appropriate instructions, such as those stored in the storage device, into memory and executes the loaded instructions.

[0044] The computer system 100 of FIG. 6 may include a main memory 132, which may be random access memory ("RAM"), and may also include a secondary memory 134. The secondary memory 134 may include, for example, a hard disk drive 136 and / or a removable storage drive 138 (e.g., a floppy disk drive, a magnetic tape drive, an optical disk drive, a flash memory drive, and the like). The removable storage drive 138 may read from and / or write to a removable storage unit 140 in a well-known manner. The removable storage unit 140 may be, for example, a floppy disk, a magnetic tape, an optical disk, a flash memory device, and the like, which may be read from and written to by the removable storage drive 138. The removable storage unit 140 may include a non-transitory computer-readable storage medium that stores computer-executable software instructions and / or data.

[0045] In further alternative embodiments, secondary memory 134 may include other computer-readable media that store computer-executable programs or other instructions to be loaded into computer system 100. Such devices may include removable storage units 144 and interfaces 142 (e.g., program cartridges and cartridge interfaces), removable memory chips (e.g., erasable programmable read-only memory ("EPROM") or programmable read-only memory ("PROM")) and associated memory sockets, and other removable storage units 144 and interfaces 142 that allow software and data to be transferred from removable storage unit 144 to other portions of computer system 100.

[0046] Computer system 100 may also include a communications interface 146 that allows software and data to be transferred between computer system 100 and external devices. Such interfaces may include a modem, a network interface (e.g., an Ethernet card or an IEEE 802.11 wireless LAN interface), a communications port (e.g., a Universal Serial Bus ("USB") port or a FireWire port), a Personal Computer Memory Card International Association ("PCMCIA") interface, Bluetooth, and the like. The software and data transferred via communications interface 146 may be in the form of signals, which may be electronic, electromagnetic, optical, or another type of signal that may be capable of being transmitted and / or received by communications interface 146. The signals may be provided to communications interface 146 via communications paths 148 (e.g., channels). Communications paths 148 may carry signals and may be implemented using wire or cable, optical fibers, telephone lines, cellular links, radio frequency ("RF") links, or the like. Communications interface 146 may be used to transfer software or data or other information between computer system 100 and a remote server or cloud-based storage device (not shown).

[0047] One or more computer programs or computer control logic may be stored in main memory 132 and / or secondary memory 134. Computer programs may also be received via communications interface 146. The computer programs may include computer-executable instructions that, when executed by computer processor 122, cause computer system 100 to perform the methods described. Thus, the computer programs may control computer system 100 and other components of light guidance system 101.

[0048] In another embodiment, the software may be stored on a non-transitory computer readable storage medium and loaded into the main memory 132 and / or secondary memory 134 of the computer system 100 using the removable storage drive 138, the hard disk drive 136, and / or the communications interface 146. The control logic (software), when executed by the processor 122, may cause the computer system 100, and more generally, the light guidance system 101 in some embodiments, to perform all or part of the methods described herein.

[0049] Finally, in another exemplary embodiment, the functionality described herein may be performed using hardware components such as ASICs, FPGAs, and the like. The implementation of such hardware configurations for performing the functionality described herein will be apparent to those skilled in the art in view of this description.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although methods and materials similar or equivalent to those described herein can be used in the practice or testing of this disclosure, suitable methods and materials are described above. All publications, patent applications, patents, and other references mentioned herein are incorporated by reference in their entirety to the extent permitted by applicable laws and regulations. The present disclosure may be embodied in other specific forms without departing from its spirit or essential attributes, and therefore, it may be desired that the present embodiments be considered in all respects as illustrative and not restrictive. Any headings used within the description are for convenience only and have no legal or limiting effect. The invention as described in the claims of the original application is set forth below. [1] A method of controlling a system for dynamically providing visual feedback about scan data collected during an intraoral scan, comprising: the visual feedback is provided at or near an intraoral scan site; The system (101) comprises: At least one processor; at least one projector (10) and at least one image sensor (18) in communication with the processor (122); The method comprises: providing a plurality of individual optical 3D measurements of a plurality of measurement surfaces (54, 56) of an intraoral cavity (52); determining whether at least one of the plurality of individual optical 3D measurements and / or the accumulated plurality of optical 3D measurements meets a predetermined recording criterion; and projecting the visual feedback onto surfaces of the intraoral cavity (52) corresponding to the individual optical 3D measurements based on the determining step. [2] The method of [1], wherein the visual feedback is projected as part of a projection image (48) including a first region corresponding to a projection light beam for 3D measurement and a second region corresponding to the visual feedback. [3] The method according to [2], wherein the first region corresponding to the projection light beam for 3D measurement and the second region corresponding to the visual feedback at least partially overlap. [4] The method of [2] or [3], wherein the visual feedback is projected using an illumination beam (20) to generate the first region as an inner region (44) of the projected image (48) from an inner ray (24) of the illumination beam and to generate the second region as an outer region (42) of the projected image (48) using an outer ray (22) of the illumination beam, or vice versa. [5] The method of [4], further comprising alternating the illumination beam between the inner light beam and the outer light beam. [6] The method of any one of [1] to [5], further comprising providing the visual feedback in real time. [7] The method of any one of [1] to [6], further comprising extracting 3D coordinates of each of the plurality of individual optical 3D measurements to determine whether a corresponding region of the intraoral cavity (52) has been previously scanned. [8] The method of any one of [1] to [7], wherein each of the plurality of individual optical 3D measurement values ​​overlaps with another individual optical 3D measurement value to form an overlapping area. [9] The method of any one of [1] to [8], wherein the predetermined recording criteria are selected from the group consisting of: (i) sufficient size of the overlapping area, (ii) sufficient waviness of the object surface in the overlapping area, (iii) sufficient roughness of the object surface in the overlapping area, (iv) a sufficient number of characteristic geometries in the overlapping area, (v) sufficient 3D point density in the overlapping area and / or areas of the oral cavity not included in the accumulated dataset.

[10] The method of any one of [1] to [9], wherein the visual feedback includes feedback about successful registration, feedback about unsuccessful registration, feedback about unscanned areas of the oral cavity, feedback about user workflow, feedback about the temperature of the intraoral camera, or feedback about a warning.

[11] A system for dynamically providing visual feedback about scan data collected during an intraoral scan, the visual feedback being provided at or near an intraoral scan site; The system (101) comprises: At least one processor configured to execute the method according to any one of [1] to

[10] ; The system includes at least one projector (10) and at least one image sensor (18) in communication with the processor (122).

[12] The system described in

[11] , wherein the projector is housed within the intraoral camera (32).

[13] The system described in

[11] , wherein the projector is separate from the intraoral camera (32) having the at least one image sensor.

[14] The system of any one of

[11] to

[13] , wherein the projector (10) is selected from the group consisting of a digital light processing projector, a light emitting diode projector, a laser projector, a liquid crystal on silicon projector, and a liquid crystal display projector.

Claims

1. 1. A method of controlling a system for dynamically providing visual feedback about scan data collected during an intraoral scan, comprising: the visual feedback is provided at or near an intraoral scan site; The system (101) comprises: At least one processor; at least one projector (10) and at least one image sensor (18) in communication with the processor (122); The method comprises: providing a plurality of individual optical 3D measurements of a plurality of measurement surfaces (54, 56) of an intraoral cavity (52); determining whether at least one of the plurality of individual optical 3D measurements and / or the accumulated plurality of optical 3D measurements meets a predetermined recording criterion; and projecting the visual feedback onto surfaces of the intraoral cavity (52) that correspond to the individual optical 3D measurements based on the determining step; said visual feedback being projected as part of a projection image (48) including a first region corresponding to a projection light beam for 3D measurement and a second region corresponding to said visual feedback; The method, wherein the visual feedback is projected using an illumination beam (20) to generate the first region as an inner region (44) of the projected image (48) from an inner ray (24) of the illumination beam and to generate the second region as an outer region (42) of the projected image (48) using an outer ray (22) of the illumination beam, or vice versa.

2. The method of claim 1 , wherein the first region corresponding to a projection light beam for 3D measurement and the second region corresponding to the visual feedback at least partially overlap.

3. The method of claim 1 or 2, further comprising alternating the illumination beam between the inner light beam and the outer light beam.

4. The method of claim 1 , further comprising providing the visual feedback in real time.

5. 5. The method of claim 1, further comprising extracting 3D coordinates of each of the plurality of individual optical 3D measurements to determine whether a corresponding region of the intraoral cavity (52) has been previously scanned.

6. The method of claim 1 , wherein each of the plurality of individual optical 3D measurements overlaps with another individual optical 3D measurement to form an overlapping area.

7. 7. The method of claim 6, wherein the predetermined recording criteria are selected from the group consisting of: (i) sufficient size of the overlapping area, (ii) sufficient waviness of the object surface in the overlapping area, (iii) sufficient roughness of the object surface in the overlapping area, (iv) a sufficient number of characteristic geometries in the overlapping area, (v) sufficient 3D point density in the overlapping area, and / or areas of the oral cavity not included in the accumulated data set.

8. 8. The method of claim 1, wherein the visual feedback comprises feedback on successful registration, feedback on unsuccessful registration, feedback on unscanned areas of the oral cavity, feedback on user workflow, feedback on intraoral camera temperature, or feedback on warnings.

9. 1. A system for dynamically providing visual feedback about scan data collected during an intraoral scan, the visual feedback being provided at or near an intraoral scan site; The system (101) comprises: At least one processor configured to load instructions into a memory and execute the loaded instructions to perform the method of any one of claims 1 to 8; The system includes at least one projector (10) and at least one image sensor (18) in communication with the processor (122).

10. The system of claim 9 , wherein the projector is housed within an intra-oral camera (32).

11. The system of claim 9 , wherein the projector is separate from the intra-oral camera (32) having the at least one image sensor.

12. The system of any one of claims 9 to 11, wherein the projector (10) is selected from the group consisting of a digital light processing projector, a light emitting diode projector, a laser projector, a liquid crystal on silicon projector, and a liquid crystal display projector.

Citation Information

Patent Citations

  • Method and system for placing orthodontic appliances

    JP2003532485A

  • Imaging by optical tomography

    JP2008504049A

  • Device for determining 3D coordinates of object, in particular of tooth

    JP2009078133A

  • 3D scan recovery

    JP2009523547A

  • System and method for processing and displaying intraoral measurement data

    JP2012055695A