Time-lapse visualization of color 3D models

By registering and correlating the current color image scan with the previous 3D scan, the problem of accurately comparing the color and texture changes of the 3D intraoral surface in existing technologies is solved, and accurate assessment and visualization of intraoral surface changes are achieved.

CN123076901APending Publication Date: 2026-11-10ALIGN TECHNOLOGY INC
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202580024490.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2025-01-17
Filing Date
2025-01-27
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing technologies struggle to accurately compare color and texture changes on three-dimensional intraoral surfaces, resulting in insufficient accuracy for dentists in assessing changes in these surfaces.

Method used

By registering and associating the current color image scan of the three-dimensional intraoral surface with the previous three-dimensional scan, and superimposing the previous color image scan onto the current three-dimensional scan, a visual comparison of color and texture changes can be achieved.

Benefits of technology

It improves dentists' accuracy in assessing changes in the color and texture of the oral surface, and promotes patient education and treatment adherence.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123076901A_ABST
    Figure CN123076901A_ABST
Patent Text Reader

Abstract

A method includes: registering a current color image scan to a current 3D scan; registering a previous color image scan to a previous 3D scan; and associating the current 3D scan with the previous 3D scan such that a given region A of the current 3D scan is associated with a corresponding region A' of the previous 3D scan. Based on the association between the current 3D scan and the previous 3D scan, the registration of the current color image scan to the current 3D scan, and the registration of the previous color image scan to the previous 3D scan, for a given region A of the current 3D scan, the method applies (i) a portion of the previous color image scan registered to the corresponding region A' of the previous 3D scan to (ii) the given region A of the current 3D scan.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure generally relates to three-dimensional imaging, and more specifically to intraoral three-dimensional imaging. Background Technology

[0002] Digital dental impressions utilize intraoral scanning to generate a three-dimensional digital model of the subject's intraoral surface. Digital intraoral scanners can use a combination of structured light projectors and cameras housed within the scanner to achieve structured light 3D imaging.

[0003] US2019 / 0388193 by Saphier et al. (assigned to the assignee of this application and incorporated herein by reference) describes an apparatus for intraoral scanning comprising an elongated handheld rod with a probe. One or more light projectors and two or more cameras are disposed within the probe. Each light projector has a pattern-generating optics element that can form a light pattern using diffraction or refraction. Each camera can be configured to focus on a lens between 1 mm and 30 mm away from the camera sensor. Other applications are also described.

[0004] Atiya et al.'s US2019 / 0388194 (assigned to the assignee of this application and incorporated herein by reference) describes a handheld stick including a probe at the distal end of an elongated handheld stick. The probe includes a light projector and a light field camera. The light projector includes a light source and a pattern generator configured to generate a light pattern. The light field camera includes a light field camera sensor. The light field camera sensor includes: (a) an image sensor including a sensor pixel array; and (b) a microlens array disposed in front of the image sensor such that each microlens is disposed above a subarray of the sensor pixel array. Other applications are also described.

[0005] US2020 / 0404243 by Saphier et al. (assigned to the assignee of this application and incorporated herein by reference) describes a method for generating 3D images, comprising driving a structured light projector to project a light pattern onto a 3D surface within an oral cavity, and driving cameras to capture images, each image including at least a portion of the projected pattern, each camera including an array of pixels. A processor compares a series of images captured by each camera and determines which portions of the projected pattern can be tracked in these images. The processor constructs a 3D model of the 3D intraoral surface based at least in part on the comparison of the series of images. Other embodiments are also described.

[0006] PCT Publication WO2018 / 152374 by Ozerov et al. (assigned to the assignee of this application and incorporated herein by reference) describes processing logic that compares first image data and second image data of a dental arch and identifies multiple spatial differences between a first representation of the dental arch in the first image data and a second representation of the dental arch in the second image data. The processing logic determines that the first spatial difference is attributable to scanner inaccuracy and the second spatial difference is attributable to clinical variations of the dental arch. The processing logic generates a third representation of the dental arch that is a modified version of the second representation, wherein the first spatial difference is removed from the third representation, and wherein the third representation includes visual enhancements that highlight the second spatial difference. Other embodiments are also described.

[0007] U.S. Provisional Application 63 / 452,875 by Levy et al. (assigned to the assignee of this application and incorporated herein by reference) describes a technique for selecting images from a plurality of images generated by an intraoral scanner. One method includes: receiving a plurality of images of a dental site generated by an intraoral scanner; identifying a subset of images from the plurality of images that satisfy one or more selection criteria; selecting the subset of images that satisfy the one or more selection criteria; and discarding or ignoring the remaining images from the plurality of images that are not included in the subset of images. Other embodiments are also described. Summary of the Invention

[0008] When considering intraoral scans of a subject's three-dimensional intraoral surface, it can be useful for dentists or dental technicians to visually compare the color of the three-dimensional intraoral surface with the color of the same three-dimensional intraoral surface captured in scans taken at an earlier time point (e.g., several months ago). This visual comparison can help dentists identify changes in color and / or texture that have occurred on the intraoral surface. For example, the comparison may indicate that plaque has accumulated in certain areas or on certain teeth, or that stains have appeared on the teeth, disappeared, or otherwise changed. However, simply viewing the current intraoral scan side-by-side with previous intraoral scans can still be challenging in accurately assessing changes. The inventors have realized that, to help dentists accurately assess changes in color and / or texture that have occurred in a given area of ​​interest on a three-dimensional intraoral surface, the corresponding area of ​​interest from a previous intraoral scan can be overlaid onto the current intraoral scan, allowing dentists to switch between visual changes that have occurred in a given area of ​​interest within the context of the current intraoral scan. Furthermore, showing patients about changes that have occurred on the intraoral surface can be helpful in terms of patient education, treatment acceptance, and treatment adherence.

[0009] Typically, intraoral scanning of a three-dimensional intraoral surface includes a three-dimensional scan of the three-dimensional intraoral surface and a color image scan of the three-dimensional intraoral surface. The color image scan is registered to the three-dimensional scan, allowing the dentist to view a three-dimensional color model of the three-dimensional intraoral surface. For some applications, finding the corresponding region of interest on a previous color image scan and overlaying it at the correct location onto the current three-dimensional scan is based on an association established between the current three-dimensional scan of the three-dimensional intraoral surface and a previous three-dimensional scan of the three-dimensional intraoral surface, each of which is registered to its corresponding color image scan. Therefore, for some applications, the current three-dimensional scan and the previous three-dimensional scan are correlated such that a given region A in the current three-dimensional scan is associated with a corresponding region A' in the previous three-dimensional scan. Based on the association between the current and previous three-dimensional scans, the registration from the current color image scan to the current three-dimensional scan, and the registration from the previous color image scan to the previous three-dimensional scan, for a given region A in the current three-dimensional scan, (i) the portion of the previous color image scan registered to the corresponding region A' in the previous three-dimensional scan is applied to (ii) the given region A in the current three-dimensional scan.

[0010] Therefore, according to some applications of the present invention, a method is provided, the method comprising using at least one computer processor (e.g., at least one computer processor of a computing device): Obtain the current 3D scan of the inner surface of the 3D opening; Acquire a current color image scan of the three-dimensional inner surface of the opening; Acquire a previous three-dimensional scan of the inner surface of the three-dimensional cavity, which was obtained at a time point prior to the current three-dimensional scan; Acquire a previous color image scan of the three-dimensional surface, which was obtained at a time point prior to the current color image scan; Register the current color image scan to the current 3D scan; Register the previous color image scan to the previous 3D scan; Associate the current 3D scan with the previous 3D scan, such that a given region A in the current 3D scan is associated with the corresponding region A' in the previous 3D scan; and Based on the association between the current 3D scan and the previous 3D scan, the registration from the current color image scan to the current 3D scan, and the registration from the previous color image scan to the previous 3D scan, for a given region A of the current 3D scan, (i) the portion of the previous color image scan that was registered to the corresponding region A' of the previous 3D scan is applied to (ii) the given region A of the current 3D scan.

[0011] For some applications: Registering the current color image scan to the current 3D scan involves associating points u and v in the current color image scan with the corresponding coordinate vertices x, y, and z in the current 3D scan. Registering a previous color image scan to a previous 3D scan involves associating points u', v' in the previous color image scan with the corresponding coordinate vertices x', y', z' in the previous 3D scan. Associating the current 3D scan with the previous 3D scan includes associating the coordinate vertices x, y, z in the current 3D scan with the corresponding coordinate vertices x', y', z' in the previous 3D scan; Applying (i) the portion of the previous color image scan associated with the corresponding region A' of the previous 3D scan to (ii) the given region A of the current 3D scan includes: For the given region A: (a) The set of points u' and v' associated with the corresponding set of coordinate vertices x', y', and z' in the previous color image scan and the previous 3D scan. Applied to (b) The set of corresponding coordinate vertices x', y', z' associated with the set of corresponding coordinate vertices x', y', z' in the current 3D scan, wherein the set of coordinate vertices x', y', z' defines a given region A' of the previous 3D scan, and the set of coordinate vertices x', y', z' in the current 3D scan defines a given region A of the current 3D scan.

[0012] For some applications, this method also includes switching between the following two: (A) Apply the portion of (i) the corresponding region A' in the previous color image scan that corresponds to the previous 3D scan to (ii) the given region A in the current 3D scan, and... (B) Apply (i) the portion of the current color image scan that is associated with a given region A of the current 3D scan to (ii) the given region A of the current 3D scan.

[0013] For some applications, the method further includes: using at least one computer processor, while the user viewing the current 3D scan manipulates the 3D view of the current 3D scan, maintaining the application of (i) a portion of a previous color image scan associated with a corresponding region A' of the previous 3D scan to (ii) a given region A of the current 3D scan. The computer processor may be a processor of a computing device such as a desktop computer, laptop computer, or server computer.

[0014] For some applications, maintaining the application of (i) the portion of the previous color image scan associated with the corresponding region A' of the previous 3D scan to (ii) the given region A of the current 3D scan includes: while the user viewing the current 3D scan rotates the 3D view of the current 3D scan, maintaining the application of (i) the portion of the previous color image scan associated with the corresponding region A' of the previous 3D scan to (ii) the given region A of the current 3D scan.

[0015] For some applications, maintaining the application of (i) the portion of the previous color image scan associated with the corresponding region A' of the previous 3D scan to (ii) the given region A of the current 3D scan includes: while the user is zooming in or out of the 3D view of the current 3D scan, maintaining the application of (i) the portion of the previous color image scan associated with the corresponding region A' of the previous 3D scan to (ii) the given region A of the current 3D scan.

[0016] For some applications, acquiring the current color image scan includes acquiring the current color image scan of the three-dimensional intraoral surface captured in the same intraoral procedure as the current three-dimensional scan.

[0017] For some applications, acquiring a previous color image scan includes acquiring a previous color image scan of the three-dimensional intraoral surface captured in the same intraoral process as capturing a previous three-dimensional scan.

[0018] For some applications, the method further includes: using at least one computer processor, in response to a user selecting a specific region of the current 3D scan, designating the specific region of the current 3D scan as a given region A.

[0019] For some applications, designating a specific region of the current 3D scan as a given region A includes: in response to a user selecting a specific feature of the intraoral surface, designating that specific feature as a given region A.

[0020] For some applications, designating a specific feature of the intraoral surface as a given region A includes: designating the given tooth as the given region A in response to a user selecting a given tooth.

[0021] For some applications, the method further includes: using at least one computer processor, in response to a user moving a region selection indicator on a 3D view of the current 3D scan, dynamically changing a given region A such that, as the user moves the region selection indicator, for each new given region A within the region selection indicator, it is displayed that (i) a portion of the previous color image scan associated with the corresponding region A' of the previous 3D scan is applied to (ii) the application of the given region A of the current 3D scan.

[0022] For some applications, the method also includes, using at least one computer processor, determining parameters for the association between (i) the current color image scan of the three-dimensional intraoral surface and (ii) the previous color image scan of the three-dimensional intraoral surface, based on the association between the current three-dimensional scan and the previous three-dimensional scan, such that points u, v in the current color image scan are associated with corresponding points u', v' in the previous color image scan.

[0023] For some applications, the parameters for determining the association include: (a) using a neural network to find features in the current color image scan that match corresponding features in a previous color image scan, and (b) using an optical flow algorithm to compute a transformation on the matched features that, when applied to the previous color image scan, shifts points u' and v' in the previous color image scan such that the positions of points u' and v' in the previous color image scan are the same as the positions of their corresponding points u and v in the current color image scan.

[0024] For some applications, determining the associated parameters includes: (a) using the Scale Invariant Feature Transform (SIFT) algorithm to find discrete features in the current color image scan that match corresponding discrete features in a previous color image scan; (b) calculating a transformation that, when applied to discrete features in a previous color image scan, shifts the discrete features such that the positions of the discrete features in the previous color image scan are the same as the positions of the matching discrete features in the current color image scan; and (c) interpolating the transformation for regions between discrete features in a previous color image scan.

[0025] For some applications, the parameters used to determine the association include: (a) Encode points u and v in the current color image scan and points u' and v' in the previous color image scan using a neural network, such that each of points u and v and each of points u' and v' is described by a vector, and the vectors of points u' and v' in the previous color image scan are matched with the vectors of the corresponding associated points u and v in the current color image scan, and (b) For a given point u, v in the current color image scan, a neural network is used to locate the associated point u', v' in the previous color image scan by searching for points u', v' in the previous color image scan whose vectors match the vectors of the given points u, v.

[0026] For some applications, using neural networks to locate associated points u' and v' in previous color image scans also includes determining the search space in the previous color image scans to search for associated points u' and v' based on the association between the current 3D scan and the previous 3D scan.

[0027] For some applications, the method further includes using at least one computer processor to normalize the previous color image scan relative to the current color image scan, such that points u, v in the current color image scan that correspond to regions of the intraoral surface whose color changes between the previous and current color image scans are emphasized relative to their corresponding points u', v' in the previous color image scan.

[0028] For some applications, normalizing a previous color image scan relative to a current color image scan involves encoding the previous color image scan and the current color image using a neural network, and using a neural style transfer algorithm to output the content of the previous color image scan using the style of the current color image scan.

[0029] For some applications, normalizing a previous color image scan relative to the current color image scan involves using a Poisson fusion algorithm to output a normalized version of the previous color image scan, in which (a) the pixel values ​​of points u' and v' are substantially the same as the pixel values ​​of the corresponding associated points u and v in the current color image scan, and (b) the gradient between adjacent points u' and v' remains substantially the same as the gradient in the previous color image scan before normalization.

[0030] For some applications, the method further includes: using at least one computer processor to analyze the current color image scan and the normalized previous color image scan, and to detect regions where there are differences between points u and v in the current color image scan and their corresponding points u' and v' in the normalized previous color image scan.

[0031] For some applications, the method further includes: using at least one computer processor, designating a specific region of the current 3D scan as a given region A based on the registration from the current color image scan to the current 3D scan, the specific region corresponding to the region where there is a difference between points u and v in the current color image scan and their corresponding points u' and v' in the normalized previous color image scan.

[0032] For some applications, the method also includes: using at least one computer processor to visually indicate a specific area of ​​the current 3D scan to a user viewing the current 3D scan.

[0033] The invention will be more fully understood through the following detailed description of its application in conjunction with the accompanying drawings, in which: Attached Figure Description

[0034] Figure 1 This is a flowchart depicting a method for applying a portion of a previous color image scan of a three-dimensional intraoral surface to a current three-dimensional scan of the three-dimensional intraoral surface, according to some applications of the present invention. Figure 2A A current three-dimensional scan of a three-dimensional intraoral surface is depicted according to some applications of the present invention, wherein a current color image scan is applied to the current three-dimensional scan; Figure 2B Depicting some applications according to the present invention Figure 2A The current 3D scan, in which a portion of a previous color image scan is applied to the current 3D scan; Figure 3A A current three-dimensional scan of a three-dimensional intraoral surface is depicted according to some applications of the present invention, wherein a current color image scan is applied to the current three-dimensional scan; Figure 3B Depicting some applications according to the present invention Figure 3A The current 3D scan, in which a portion of the previous color image scan is applied to the current 3D scan; Figures 4A to 4B The visualizations depicting the encoding of points in a color image scan using a neural network for current and previous color image scans, respectively, according to some applications of the present invention; and Figures 5A to 5D Various viewing options for current 3D scans are described in some applications according to the present invention.

[0035] Figure 6 This is a block diagram illustrating a computer system according to some embodiments. Detailed Implementation

[0036] Now for reference Figure 1This is a flowchart illustrating a method 20 for applying a portion of a previous color image scan of a three-dimensional intraoral surface to a current three-dimensional scan of the three-dimensional intraoral surface according to some applications of the invention. The method 20 is performed using at least one computer processor 21 (e.g., at least one computer processor of a computing device). For some applications, the method includes: acquiring a current three-dimensional scan of the three-dimensional intraoral surface (step 23), and acquiring a current color image scan of the three-dimensional intraoral surface (step 22), for example, a current color image scan of the three-dimensional intraoral surface captured during the same intraoral process as the current three-dimensional scan. The method also includes: acquiring a previous three-dimensional scan of the three-dimensional intraoral surface (step 24), which was acquired at a time point prior to the current three-dimensional scan; and acquiring a previous color image scan of the three-dimensional surface (step 26), which was acquired at a time point prior to the current color image scan, for example, a previous color image scan of the three-dimensional intraoral surface captured during the same intraoral process as the previous three-dimensional scan. For example, the same patient may have had an intraoral scan taken at an earlier time point (e.g., several months before the current scan) (referred to as the previous scan in this article) and a subsequent intraoral scan taken later (referred to as the current scan in this article).

[0037] Referring to current and previous color image scans, some applications are described. These current and previous color image scans can be two-dimensional (2D) or three-dimensional (3D) color images of a three-dimensional (3D) intraoral surface, or a collection of 2D or 3D color images. Color image scans can include color images generated under white light illumination, color images generated using fluorescence imaging, and / or color images generated under other illumination conditions. For example, one type of color image scan is a fluorescence imaging scan, in which a current fluorescence imaging scan can be registered and compared with a previous fluorescence imaging scan to determine the differences between them. For fluorescence imaging, an intraoral scanner can emit light at one or more specific frequencies. Light at one or more specific frequencies can cause the illuminated tooth to fluoresce. For example, areas of a tooth with cavities, tartar, etc., may exhibit different fluorescence compared to healthy areas of the tooth. The fluorescence of the tooth may have different colors, which can be captured in the color image.

[0038] It should be understood that the applications described in the reference color image scanning description are also applicable to other types of 2D images, such as near-infrared (NIR) 2D images, 2D images generated under specific lighting conditions, and so on. For example, some applications described in the reference color image scanning description are also applicable to NIR image scanning, where the current NIR image scan can be registered and compared with a previous NIR image scan. For example, based on such a comparison, the differences in intensity and / or brightness of various parts of the 3D intraorific surface between the current time and a previous time can be determined.

[0039] For certain applications, the techniques described in the following documents can be used to obtain the 3D scanning and color image scanning described herein: US2019 / 0388193 by Saphier et al., US2019 / 0388194 by Atiya et al., and / or US2020 / 0404243 by Saphier et al., each of which has been assigned to the assignee of this application and is incorporated herein by reference.

[0040] In step 28, the current color image scan is registered to the current 3D scan, and in step 30, the previous color image scan is registered to the previous 3D scan. Typically, a 3D scan produces a list of coordinate vertices corresponding to 3D points in space on a 3D surface, while a color image scan produces a 2D image composed of points u, v (e.g., pixels). For some applications, registering the current color image scan to the current 3D scan involves associating points u, v in the current color image scan with corresponding coordinate vertices x, y, z in the current 3D scan. Similarly, registering a previous color image scan to the previous 3D scan involves associating points u', v' in the previous color image scan with corresponding coordinate vertices x', y', z' in the previous 3D scan.

[0041] In step 32, the current 3D scan is associated with the previous 3D scan, such that a given region A in the current 3D scan is associated with the corresponding region A' in the previous 3D scan. For some applications, the current 3D scan includes a current virtual 3D model of the 3D intraoral surface generated by registering and stitching together multiple current 3D point clouds, and the previous 3D scan includes a previous virtual 3D model of the intraoral surface generated by registering and stitching together multiple previous 3D point clouds. For some applications, the current 3D scan includes the current 3D point cloud of the 3D intraoral surface, and the previous 3D scan includes the previous 3D point cloud of the intraoral surface. For some applications, associating the current 3D scan with the previous 3D scan includes associating the coordinate vertices x, y, z in the current 3D scan with the corresponding coordinate vertices x', y', z' in the previous 3D scan, such that the output is a list of local vertex registrations between the coordinate vertices x, y, z in the current 3D scan and the corresponding coordinate vertices x', y', z' in the previous 3D scan. For some applications, the association is interpolated to obtain the complete association between the coordinate vertices of the current 3D scan and the coordinate vertices of the previous 3D scan.

[0042] In step 34, based on the association between the current 3D scan and the previous 3D scan, the registration from the current color image scan to the current 3D scan, and the registration from the previous color image scan to the previous 3D scan, for a given region A in the current 3D scan, the colors from the previous color image scan are overlaid and displayed at the corresponding position in the current 3D scan in the following manner: (i) the portion of the previous color image scan registered to the corresponding region A' of the previous 3D scan is applied to (ii) the given region A in the current 3D scan. For example, given region A is defined as the set of coordinate vertices x, y, z, and based on the association between the current 3D scan and the previous 3D scan, given region A' is defined as the set of corresponding coordinate vertices x', y', z'. Therefore, in step 34, (a) the set of points u', v' of the set of coordinate vertices x', y', z' that define the given region A' in the previous color image scan (based on the registration in step 30) is applied to (b) the set of coordinate vertices x, y, z that define the given region A in the current 3D scan.

[0043] Now for reference Figures 2A to 2B They respectively depict some applications according to the invention: (a) a current three-dimensional scan 36 of the three-dimensional inner surface of the mouth, wherein a current color image scan is applied to the current three-dimensional scan 36, and (b) a current three-dimensional scan 36, wherein a portion of a previous color image scan is applied to the current three-dimensional scan 36. Figure 2AThe diagram shows a given region A in the current 3D scan 36, where the portion of the current color image scan associated with region A is applied to region A. That is, the set of coordinate vertices x, y, z in the current color image scan that defines region A is associated (based on...). Figure 1 The set of points u and v (registration in step 28) is applied to the given region A.

[0044] exist Figure 2B In China, use Figure 1 Step 34: The portion of the previous color image scan associated with a given region A' of the previous 3D scan is applied to the given region A of the current 3D scan 36. That is, the set of coordinate vertices x', y', z' of the previous color image scan that define the given region A' of the previous 3D scan is associated (based on...). Figure 1 The set of points u' and v' from the registration in step 30 is applied to a given region A of the current 3D scan 36. The result is that the user continues to see the current 3D scan 36 on the screen, but the color of a specific portion of the 3D inner surface is displayed in the given region A, as those colors were presented in the previous color image scan. Figure 2B As shown, the color applied to region A of the current 3D scan 36 is different from the color applied to... Figure 2A The color of region A is different. In this particular example, when the color from a previous color image scan is applied to a given region A, the dentist or dental technician can see that there was no plaque in the given region A at the time the previous color image scan was obtained, indicating that plaque has accumulated on the patient's teeth in that region since the time the previous color image scan was obtained.

[0045] Now for reference Figures 3A to 3B They respectively depict some applications according to the invention: (a) a current three-dimensional scan 36 of the three-dimensional inner surface of the mouth, wherein a current color image scan is applied to the current three-dimensional scan 36, and (b) a current three-dimensional scan 36, wherein a portion of a previous color image scan is applied to the current three-dimensional scan 36. Figures 3A to 3B Another example is shown where, when color from a previous color image scan is applied to a given area A, a dentist or dental technician can see the stains that have formed on the teeth since the previous color image scan was obtained. The current color image scan applied to the given area A is shown in the example. Figure 3A In the image, stains on the teeth are visible in a given region A, while in the portion shown in the previous color image scan applied to a given region A... Figure 3B There were no stains.

[0046] For some applications, step 34 also includes switching between: (A) applying the portion of the previous color image scan associated with the corresponding region A' of the previous 3D scan to the given region A of the current 3D scan (i.e., by way of example in...). Figure 2B and Figure 3B (A) and (B) shown apply the portion of the current color image scan associated with a given region A of the current 3D scan to the given region A of the current 3D scan (i.e., by way of example in...). Figure 2A and Figure 3A (As shown in the diagram). The effect of this switching is that the dentist or dental technician can view the current 3D scan of the patient's intraoral surface and, for a given area of ​​the 3D scan, switch the color of the 3D intraoral surface between (i) the color obtained when the previous color image scan was taken and (ii) the current color. It should be noted that the switching is performed for a given area, while another area of ​​the current 3D scan remains under the current color image scan.

[0047] For some applications, step 34 also includes designating the specific region of the current 3D scan as a given region A in response to the user selecting a specific region of the current 3D scan. For example, the user can utilize region selection indicator 38 (e.g., Figures 2A to 2B The virtual magnifying glass shown in the image or Figures 3A to 3B The scan area is marked by parentheses shown in the figure. Processor 21 performs step 34 by first limiting the area within the area selection indicator to the given area A of the current 3D scan.

[0048] For some applications, the user can dynamically move the region selection indicator 38 around in the 3D view of the current 3D scan 36, and step 34 includes dynamically changing a given region A in response to the user moving the region selection indicator 38. As the user moves the region selection indicator 38, for each new given region A within the region selection indicator 38, the processor 21 displays (i) the portion of the previous color image scan associated with the corresponding region A' of the previous 3D scan to (ii) the application of the given region A of the current 3D scan 36. For some applications, if the user holds the region selection indicator 38 stably in a specific region of the current 3D scan 36, the processor 21 can switch the color view between the previous color image scan and the current color image scan for regions within the region selection indicator, as described above. Typically, switching is performed only in the region selected by the region selection indicator 38, while another region of the current 3D scan that the user is viewing (e.g., the remainder of the current 3D scan) remains as if the current color image scan has been applied.

[0049] For some applications, processor 21 can calculate the application of a previous color image to the current 3D scan over the entire intraoral surface; that is, by executing step 34, and then, based on the user's selection of a given region A using region selection indicator 38, displaying the application of a portion of the previous color image scan to the given region A of the current 3D scan. Alternatively, processor 21 can, in response to the user's selection of a given region A using region selection indicator 38, calculate in real time the application of a specific portion of the previous color image scan corresponding to region A' to the given region A of the current 3D scan.

[0050] For some applications, the user can select a specific feature of the intraoral surface, and the processor 21, in response to the user's selection of a specific feature, designates the selected specific feature of the intraoral surface as a given region A. For example, the user can select a given tooth, and the processor 21 designates the given region A of the current 3D scan 36 as that given tooth and displays the application of the portion of the given tooth from the previous color image scan to the given region A of the current 3D scan 36.

[0051] For some applications, the advantage of using digital 3D scanning is that the user viewing the 3D scan can manipulate the 3D view of the current 3D scan to see the intraoral surface from different perspectives. If the user is viewing a portion of a given region A in a previous color image scan that was applied to the current 3D scan, it is advantageous to maintain the colors of the earlier scan in the correct position on the current 3D scan while the user manipulates the 3D view. Therefore, for some applications, step 34 also includes maintaining (i) the portion of the previous color image scan associated with the corresponding region A' of the previous 3D scan applied to (ii) the given region A of the current 3D scan while the user viewing the current 3D scan manipulates the 3D view of the current 3D scan. For example, for some applications, maintaining (i) the portion of the previous color image scan associated with the corresponding region A' of the previous 3D scan applied to (ii) the given region A of the current 3D scan while the user viewing the current 3D scan (a) rotates the 3D view of the current 3D scan and / or (b) zooms in or out of the 3D view of the current 3D scan.

[0052] Now refer to it again Figure 1The following situations may arise: differences between the current scan set (3D and color images) and the previous scan set (3D and color images) can pose challenges in applying a portion of the previous color image scan to the current 3D scan. For example, if the previous and current color image scans are obtained at different exposure levels, an observer may see color variations that are not actually caused by changes occurring on the intraoral surface; or, if teeth have moved between the previous and current scan sets, correctly applying the color from the previous scan to the correct tooth in the current 3D scan can be challenging. The inventors have realized that finding the correlation between two color image scans enables the accurate application of a portion of the previous color image scan to a given region A in the current 3D scan. Therefore, for certain applications, Figure 1 The method further includes an additional step 40, in which, based on the association between the current three-dimensional scan and the previous three-dimensional scan (step 32), the processor 21 determines parameters relating (i) the current color image scan of the three-dimensional intraorifice surface to (ii) the previous color image scan of the three-dimensional intraorifice surface, such that points u and v in the current color image scan are associated with corresponding points u' and v' in the previous color image scan.

[0053] For some applications, parameters for determining the association between a previous color image scan and a current color image scan may include: (a) using a neural network to find features in the current color image scan that match corresponding features (e.g., points u', v') in the previous color image scan; and (b) using an optical flow algorithm (e.g., the Horn-Schunck method) to compute a transformation on the matching features that, when applied to the previous color image scan, shifts the points u', v' in the previous color image scan such that the positions of points u', v' in the previous color image scan are the same as the positions of their corresponding points u, v in the current color image scan. This allows the processor 21 to determine whether portions of the previous color image scan have shifted relative to their corresponding portions in the current color image scan. For example, if a patient's tooth has shifted between the previous and current scan sets, the points u', v' identified as points on that tooth in the previous color image scan will be in different positions in the previous color image scan compared to their corresponding points u, v identified as points on the same tooth in the current color image scan.

[0054] Alternatively or additionally, for some applications, the parameters relating the previous and current color image scans can be determined by finding matching discrete features (e.g., discrete features on the intraoral surface) in the two color image scans. For example, the Scale Invariant Feature Transform (SIFT) algorithm can be used to find discrete features in the current color image scan that match the corresponding discrete features in the previous color image scan. Using the matching discrete features, a transform is computed that, when applied to the discrete features in the previous color image scan, shifts the discrete features so that the position of the discrete features in the previous color image scan is the same as the position of its matching discrete features in the current color image scan. For teeth that have shifted between the previous and current scan sets, this transform can help apply portions of the previous color image scan to the current 3D scan. For some applications, the transform is interpolated for regions between discrete features in the previous color image scan, such as using RBF interpolation or B-spline interpolation.

[0055] For further reference Figures 4A to 4B The diagrams depict visualizations of how points in a color image scan are encoded using a neural network for a current color image scan and a previous color image scan, according to some applications of the invention. For some applications, parameters relating the previous color image scan to the current color image scan can be determined by using a neural network to register the previous color image scan to the current color image scan. Therefore, for some applications, step 40 includes: (a) encoding points u, v in the current color image scan and points u', v' in the previous color image scan using a neural network, such that each of points u, v and each of points u', v' is described by a vector.

[0056] exist Figures 4A to 4B In the diagram, the vectors of points u, v and u', v' are represented by different shades of boxes, with each box representing a different shade. Figure 4A In the 'u' symbol, points u and v (e.g., pixels u and v) are represented. Figure 4B The vector represents points u' and v'. Typically, the given points u' and v' that are correlated in a previous color image scan and the current color image scan are encoded by a neural network using matched vectors. Figures 4A to 4B In the example, Figure 4A The box 42, shown in the lightest shade (representing the current color image scan), is encoded using vector V. Figure 4BThe box 44 (representing a previous color image scan), shown in the same lightest shade, is encoded with the same vector and labeled V'. Matching vectors can be used to find associated points; that is, for a given point u, v in the current color image scan, a neural network is used to locate associated points u', v' in the previous color image scan by searching for points u', v' in the previous color image scan whose vector V' matches the vector V of the given point u, v. Figures 4A to 4B In the example, it is shown that the points u' and v' associated with points u and v have been shifted and did not appear in the same positions in the previous color image scan as points u and v appear in the current color image scan. Once the neural network finds a matching pair of points, a transformation can be applied to the previous color image scan to shift points u' and v' so that points u' and v' can be correctly applied to the current 3D scan.

[0057] For some applications, the association between the current 3D scan and the previous 3D scan in step 32 can be used to obtain an initial guess of where the neural network should search for points u' and v' in the previous color image scan that have the same vector as the given points u and v in the current color image scan. Therefore, for some applications, based on the association between the current 3D scan and the previous 3D scan, a search space 46 is determined in the previous color image scan in which to search for associated points u' and v'. Figure 4B (Represented by dashed circles in the diagram). For example, the list of local vertex registrations between the coordinate vertices x, y, z in the current 3D scan and the corresponding coordinate vertices x', y', z' in the previous 3D scan, output by step 32, can be used to generate a first estimate for the neural network to use when attempting to locate the associated points u', v' of a given point u, v.

[0058] As mentioned above, color differences may exist between previous and current color image scans. These differences are not due to changes in the intraoral surface over time, but rather to factors related to the scan, such as exposure conditions during the color image scan. Therefore, for certain applications, to help dentists or dental technicians identify changes occurring in the actual intraoral surface when applying a portion of a previous color image scan to the current 3D scan, processor 21 can normalize the previous color image scan relative to the current color image scan. Figure 1 (Step 41 in the previous color image scan). Thus, points u and v in the current color image scan corresponding to regions on the intraoral surface whose color changed between the previous and current color image scans are highlighted relative to their corresponding points u' and v' in the previous color image scan. It should be noted that for some applications, step 41 may be performed before step 40.

[0059] Typically, after normalization, areas of the oral surface whose color has not actually changed in the current color scan appear identical to their corresponding areas in a previous color scan. This allows for a seamless color transition between the previous color scan and the current color scan when applying portions of the previous color scan to the current 3D scan, and enables dentists or dental technicians to more accurately identify where changes have occurred on the oral surface. See below for more details. Figures 5A to 5C Furthermore, the normalization process also helps the processor 21 detect changes that have occurred on the surface of the port through image processing.

[0060] For some applications, the previous color image scan can be normalized relative to the current color image by using a neural network to encode the previous color image scan and the current color image, and using a neural style transfer algorithm to output the content of the previous color image scan using the style of the current color image scan.

[0061] Alternatively or additionally, for certain applications, a Poisson fusion algorithm can be used to normalize a previous color image scan relative to the current color image scan, outputting a normalized version of the previous color image scan in which (a) the pixel values ​​of points u' and v' are substantially the same as the pixel values ​​of the corresponding associated points u and v in the current color image scan, and (b) the gradient between adjacent points u' and v' is substantially the same as the gradient in the previous color image scan before normalization. That is, the image features in the previous color image scan remain unchanged, but are displayed using the palette of the current color image scan. For certain applications, processor 21 can perform Poisson fusion using the following equation:

[0062] Where f is the weighting function (e.g., a Gaussian function), φ is the desired solution, psi_0 is the source color map, and psi_1 is the target color map. It's convolution.

[0063] Now for reference Figures 5A to 5D This describes various viewing options for the current 3D scan 47 according to some applications of the invention. For some applications, once the previous color image scan is normalized relative to the current color image scan, step 34 may further include the processor 21 using image processing, for example, to analyze the current color image scan and the normalized previous color image scan, and to detect areas where there are differences between points u, v in the current color image scan and their corresponding points u', v' in the normalized previous color image scan. Based on the detected color differences between the current color image scan and the normalized previous color image scan, the processor 21 can determine points of interest on the intraoral surface where changes may have occurred.

[0064] Therefore, for some applications, based on the registration from the current color image scan to the current 3D scan, the processor 21 can designate a specific region of the current 3D scan 47 as a given region A, which corresponds to the region where there is a difference between points u and v in the current color image scan and their corresponding points u' and v' in the normalized previous color image scan. For some applications, such as Figure 5A As shown, processor 21 can visually indicate to the user, by displaying region indicators (such as brackets 48) on the 3D view of the current 3D scan 47, that a specific region of the current 3D scan 47 is a region on the intraoral surface where changes may have occurred. For some applications, such as... Figure 5B As shown, the processor 21 can automatically apply the corresponding portion of the previous color image scan to a specific area of ​​the current 3D scan 47, and visually indicate to the user, for example, using brackets 48, that the specific area of ​​the current 3D scan 47 shows the color of the previous color image scan.

[0065] Alternatively or additionally, the processor 21 may visually indicate to the user, for example, using brackets 48, a specific area of ​​the current 3D scan 47 in which changes may have occurred on the intraoral surface, and the user (e.g., a dentist or dental technician) may choose to move the area selection indicator 38 (as described above) to the specific area indicated by brackets 48, in response to which the processor 21 applies the corresponding portion of the previous color image scan to that specific area of ​​the current 3D scan 47.

[0066] It should be noted that, for some applications, the application of a portion of the previous color image scan to a given region A of the current 3D scan can be performed without associating the previous color image scan with the previous 3D scan. That is, step 34 can be performed without a previous 3D scan, for example, by using a neural network (as described above) to associate the previous color image scan with the current color image scan.

[0067] The application of the disclosure described herein may take the form of a computer program product accessible from a computer-usable or computer-readable medium (e.g., a non-transitory computer-readable medium) providing program code that can be used or combined with a computer or any instruction execution system (such as the processing apparatus disclosed above). For the purposes of this specification, a computer-usable or computer-readable medium can be any device capable of including, storing, communicating, propagating, or transmitting a program for use or combined with an instruction execution system, device, or apparatus. The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or device or apparatus), or a propagation medium. Typically, a computer-usable or computer-readable medium is a non-transitory computer-usable or computer-readable medium.

[0068] Examples of computer-readable media include semiconductor or solid-state memory, magnetic tape, removable computer floppy disks, random access memory (RAM), read-only memory (ROM), rigid disks, and optical discs. Current examples of optical discs include read-only optical disc drives (CD-ROM), read-write optical discs (CD-R / W), and DVDs. For some applications, cloud storage and / or storage on remote servers are used.

[0069] A data processing system suitable for storing and / or executing program code will include at least one processor (e.g., processor 21 disclosed above) directly or indirectly coupled to memory elements via a system bus. The memory elements may include local memory, mass storage, and caches used during actual execution of the program code, providing temporary storage for at least some of the program code to reduce the number of times code must be retrieved from mass storage during execution. The system can read inventive instructions on a program storage device and follow those instructions to perform the methods of embodiments of this disclosure.

[0070] A network adapter can be coupled to a processor, enabling the processor to be coupled to other processors, remote printers, or storage devices via an intermediate private or public network. Modems, cable modems, and Ethernet cards are just a few of the types of network adapters currently available.

[0071] Computer program code used to perform the operations of this disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Java, Smalltalk, C++, Python, etc.) and conventional procedural programming languages ​​(such as C programming language or similar programming languages).

[0072] Figure 6The illustration shows a machine in an example form of computing device 600, in which an instruction set can be executed to cause the machine to perform any or more methods discussed herein. In alternative embodiments, the machine may be connected (e.g., networked) to other machines on a local area network (LAN), intranet, extranet, or the Internet. For example, computing device 600 may correspond to... Figure 1 The computer processor 21. The machine can operate as a server or client machine in a client-server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine can be a personal computer (PC), tablet computer, set-top box (STB), personal digital assistant (PDA), cellular phone, network device, server, network router, switch, or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) specifying the actions to be taken by the machine. Furthermore, although only a single machine is shown, the term "machine" should also be considered to include any collection of machines (e.g., computers) that individually or jointly execute one or more sets of instructions to perform any one or more methods discussed herein.

[0073] Example computing device 600 includes processing device 602, main memory 604 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM), etc.), static memory 606 (e.g., flash memory, static random access memory (SRAM), etc.), and auxiliary memory (e.g., data storage device 628), which communicate with each other via bus 608.

[0074] Processing device 602 represents one or more general-purpose processors, such as microprocessors, central processing units, etc. More specifically, processing device 602 may be a Complex Instruction Set Computing (CISC) microprocessor, a Reduced Instruction Set Computing (RISC) microprocessor, a Very Long Instruction Word (VLIW) microprocessor, a processor implementing other instruction sets, or a processor implementing combinations of instruction sets. Processing device 602 may also be one or more special-purpose processing devices, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), network processors, etc. Processing device 602 is configured to execute processing logic (instructions 626) for performing the operations and steps discussed herein.

[0075] The computing device 600 may also include a network interface device 622 for communicating with the network 664. The computing device 600 may also include a video display unit 610 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device 612 (e.g., a keyboard), a cursor control device 614 (e.g., a mouse), and a signal generation device 620 (e.g., a speaker).

[0076] Data storage device 628 may include a machine-readable storage medium (or more specifically, a non-transitory computer-readable storage medium) 624 on which a set of one or more instructions 626 is stored, the instructions 626 (such as instructions for an intraoral scanning application 616) embodying any one or more methods or functions described herein. Non-transitory storage medium refers to storage media other than a carrier wave. Instructions 626 may also reside wholly or at least partially within main memory 604 and / or processing device 602 during execution by computer device 600, which also constitute computer-readable storage media.

[0077] Computer-readable storage medium 624 can also be used to store an intraoral scanning application 626 that can perform the operations described herein. Computer-readable storage medium 624 can also store a software library including methods for the intraoral scanning application 626. Although computer-readable storage medium 624 is shown as a single medium in the example embodiment, the term "computer-readable storage medium" should be considered to include a single medium or multiple media (e.g., a centralized or distributed database and / or associated caches and servers) storing one or more sets of instructions. The term "computer-readable storage medium" should also be considered to include any medium excluding a carrier wave capable of storing or encoding a set of instructions for machine execution and enabling a machine to perform any one or more methods of this disclosure. Therefore, the term "computer-readable storage medium" should be considered to include, but is not limited to, solid-state memory and optical and magnetic media.

[0078] It will be understood that the methods described herein can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions, which execute via the processor of the computer (e.g., processor 21 disclosed above) or the other programmable data processing apparatus, create means for implementing the functions / actions specified in the methods described herein. These computer program instructions can also be stored in a computer-readable medium (e.g., a non-transitory computer-readable medium) that can instruct a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable medium produce an article of writing comprising instruction means for implementing the functions / actions specified in the methods described herein. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus, thereby producing a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide a process for implementing the functions / actions specified in the methods described herein.

[0079] The processing apparatus disclosed above is typically a hardware device programmed with computer program instructions to generate a dedicated computer. For example, when programmed to perform the methods described herein, the processing apparatus typically acts as a dedicated processing apparatus. Generally, depending on the technology of the memory used, the operations performed by the computer processor described herein will transform the physical state of the memory, which is a real object, into having different magnetic polarities, charges, etc.

[0080] It should be noted that the techniques described herein may be used in conjunction with those described in the following documents: US2019 / 0388193 by Saphier et al., published December 26, 2019; US2019 / 0388194 by Atiya et al., published December 26, 2019; US2020 / 0404243 by Saphier et al., published December 4, 2020; PCT publication WO2018 / 152374 by Ozerov et al., published August 23, 2018; and US application 18 / 605,783 by Levy et al., filed March 14, 2024, each of which is incorporated herein by reference.

[0081] Those skilled in the art will recognize that the present invention is not limited to the contents specifically shown and described above. Rather, the scope of the invention includes combinations and sub-combinations of the various features described above, as well as variations and modifications not found in the prior art, which would be apparent to those skilled in the art upon reading the foregoing description.

Claims

1. A method comprising using at least one computer processor: Obtain the current 3D scan of the inner surface of the 3D opening; Obtain the current color image scan of the three-dimensional inner surface of the mouth; Obtain a previous three-dimensional scan of the inner surface of the three-dimensional cavity, the previous three-dimensional scan being obtained at a time point prior to the current three-dimensional scan; Acquire a previous color image scan of the three-dimensional surface, the previous color image scan being obtained at a time point prior to the current color image scan; Register the current color image to the current 3D scan; Register the previous color image scan to the previous 3D scan; The current 3D scan is associated with the previous 3D scan, such that a given region A of the current 3D scan is associated with the corresponding region A' of the previous 3D scan; as well as Based on the association between the current 3D scan and the previous 3D scan, the registration of the current color image scan to the current 3D scan, and the registration of the previous color image scan to the previous 3D scan, for a given region A of the current 3D scan, (i) the portion of the previous color image scan registered to the corresponding region A' of the previous 3D scan is applied to (ii) the given region A of the current 3D scan.

2. The method according to claim 1, wherein: Registering the current color image scan to the current 3D scan includes associating points u and v in the current color image scan with the corresponding coordinate vertices x, y, and z of the current 3D scan; Registering the previous color image scan to the previous 3D scan includes associating points u', v' in the previous color image scan with the corresponding coordinate vertices x', y', z' in the previous 3D scan; Associating the current 3D scan with the previous 3D scan includes associating the coordinate vertices x, y, z in the current 3D scan with the corresponding coordinate vertices x', y', z' in the previous 3D scan; Applying (i) the portion of the previous color image scan associated with the corresponding region A' of the previous 3D scan to (ii) the given region A of the current 3D scan includes: for the given region A: The set of points u', v' associated with the corresponding set of coordinate vertices x', y', z' in the previous color image scan (a) is applied to the set of corresponding coordinate vertices x', y', z' in the current 3D scan associated with the corresponding set of coordinate vertices x', y', z' in the previous 3D scan, wherein the set of coordinate vertices x', y', z' defines a given region A' of the previous 3D scan, and the set of coordinate vertices x', y', z' in the current 3D scan defines a given region A of the current 3D scan.

3. The method according to claim 1, wherein, The method also includes switching between the following two: (A) Apply the portion of (i) the previous color image scan associated with the corresponding region A' of the previous 3D scan to (ii) the given region A of the current 3D scan, and... (B) Apply the portion of (i) the current color image scan associated with the given region A of the current three-dimensional scan to (ii) the given region A of the current three-dimensional scan.

4. The method according to claim 1, wherein, The method further includes: using the at least one computer processor, while viewing a user-manipulated 3D view of the current 3D scan, maintaining (i) the portion of the previous color image scan associated with the corresponding region A' of the previous 3D scan applied to (ii) the given region A of the current 3D scan.

5. The method according to claim 4, wherein, Maintaining the application of (i) the portion of the previous color image scan associated with the corresponding region A' of the previous 3D scan to (ii) the given region A of the current 3D scan includes: while the user viewing the current 3D scan rotates the 3D view of the current 3D scan, maintaining the application of (i) the portion of the previous color image scan associated with the corresponding region A' of the previous 3D scan to (ii) the given region A of the current 3D scan.

6. The method according to claim 4, wherein, Maintaining the application of (i) the portion of the previous color image scan associated with the corresponding region A' of the previous 3D scan to (ii) the given region A of the current 3D scan includes: while a user viewing the current 3D scan zooms in or out of the 3D view of the current 3D scan, maintaining the application of (i) the portion of the previous color image scan associated with the corresponding region A' of the previous 3D scan to (ii) the given region A of the current 3D scan.

7. The method according to claim 1, wherein, Acquiring the current color image scan includes acquiring the current color image scan of the three-dimensional intraoral surface captured in the same intraoral process as the current three-dimensional scan.

8. The method according to claim 1, wherein, Acquiring the previous color image scan includes acquiring a previous color image scan of the three-dimensional intraoral surface captured in the same intraoral process as that used to capture the previous three-dimensional scan.

9. The method according to claim 1, wherein, The method further includes: using the at least one computer processor, in response to a user selecting a specific region of the current 3D scan, designating the specific region of the current 3D scan as the given region A.

10. The method according to claim 9, wherein, Designating a specific region of the current 3D scan as the given region A includes: in response to a user selecting a specific feature of the intraoral surface, designating the specific feature as the given region A.

11. The method according to claim 10, wherein, Designating a specific feature of the intraoral surface as the given region A includes: designating the given tooth as the given region A in response to a user selecting a given tooth.

12. The method according to claim 9, wherein, The method further includes: using the at least one computer processor, in response to a user moving a region selection indicator on a three-dimensional view of the current three-dimensional scan, dynamically changing the given region A such that, as the user moves the region selection indicator, for each new given region A within the region selection indicator, displaying that (i) a portion of the previous color image scan associated with the corresponding region A' of the previous three-dimensional scan is applied to (ii) the given region A of the current three-dimensional scan.

13. The method according to claim 1, wherein, The method further includes: based on the association between the current three-dimensional scan and the previous three-dimensional scan, using the at least one computer processor, determining (i) the association parameters between the current color image scan of the three-dimensional intraoral surface and (ii) the previous color image scan of the three-dimensional intraoral surface, such that points u and v in the current color image scan are associated with corresponding points u' and v' in the previous color image scan.

14. The method according to claim 13, wherein, Determining the associated parameters includes: (a) using a neural network to find features in the current color image scan that match corresponding features in the previous color image scan, and (b) using an optical flow algorithm to calculate a transformation on the matched features, the transformation offsetting points u' and v' in the previous color image scan when applied to the previous color image scan, such that the positions of points u' and v' in the previous color image scan are the same as the positions of their corresponding points u and v in the current color image scan.

15. The method according to claim 13, wherein, Determining the associated parameters includes: (a) using the Scale Invariant Feature Transform (SIFT) algorithm to find discrete features in the current color image scan that match corresponding discrete features in the previous color image scan; (b) calculating a transformation that, when applied to discrete features in the previous color image scan, shifts the discrete features such that the position of the discrete features in the previous color image scan is the same as the position of their matching discrete features in the current color image scan; and (c) interpolating the transformation for regions in the previous color image scan between the discrete features.

16. The method according to claim 13, wherein, The parameters for determining the association include: (a) Encoding points u, v in the current color image scan and points u', v' in the previous color image scan using a neural network, such that each of points u, v and each of points u', v' is described by a vector, and the vectors of points u', v' in the previous color image scan are matched with the vectors of the corresponding associated points u, v in the current color image scan, and (b) For a given point u, v in the current color image scan, the neural network is used to locate the associated point u', v' in the previous color image scan by searching for points u', v' in the previous color image scan whose vectors match the vectors of the given point u, v.

17. The method according to claim 16, wherein, Using the neural network to locate associated points u' and v' in the previous color image scan further includes: determining a search space in the previous color image scan in which to search for the associated points u' and v' based on the association between the current 3D scan and the previous 3D scan.

18. The method according to claim 13, wherein, The method further includes: using the at least one computer processor to normalize the previous color image scan relative to the current color image scan, such that points u and v in the current color image scan corresponding to the region of the intraoral surface where the color changes between the previous color image scan and the current color image scan are emphasized relative to the corresponding points u' and v' in the previous color image scan.

19. The method according to claim 18, wherein, Normalizing the previous color image scan relative to the current color image scan includes: encoding the previous color image scan and the current color image using a neural network, and using a neural style transfer algorithm to output the content of the previous color image scan using the style of the current color image scan.

20. The method according to claim 18, wherein, Normalizing the previous color image scan relative to the current color image scan includes: using a Poisson fusion algorithm to output a normalized version of the previous color image scan, in which (a) the pixel values ​​of points u' and v' are substantially the same as the pixel values ​​of the corresponding associated points u and v in the current color image scan, and (b) the gradient between adjacent points u' and v' remains substantially the same as the gradient in the previous color image scan before normalization.

21. The method according to claim 18, wherein, The method further includes: using the at least one computer processor to analyze the current color image scan and the normalized previous color image scan, and detecting regions where there are differences between points u and v in the current color image scan and their corresponding points u' and v' in the normalized previous color image scan.

22. The method according to claim 21, wherein, The method further includes: using the at least one computer processor, based on the registration from the current color image scan to the current three-dimensional scan, designating a specific region of the current three-dimensional scan as the given region A, the specific region corresponding to a region where there is a difference between points u and v in the current color image scan and their corresponding points u' and v' in the normalized previous color image scan.

23. The method according to claim 22, wherein, The method further includes: using the at least one computer processor to visually indicate a specific area of ​​the current 3D scan to a user viewing the current 3D scan.

24. A method comprising using at least one computer processor: Acquire the current 3D scan of the 3D inner surface of the mouth, the current 2D scan of the 3D inner surface of the mouth, the previous 3D scan of the 3D inner surface of the mouth, and the previous 2D scan of the 3D inner surface of the mouth, wherein the previous 3D scan was obtained at a time point before the current 3D scan, and the previous 2D scan was obtained at a time point before the current 2D scan. Register the current 2D scan to the current 3D scan; Register the previous 2D scan to the previous 3D scan; The current 3D scan is associated with the previous 3D scan, such that a given region A of the current 3D scan is associated with the corresponding region A' of the previous 3D scan; Based on the association between the current 3D scan and the previous 3D scan, the registration of the current 2D scan to the current 3D scan, and the registration of the previous 2D scan to the previous 3D scan, for a given region A of the current 3D scan, (i) the portion of the previous 2D scan registered to the corresponding region A' of the previous 3D scan is applied to (ii) the given region A of the current 3D scan.

25. The method according to claim 24, wherein, The current two-dimensional scan includes the current color image scan, and the previous two-dimensional scan includes the previous color image scan.

26. The method according to claim 24, wherein, The current two-dimensional scan includes the current near-infrared (NIR) scan, and the previous two-dimensional scan includes the previous NIR scan.

27. The method according to claim 24, wherein, The current two-dimensional scan includes the current fluorescence imaging scan, and the previous two-dimensional scan includes the previous fluorescence imaging scan.

28. An intraoral scanning system, comprising: Intraoral scanner; as well as A computing device for performing the method according to any one of claims 1-27.

29. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 1-27.

Citation Information

Patent Citations

  • Automatic image selection for images of dental sites

    US12788540B2

  • Intraoral 3D scanner employing multiple miniature cameras and multiple miniature pattern projectors

    US20190388193A1

  • Light field intraoral 3D scanner with structured light illumination

    US20190388194A1

  • Intraoral 3D scanner employing multiple miniature cameras and multiple miniature pattern projectors

    US20200404243A1

  • Longitudinal analysis and visualization under limited accuracy system

    WO2018152374A1