Oral scanning device and method

Abstract

The present application provides an oral cavity scanning device and method. The oral cavity scanning device comprises an image extraction unit, an inertial measurement unit circuit and a processing unit. The processing unit obtains distance values corresponding to a first image and a second image according to inertial measurement unit information. The processing unit obtains a first contour corresponding to a first target object in the first image and a second contour corresponding to a second target object in the second image by a contour algorithm. The processing unit obtains a plurality of first sampling points and a plurality of second sampling points according to the first contour and the second contour. The processing unit finds out feature points corresponding to the first sampling points and the second sampling points by a feature point algorithm. The processing unit obtains depth information corresponding to each feature point according to the distance values and position information corresponding to the feature points by a depth information algorithm.

Description

Technical Field

[0001] This invention relates to an oral cavity scanning technology, and more particularly to an oral cavity scanning technology that uses a depth algorithm to construct a three-dimensional image of the oral cavity based on information obtained from an oral cavity scanning device. Background Technology

[0002] With advancements in technology, the application of oral scanning is becoming increasingly widespread. The results of oral scans can be used to construct a comprehensive image of a patient's oral cavity, allowing dentists to provide patients with a clearer understanding of the condition of their teeth. Therefore, how to generate more accurate oral scan results is a topic worthy of further research. Summary of the Invention

[0003] In view of the problems of the prior art described above, embodiments of the present invention provide an oral scanning device and method.

[0004] An oral scanning device is provided according to an embodiment of the present invention. The oral scanning device includes an image extraction unit, an inertial measurement unit circuit, and a processing unit. The image extraction unit acquires a first image and a second image. The inertial measurement unit circuit acquires inertial measurement unit information corresponding to when the image extraction unit generates the first and second images. The processing unit acquires the first and second images from the image extraction unit and the inertial measurement unit information from the inertial measurement unit circuit. The processing unit acquires a distance value corresponding to the first and second images based on the inertial measurement unit information. The processing unit acquires a first contour corresponding to a first target object in the first image and a second contour corresponding to a second target object in the second image using a contour algorithm. The processing unit acquires a plurality of first sampling points based on the first contour and a plurality of second sampling points based on the second contour. The processing unit finds feature points corresponding to the first and second sampling points using a feature point algorithm. Furthermore, the processing unit acquires depth information corresponding to each feature point based on the distance value and the position information corresponding to the feature point using a depth information algorithm.

[0005] According to some embodiments of the present invention, the oral scanning device further includes a light source unit. The light source unit provides a light source for the image extraction unit.

[0006] According to some embodiments of the present invention, the processing unit takes multiple first sub-contours proportionally within the inner layer of the first target object based on a first contour, and obtains multiple first sampling points on the first contour and each first sub-contour; and the processing unit takes multiple second sub-contours proportionally within the inner layer of the second target object based on a second contour, and obtains multiple second sampling points on the second contour and each of the aforementioned second sub-contours. Then, the processing unit uses a feature point algorithm to find corresponding feature points from all the first sampling points and all the second sampling points.

[0007] According to some embodiments of the present invention, the processing unit obtains a three-dimensional image based on the inertial measurement unit information and the movement and depth information corresponding to each feature point.

[0008] According to an embodiment of the present invention, an oral cavity scanning method is provided. The oral cavity scanning method is applicable to an oral cavity scanning device. The steps of the oral cavity scanning method include: acquiring a first image and a second image using an image extraction unit of the oral cavity scanning device; acquiring inertial measurement unit (IMU) information corresponding to when the image extraction unit generates the first image and the second image using an inertial measurement unit (IMU) circuit of the oral cavity scanning device; acquiring a distance value corresponding to the first image and the second image using a processing unit of the oral cavity scanning device based on the IMU information; acquiring a first contour corresponding to a first target object in the first image and a second contour corresponding to a second target object in the second image using a contour algorithm using the processing unit; acquiring a plurality of first sampling points based on the first contour and a plurality of second sampling points based on the second contour using the processing unit; finding feature points corresponding to the first sampling points and the second sampling points using a feature point algorithm using the processing unit; and acquiring depth information corresponding to each feature point using a depth information algorithm using the distance value and the position information corresponding to the feature point.

[0009] Other additional features and advantages of the present invention can be obtained by those skilled in the art through modifications and refinements based on the oral scanning device and method disclosed in the embodiments of this application, without departing from the spirit and scope of the present invention. Attached Figure Description

[0010] Figure 1 This is a block diagram showing an oral scanning device 100 according to an embodiment of the present invention.

[0011] Figure 2 This is a schematic diagram of the outline of the target object in the first and second images according to an embodiment of the present invention.

[0012] Figure 3 This is a schematic diagram of multiple sampling points of a target object in a first image and a second image according to an embodiment of the present invention.

[0013] Figure 4 This is a flowchart of an oral cavity scanning method according to an embodiment of the present invention.

[0014] [Symbol Explanation]

[0015] 100: Oral scanning device

[0016] 110: Image Extraction Unit

[0017] 120: Inertial Measurement Unit Circuit

[0018] 130: Processing Unit

[0019] A, B: Outlines

[0020] A1~A n B1~B n Sub-outline

[0021] I1: First image

[0022] I2: Second image

[0023] P1, P2: Sampling points

[0024] S410~S470: Steps Detailed Implementation

[0025] This section describes preferred embodiments of the invention and is intended to illustrate the spirit of the invention rather than to limit its scope of protection. The scope of protection of the invention shall be determined by the appended claims.

[0026] Figure 1 This is a block diagram showing an oral cavity scanning device 100 according to an embodiment of the present invention. Figure 1 As shown, the oral cavity scanning device 100 may include an image extraction unit 110, an inertial measurement unit (IMU) circuit 120, and a processing unit 130. Note that in Figure 1 The block diagrams shown are for illustrative purposes only and are not intended to represent embodiments of the present invention. Figure 1 This is not a limitation. The oral scanning device 100 may also include other components or other connection methods. According to one embodiment of the present invention, the oral scanning device 100 may be a dental endoscope device, but the present invention is not limited thereto.

[0027] According to one embodiment of the present invention, the processing unit 130 may also be configured in another electronic device (e.g., a laptop, a mobile phone, etc., but the present invention is not limited thereto) and obtain relevant information about the oral cavity scan from the oral cavity scanning device 100 for subsequent operations to build a three-dimensional image. In this embodiment, the oral cavity scanning device 100 may transmit the relevant information about the oral cavity scan to the electronic device via a wireless communication method, such as Bluetooth, Wi-Fi, or mobile communication (cellular network), but the present invention is not limited thereto.

[0028] According to one embodiment of the present invention, the oral scanning device 100 may also include a storage unit (not shown). The storage unit may be a volatile memory (e.g., random access memory (RAM)), a non-volatile memory (e.g., flash memory, read-only memory (ROM)), a hard disk, or a combination of the above. The storage unit can be used to store files and data required for performing an oral scan.

[0029] According to an embodiment of the present invention, the image extraction unit 110 may be a camera built into the oral cavity scanning device 100. The image extraction unit 110 can be used to extract image data and provide the extracted image data to the processing unit 130 for processing, or store it in a storage unit (not shown). According to an embodiment of the present invention, the oral cavity scanning device 100 may include a light source unit (not shown). The light source unit can be used to provide the light source required by the image extraction unit 120 when capturing images. The image extraction unit 110 can use the light source provided by the light source unit to capture images of the oral cavity for oral cavity scanning.

[0030] According to one embodiment of the present invention, the inertial measurement unit circuit 120 may be an inertial measurement unit chip. The inertial measurement unit circuit 120 may include a gyroscope and an accelerometer to obtain the position, angle and movement information of the lens of the image extraction unit 110 when the image extraction unit 110 captures different images.

[0031] According to one embodiment of the present invention, when scanning an oral cavity to create a three-dimensional image of the oral cavity, a user can use an oral cavity scanning device 100 to sequentially scan the oral cavity (e.g., scanning the upper and lower jaws from left to right, but the present invention is not limited thereto) to generate multiple oral cavity images (e.g.: Figure 2 The first image I1 and the second image I2 are shown, but the present invention is not limited thereto. Specifically, when a user scans the oral cavity using the oral cavity scanning device 100, the image extraction unit 110 can extract multiple oral cavity images at different time points. At different time points, the image extraction unit 110 is moved to different positions in the oral cavity to extract oral cavity images corresponding to different oral cavity regions. When the image extraction unit 110 is extracting images, the inertial measurement unit circuit 120 can obtain the inertial measurement unit information corresponding to each image captured by the image extraction unit 110. The inertial measurement unit information may include the position, angle, and movement information corresponding to the lens of the image extraction unit 110.

[0032] According to an embodiment of the present invention, after the image extraction unit 110 extracts the image, the processing unit 130 can obtain each image from the image extraction unit 110 (or from a storage unit), and obtain the inertial measurement unit information (i.e., position, angle and movement information) corresponding to the lens of the image extraction unit 110 when the image extraction unit 110 captures each image from the inertial measurement unit circuit 120 (or from a storage unit).

[0033] According to an embodiment of the present invention, the processing unit 130 can obtain a distance value corresponding to each image and the next image based on the inertial measurement unit information (i.e., position, angle, and movement information) corresponding to the lens of the image extraction unit 110. Specifically, the processing unit 130 can obtain the position change of the image extraction unit 110 when capturing the first image and the second image (i.e., the distance value corresponding to the first image and the second image) based on the inertial measurement unit information corresponding to the lens of the image extraction unit 110 when capturing the first image and the second image at a certain time point, and based on the inertial measurement unit information corresponding to the lens of the image extraction unit 110 when capturing the second image at the next time point.

[0034] Furthermore, according to embodiments of the present invention, the processing unit 130 can obtain the contour and multiple sub-contours corresponding to the target object (e.g., teeth) in each image using a contour algorithm (e.g., OpenCY-python edge detection, but the present invention is not limited thereto). Figure 2 For example, Figure 2 This is a schematic diagram of the outline of a target object in a first image and a second image according to an embodiment of the present invention. Figure 2 As shown, the processing unit 130 can obtain the contour A (represented by a dashed line) of the teeth in the first image I1 and the contour B (represented by a dashed line) of the teeth in the second image I2 using a contour algorithm. Then, as... Figure 2 As shown, after the processing unit 130 obtains the tooth contour A in the first image I1 and the tooth contour B in the second image I2, the processing unit 130 moves inward proportionally according to the tooth contour A in the first image I1 and the tooth contour B in the second image I2 to obtain multiple sub-contours (i.e., sub-contours A1 to A2) corresponding to the inner side of the teeth in the first image I1 and the second image I2. n Contour of the sub-section B1~B n In this embodiment, the proportional inward movement means that the processing unit 130 can divide the tooth from the outermost ring to the center into multiple intervals proportionally according to a displacement parameter (e.g., 10%, but the invention is not limited thereto), and take a sub-contour in each interval. For example, if the displacement parameter is 10%, the processing unit 130 can move inward by 10% of the interval from contour A to obtain sub-contour A1, but the invention is not limited thereto.

[0035] According to an embodiment of the present invention, after the processing unit 130 obtains the contour and multiple sub-contours corresponding to the target object (e.g., teeth) in each image, the processing unit 130 takes multiple sampling points on each contour and sub-contour. Figure 3 For example, Figure 3 This is a schematic diagram of multiple sampling points of a target object in a first image and a second image according to an embodiment of the present invention. For example... Figure 3 As shown, the processing unit 130 takes 40 sampling points on the contours and multiple sub-contours corresponding to the teeth in the first image I1 and the second image I2, but the present invention is not limited thereto.

[0036] According to an embodiment of the present invention, after the processing unit 130 obtains the sampling points included in the contour and multiple sub-contours corresponding to each image, the processing unit 130 can use a feature point algorithm (e.g., OpenCV Feature2D algorithm, but the present invention is not limited thereto) to find multiple feature points corresponding to the sampling points contained in one image and the sampling points contained in the next image. If the target object in the image is a tooth, the processing unit 130 can use the feature point algorithm to determine, based on features such as the curvature of the tooth, the shape of the plaque, cavities, and missing corners, the multiple feature points corresponding to the sampling points contained in one image and the sampling points contained in the next image (i.e., determining whether the teeth contained in one image and the teeth contained in the next image are the same). Figure 3 For example, by using the feature point algorithm, the processing unit 130 can find that the sampling point P1 of the first image I1 and the sampling point P2 of the second image I2 are corresponding feature points, but the present invention is not limited thereto.

[0037] According to an embodiment of the present invention, after the processing unit 130 obtains multiple feature points corresponding to each image and its next image, the processing unit 130 uses a depth information algorithm (e.g., a binocular depth information algorithm) to obtain the depth information corresponding to each feature point based on the distance value between each image and the next image (i.e., the position change of the image extraction unit 110 when capturing each image and its next image) and the position information of the feature points (e.g., the coordinates corresponding to the feature points). The depth information algorithm can be expressed by the following formula:

[0038]

[0039] Where Z represents the depth corresponding to a feature point, f represents the focal length of the image extraction unit 110, and D... x x1 represents the x-coordinate of a feature point in one image and the next image; x2 represents the x-coordinate of the corresponding feature point in the next image; D represents the x-coordinate of the feature point in the next image. yy1 represents the y-coordinate of a feature point in one image and the next image, y2 represents the y-coordinate of the corresponding feature point in the next image, and y1 represents the y-coordinate of the feature point in the next image. Figure 3 For example, when sampling point P1 of the first image I1 and sampling point P2 of the second image I2 are corresponding feature points, and the coordinates of sampling point P1 are (x1, y1) and the coordinates of sampling point P2 are (x2, y2), the processing unit 130 can substitute the x coordinates of sampling point P1 and sampling point P2 into the above formula to obtain the depth (i.e., Z) corresponding to sampling point P1.

[0040] It should be noted that the present invention Figure 2 and 3 Each image only displays an image of one plane corresponding to the tooth, but this invention is not limited thereto. The oral scanning device 100 also acquires images of different planes of the tooth and acquires depth information of all feature points in the images of other planes.

[0041] After obtaining the depth information of all feature points, the processing unit 130 can obtain a three-dimensional image corresponding to each image based on the inertial measurement unit information corresponding to each image and the position and depth information of all feature points in each image. Then, the processing unit 130 can combine or stitch all the three-dimensional images to generate a three-dimensional image of the corresponding oral cavity.

[0042] Figure 4 This is a flowchart of an oral cavity scanning method according to an embodiment of the present invention. The oral cavity scanning method is applicable to an oral cavity scanning device 100. Figure 4 As shown, in step S410, the oral cavity scan uses an image extraction unit to obtain a first image and a second image.

[0043] In step S420, an inertial measurement unit circuit of the oral scanning device 100 obtains the inertial measurement unit information corresponding to the first image and the second image generated by the image extraction unit of the oral scanning device 100.

[0044] In step S430, a processing unit of the oral cavity scanning device 100 obtains a distance value corresponding to the first image and the second image based on the information from the inertial measurement unit.

[0045] In step S440, the processing unit of the oral cavity scanning device 100 obtains a first contour corresponding to a first target object in the first image and a second contour corresponding to a second target object in the second image by means of a contour algorithm.

[0046] In step S450, the processing unit of the oral cavity scanning device 100 obtains a plurality of first sampling points according to the first contour and a plurality of second sampling points according to the second contour.

[0047] In step S460, the processing unit of the oral cavity scanning device 100 uses a feature point algorithm to find the feature points corresponding to the first sampling point and the second sampling point.

[0048] In step S470, the processing unit of the oral cavity scanning device 100 obtains the depth information corresponding to each feature point by means of a depth information algorithm, based on the distance value and the position information corresponding to the feature point.

[0049] According to one embodiment of the present invention, in the oral cavity scanning method, a light source device of the oral cavity scanning apparatus 100 can provide a light source for the image extraction unit.

[0050] According to an embodiment of the present invention, in the oral cavity scanning method, steps S450 to S460 further include: the processing unit of the oral cavity scanning device 100, based on a first contour, proportionally extracts a plurality of first sub-contours within the inner layer of a first target object, and obtains a plurality of first sampling points on the first contour and each first sub-contour. Furthermore, the processing unit of the oral cavity scanning device 100, based on a second contour, proportionally extracts a plurality of second sub-contours within the inner layer of a second target object, and obtains a plurality of second sampling points on the second contour and each of the aforementioned second sub-contours. Next, the processing unit of the oral cavity scanning device 100 uses a feature point algorithm to find corresponding feature points from all the first sampling points and all the second sampling points.

[0051] According to an embodiment of the present invention, the steps in the oral cavity scanning method further include that the processing unit of the oral cavity scanning device 100 obtains a three-dimensional image based on the inertial measurement unit information and the movement information and depth information corresponding to each feature point.

[0052] According to the oral cavity scanning method proposed in this invention, a three-dimensional image of the oral cavity can be established using information obtained from the image extraction unit and inertial measurement unit circuit of the oral cavity scanning device.

[0053] The serial numbers in this specification and the claims, such as "first," "second," etc., are for ease of explanation only and there is no sequential relationship between them.

[0054] The steps of the methods and algorithms disclosed in this specification can be directly applied to hardware and software modules or a combination of both by executing a processor. A software module (including execution instructions and related data) and other data can be stored in a data storage device, such as random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electronically erasable programmable read-only memory (EEPROM), registers, hard disks, portable optical discs, optical disc read-only memory (CD-ROM), DVDs, or any other computer-readable storage medium format known in the art. A storage medium can be coupled to a machine device, for example, such as a computer / processor (referred to as a processor in this specification for convenience), through which the processor can read information (such as program code) and write information to the storage medium. A storage medium can integrate a processor. An application-specific integrated circuit (ASIC) includes a processor and a storage medium. A user equipment includes an application-specific integrated circuit. In other words, the processor and storage medium are included in the user equipment in a manner that is not directly connected to the user equipment. Furthermore, in some embodiments, any product suitable for a computer program includes a readable storage medium, wherein the readable storage medium includes program code associated with one or more of the disclosed embodiments. In some embodiments, the product of the computer program may include packaging material.

[0055] The preceding paragraphs use multiple levels of description. Clearly, the teachings herein can be implemented in various ways, and any particular architecture or functionality disclosed in the examples is merely a representative case. Based on the teachings herein, those skilled in the art will understand that the various levels disclosed herein can be implemented independently or that two or more levels can be implemented in combination.

[0056] Although this disclosure has been provided above with reference to embodiments, it is not intended to limit this disclosure. Any person skilled in the art may make some modifications and refinements without departing from the spirit and scope of this disclosure. Therefore, the scope of protection of the invention shall be determined by the scope defined in the appended claims.

Claims

1. An oral cavity scanning device, comprising: The image extraction unit acquires the first image and the second image; The inertial measurement unit circuit acquires the inertial measurement unit information corresponding to when the image extraction unit generates the first image and the second image; and The processing unit obtains the first image and the second image from the image extraction unit, and obtains the inertial measurement unit information from the inertial measurement unit circuit. The processing unit obtains the distance values ​​corresponding to the first image and the second image based on the information from the inertial measurement unit. The processing unit uses a contour algorithm to obtain a first contour corresponding to a first target object in the first image and a second contour corresponding to a second target object in the second image. The aforementioned processing unit obtains multiple first sampling points based on the first contour and multiple second sampling points based on the second contour. The aforementioned processing unit uses a feature point algorithm to find the feature points corresponding to the first sampling point and the second sampling point, and The aforementioned processing unit uses a depth information algorithm to obtain the depth information corresponding to each of the aforementioned feature points based on the aforementioned distance value and the location information corresponding to the aforementioned feature points.

2. The oral cavity scanning device as claimed in claim 1, further comprising: The light source unit provides a light source for the image extraction unit mentioned above.

3. The oral scanning device as claimed in claim 1, wherein the processing unit takes a plurality of first sub-contours proportionally in the inner layer of the first target object according to the first contour, and obtains the plurality of first sampling points on each of the first sub-contours; and the processing unit takes a plurality of second sub-contours proportionally in the inner layer of the second target object according to the second contour, and obtains the plurality of second sampling points on each of the second sub-contours.

4. The oral scanning device as claimed in claim 3, wherein the processing unit uses the feature point algorithm to find the corresponding feature points from all the first sampling points and the second sampling points.

5. The oral cavity scanning device as claimed in claim 1, wherein the processing unit obtains a three-dimensional image based on the inertial measurement unit information, the movement information corresponding to each of the aforementioned feature points, and the aforementioned depth information.

6. An oral cavity scanning method, applicable to an oral cavity scanning device, comprising: The first image and the second image are obtained by the image extraction unit of the oral scanning device described above; By means of the inertial measurement unit circuit of the above-mentioned oral cavity scanning device, the inertial measurement unit information corresponding to the generation of the first image and the second image by the image extraction unit is obtained; and The processing unit of the oral cavity scanning device obtains the distance values ​​corresponding to the first image and the second image based on the information from the inertial measurement unit. The processing unit uses a contour algorithm to obtain a first contour corresponding to a first target object in the first image and a second contour corresponding to a second target object in the second image. The processing unit obtains a plurality of first sampling points based on the first contour and a plurality of second sampling points based on the second contour. The aforementioned processing unit uses a feature point algorithm to find the feature points corresponding to the first sampling point and the second sampling point; and The processing unit uses a depth information algorithm to obtain the depth information corresponding to each of the aforementioned feature points based on the distance value and the location information corresponding to the feature points.

7. The oral cavity scanning method as described in claim 6, further comprising: The light source device of the oral scanning device provides a light source to the image extraction unit.

8. The oral cavity scanning method as described in claim 6, further comprising: The processing unit takes multiple first sub-contours proportionally within the inner layer of the first target object based on the first contour. The plurality of first sampling points are obtained on the first contour and each of the first sub-contours described above; The processing unit, based on the second contour, proportionally extracts multiple second sub-contours within the inner layer of the second target object; and The aforementioned plurality of second sampling points are obtained on the aforementioned second contour and each of the aforementioned second sub-contours.

9. The oral cavity scanning method as described in claim 8, further comprising: The processing unit uses the feature point algorithm described above to find the corresponding feature points from all the first sampling points and all the second sampling points.

10. The oral cavity scanning method as described in claim 6, further comprising: The processing unit obtains a three-dimensional image based on the information from the inertial measurement unit, the movement information corresponding to each feature point, and the depth information.

Images