Image processing method, image processing device, and program

Abstract

An image processing method for displaying a tartar region on the basis of an image obtained by imaging teeth and tartar subjected to a fluorescence reaction caused by irradiating the teeth with light of a wavelength band including blue light (S201-S206), in which a natural tooth region among the tooth regions is detected on the basis of the image; detecting a tartar region on the basis of the image (S207); and displaying a tartar region (S105), limited only to a natural tooth region. The image processing method may, for example, detect a first region, which is (i) a region having a luminance value equal to or greater than a preset first threshold value among all pixel regions of the image, and (ii) a region having a luminance value equal to or greater than a preset second threshold value among all pixel regions of the image, and detect a natural tooth region on the basis of the first region during detection of the natural tooth region. Or (ii) a region in which the green pixel value is equal to or greater than a preset second threshold value among all pixel regions of the image.

Description

Technical Field

[0001] This disclosure relates to image processing methods, image processing apparatus, and programs. Background Technology

[0002] Devices for detecting dental plaque based on images obtained by photographing teeth inside the mouth are known. For example, Patent Document 1 discloses a technique for storing a care area in the oral cavity and the amount of dental plaque detected corresponding to that care area. Furthermore, Patent Document 2 discloses a technique for correcting dental images.

[0003] Existing technical documents Patent documents Patent Document 1: Japanese Patent Application Publication No. 2013-042906 Patent Document 2: Japanese Patent Application Publication No. 2009-37414 Summary of the Invention

[0004] The problem that the invention aims to solve The goal of such a device for detecting dental plaque is to improve the accuracy of plaque detection.

[0005] Therefore, this disclosure provides an image processing method or image processing apparatus that can improve the detection accuracy of dental plaque.

[0006] Methods for solving problems One aspect of this disclosure is an image processing method that displays a tartar area based on an image obtained by photographing a tooth and tartar exhibiting fluorescence, wherein the fluorescence is induced by irradiating the tooth with light of a wavelength containing blue light, wherein, based on the image, a natural tooth area is detected within the tooth area; a tartar area is detected based on the image; and the tartar area is displayed only within the natural tooth area.

[0007] Invention Effects This disclosure provides an image processing method or image processing apparatus that can improve the accuracy of dental plaque detection. Attached Figure Description

[0008] Figure 1 This is a three-dimensional view of the intraoral camera in the intraoral camera system of the embodiment.

[0009] Figure 2 This is a schematic cross-sectional view showing the photographic optical system of the intraoral camera assembled in the intraoral camera system of the embodiment.

[0010] Figure 3 This is a schematic structural diagram of the intraoral camera system according to the implementation method.

[0011] Figure 4This is a flowchart illustrating the operation of the intraoral camera system in the embodiment.

[0012] Figure 5 This is a functional block diagram of the portable terminal used in the implementation method.

[0013] Figure 6 This is a flowchart of the image generation process in the implementation method.

[0014] Figure 7 This is a diagram showing an example of an RGB image (4th RGB image) of an implementation method.

[0015] Figure 8 This is a diagram illustrating an example of the first natural tooth region in an embodiment.

[0016] Figure 9 This is a diagram illustrating an example of the gingival region in an implementation method.

[0017] Figure 10 This is a diagram illustrating an example of the second natural tooth region in an embodiment.

[0018] Figure 11 This is a diagram illustrating an example of the first tartar region in an embodiment.

[0019] Figure 12 This is a diagram illustrating an example of a mask image used in an implementation.

[0020] Figure 13 This is a diagram illustrating an example of the second tooth region in an embodiment.

[0021] Figure 14 This is a diagram showing an example of a display image of an implementation method. Detailed Implementation

[0022] When blue light is shone into the oral cavity to detect tartar, the blue light is reflected off the surface of restorations made of materials such as gold or silver. This can lead to the misdetection of red wavelengths within the blue light as red fluorescence from the tartar. Furthermore, when a user uses a handheld intraoral camera to photograph the tooth surface, it is difficult to distinguish between the red fluorescence of the tartar and the reflected light from the restoration surface.

[0023] In view of the aforementioned issues, this disclosure provides a method for detecting plaque only in natural tooth areas, excluding areas containing artificial teeth such as restorations. This prevents false detection of plaque caused by restorations.

[0024] One aspect of this disclosure is an image processing method that displays a tartar area based on an image obtained by photographing a tooth and tartar exhibiting fluorescence, wherein the fluorescence is induced by irradiating the tooth with light of a wavelength containing blue light, wherein, based on the image, a natural tooth area is detected within the tooth area; a tartar area is detected based on the image; and the tartar area is displayed only within the natural tooth area.

[0025] Therefore, this image processing method can suppress the misdetection of repairs and other materials as dental plaque. Thus, this image processing method can improve the detection accuracy of dental plaque.

[0026] For example, the image processing method may also detect a first region in the detection of the natural tooth region, and detect the natural tooth region based on the first region, wherein the first region is (i) a region in the entire pixel region of the image whose brightness value is above a preset first threshold, or (ii) a region in the entire pixel region of the image whose green pixel value is above a preset second threshold.

[0027] Therefore, this image processing method can detect natural tooth regions with high precision using brightness values ​​or green pixel values.

[0028] For example, the image processing method can also detect the natural tooth region as the region in which the difference between the green pixel value, red pixel value and blue pixel value in the first region is less than a preset third threshold.

[0029] Therefore, this image processing method can use the differences between green, red, and blue pixel values ​​to detect natural tooth regions with high precision.

[0030] For example, the image processing method can also detect the gingiva region based on the hue of the image in the detection of the natural tooth region, and detect the region obtained by excluding the gingiva region from the first region as the natural tooth region.

[0031] Therefore, this image processing method can suppress the gingival region from being incorrectly identified as a natural tooth region, thus improving the detection accuracy of natural tooth regions.

[0032] For example, the image processing method can also detect the natural tooth region by excluding regions with an area smaller than a preset area from the first region.

[0033] Therefore, this image processing method can suppress the misidentification of high-brightness areas caused by reflected light and other factors outside the natural tooth area as natural tooth areas, thus improving the detection accuracy of natural tooth areas.

[0034] For example, the image processing method can also detect the natural tooth region by extending the boundary of the first region outward by a predetermined amount.

[0035] Therefore, this image processing method can suppress the excessive underdisplay of tartar areas by expanding the natural tooth area.

[0036] Furthermore, one aspect of the image processing apparatus disclosed herein displays a tartar area based on an image obtained by photographing a tooth and tartar exhibiting fluorescence, wherein the fluorescence is induced by irradiating the tooth with light in the wavelength range containing blue light. The apparatus comprises: a natural tooth area detection unit that detects natural tooth areas within the tooth region based on the image; a tartar area detection unit that detects tartar areas based on the image; and a display unit that displays the tartar area limited to the natural tooth area.

[0037] Therefore, this image processing device can suppress the misdetection of repairs and other materials as dental plaque. Thus, this image processing device can improve the detection accuracy of dental plaque.

[0038] Additionally, one aspect of the program disclosed herein is a program for causing a computer to perform the image processing method.

[0039] Furthermore, these general or specific methods can be implemented either through systems, methods, integrated circuits, computer programs, or recording media such as computer-readable CD-ROMs, or through any combination of systems, methods, integrated circuits, computer programs, and recording media.

[0040] Hereinafter, embodiments will be described in detail with appropriate reference to the accompanying drawings. However, sometimes unnecessary detailed descriptions are omitted. For example, detailed descriptions of matters already known or repeated descriptions of substantially the same structures are sometimes omitted. This is to avoid making the following description unnecessarily lengthy and to facilitate understanding by those skilled in the art.

[0041] Furthermore, the inventors have provided the accompanying drawings and the following description in order to enable those skilled in the art to fully understand this disclosure, but these are not intended to limit the subject matter of the claims.

[0042] (Implementation Method) Figure 1 This is a three-dimensional view of the intraoral camera in the intraoral camera system of this embodiment. (As shown) Figure 1As shown, the intraoral camera 10 has a toothbrush-shaped frame that can be operated with one hand. The frame includes a head 10a that is positioned inside the user's mouth when photographing the teeth, a handle 10b that is held by the user, and a neck 10c that connects the head 10a and the handle 10b.

[0043] Figure 2 This is a schematic cross-sectional view showing the photographic optical system 12 assembled in the intraoral camera 10. (See diagram below.) Figure 2 As shown, in this embodiment, the imaging optics system 12 of the intraoral camera 10 is assembled on the head 10a and neck 10c. The imaging optics system 12 includes an imaging element 14 and a lens 16 disposed on its optical axis LA.

[0044] The imaging element 14 is, for example, a C-MOS (Complementary Metal-Oxide-Semiconductor) sensor or a CCD (Charge Coupled Device) element, which images the tooth D through the lens 16. The imaging element 14 outputs a signal (image data) corresponding to the image to the outside.

[0045] Lens 16, for example, is a condenser lens that images the incident tooth D onto the imaging element 14. Alternatively, lens 16 can be a single lens or a lens group consisting of multiple lenses.

[0046] In this embodiment, the photographic optical system 12 further includes: a mirror 18 that reflects the image of the tooth D toward the lens 16; a blue light cutoff filter (blue blocking element) 20 disposed between the mirror 18 and the lens 16; and an aperture 24 disposed between the lens 16 and the imaging element 14.

[0047] The lens 18 is positioned on the optical axis LA of the photographic optical system 12 in such a way that the image of the tooth D, which has passed through the entrance port 12a of the photographic optical system 12, is reflected toward the lens 16.

[0048] The blue light cutoff filter 20 is a filter that cuts off the blue wavelength component of the light incident on the imaging element 14. When detecting plaque by irradiating teeth with light containing a blue wavelength, if the blue wavelength is enhanced to increase the excitation fluorescence of the plaque, the first RGB image will appear bluish overall. In this state, blue pixel values ​​dominate compared to red and green pixel values, thus sometimes reducing the ability to easily distinguish plaque areas through image processing (exposure control processing and white balance adjustment processing) described later. As a countermeasure, the blue light cutoff filter 20 cuts off the blue wavelength component of the light incident on the imaging element 14.

[0049] Aperture 24 is a plate-shaped component with a through hole on the optical axis LA of the photographic optical system 12, achieving a deeper depth of focus. This allows for focusing in the depth direction within the oral cavity, resulting in a clearly defined image of the dental arch.

[0050] Additionally, the intraoral camera 10 is equipped with multiple LEDs 26A to 26D as illumination devices for illuminating the teeth D of the photographic subject during shooting. These LEDs 26A to 26D are, for example, blue LEDs (Light Emitting Diodes). Furthermore, as... Figure 1 As shown, in this embodiment, multiple LEDs 26A-26D are arranged to surround the entrance port 12a. Furthermore, a light-transmitting cover 28 is provided on the head 10a to cover the multiple LEDs 26A-26D and the entrance port 12a, preventing the gums G and other components from coming into contact with the multiple LEDs 26A-26D and causing insufficient illumination. Alternatively, a portion of the multiple LEDs 26A-26D can be made white LEDs. By making a portion of the multiple LEDs 26A-26D white LEDs, the first RGB image can be brightened, and the balance of blue pixel values ​​relative to red and green pixel values ​​can be improved.

[0051] Furthermore, in the case of this embodiment, such as Figure 2 As shown, the intraoral camera 10 has a composition adjustment mechanism 30 and a focus adjustment mechanism 32.

[0052] The composition adjustment mechanism 30 comprises a frame 34 that holds the imaging element 14 and the lens 16, and an actuator 36 that moves the frame 34 in the extension direction of the optical axis LA. The viewing angle, i.e., the size of the tooth row imaged on the imaging element 14, is adjusted by adjusting the position of the frame 34 via the actuator 36. Furthermore, the composition adjustment mechanism 30 automatically adjusts the position of the frame 34, for example, so that a single tooth is projected into the photographic image. Additionally, the composition adjustment mechanism 30 adjusts the position of the frame 34 based on user input to obtain the user's desired viewing angle.

[0053] The focus adjustment mechanism 32 is held within the frame 34 of the composition adjustment mechanism 30 and consists of a lens holder 38 that holds the lens 16 and an actuator 40 that moves the lens holder 38 in the extension direction of the optical axis LA. The actuator 40 adjusts the relative position of the lens holder 38 with respect to the imaging element 14, thereby adjusting the focus, i.e., focusing. Furthermore, the focus adjustment mechanism 32 automatically adjusts the position of the lens holder 38, for example, to focus on the teeth located in the center of the photographic image. Additionally, the focus adjustment mechanism 32 adjusts the position of the lens holder 38 based on user input.

[0054] In addition, the components of the photographic optical system 12, other than the reflector 18, can also be provided in the handle portion 10b of the intraoral camera 10.

[0055] The image output by the camera element 14 is an RGB image in which multiple pixels constituting the image each have RGB sub-pixels.

[0056] In addition, the intraoral camera 10 is equipped with multiple LEDs 26A to 26D as an illumination device for illuminating the teeth of the photographic subject during shooting. The multiple LEDs 26A to 26D are, for example, blue LEDs that illuminate blue light with a wavelength having a peak value of 405nm. Furthermore, the multiple LEDs 26A to 26D can be any light source that illuminates light in the blue light band, and are not limited to blue LEDs.

[0057] Figure 3 This is a schematic structural diagram of the intraoral camera system according to this embodiment. Figure 3 As shown, the intraoral camera system of this embodiment is generally configured to use an intraoral camera 10 to photograph the dental arch and perform image processing on the photographic images.

[0058] like Figure 3 As shown, the intraoral camera system includes an intraoral camera 10, a portable terminal 70, and a cloud server 80. The portable terminal 70 is, for example, a smartphone or tablet capable of wireless communication. The portable terminal 70 includes, for example, a touchscreen 72 capable of displaying images of the dental arch as both an input and output device. The portable terminal 70 functions as the user interface for the intraoral camera system.

[0059] The cloud server 80 is a server capable of communicating with the portable terminal 70 via the Internet or other means, providing the portable terminal 70 with applications for using the intraoral camera 10. For example, a user downloads the application from the cloud server 80 and installs it on the portable terminal 70. In addition, the cloud server 80 obtains images of the dental arch captured by the intraoral camera 10 via the portable terminal 70.

[0060] The intraoral camera 10 includes a central control unit 50, which is the main part for system control; an LED control unit 54, which controls multiple LEDs 26A to 26D; a lens driver 56, which controls the actuator 36 of the composition adjustment mechanism 30 and the actuator 40 of the focus adjustment mechanism 32; and a position sensor 90.

[0061] In addition, the intraoral camera 10 has a wireless communication module 58 for wireless communication with the portable terminal 70, and a power control unit 60 for supplying power to the central control unit 50 and the like.

[0062] The central control unit 50 of the intraoral camera 10 is mounted, for example, on the handle portion 10b of the intraoral camera 10. The central control unit 50 may include, for example, a controller 62 such as a CPU (Central Processing Unit) or MPU (Micro Processing Unit) that performs the various processes described later, and a memory 64 such as RAM (Random Access Memory) or ROM (Read Only Memory) that stores programs for causing the controller 62 to perform various processes. Furthermore, in addition to the programs, the memory 64 stores images of the teeth captured by the imaging element 14 (image data) and various setting data. The image of the teeth captured by the imaging element 14 is an example of a first RGB image.

[0063] The controller 62 transmits the tooth pattern image output from the camera element 14 to the portable terminal 70 via the wireless communication module 58. The portable terminal 70 displays the transmitted tooth pattern image on the touch screen 72, thereby displaying the tooth pattern image to the user.

[0064] The LED control unit 54, for example, is mounted on the handle 10b of the intraoral camera 10. Based on control signals from the controller 62, it controls the lighting and extinguishing of multiple LEDs 26A to 26D. The LED control unit 54 is, for example, composed of circuitry. For instance, when a user activates the intraoral camera 10 using the touchscreen 72 of the portable terminal 70, the portable terminal 70 sends a corresponding signal to the controller 62 via the wireless communication module 58. Based on the received signal, the controller 62 sends a control signal to the LED control unit 54 to illuminate the multiple LEDs 26A to 26D.

[0065] The lens driver 56, for example, is mounted on the handle 10b of the intraoral camera 10. Based on control signals from the controller 62 of the central control unit 50, it controls the actuator 36 of the composition adjustment mechanism 30 and the actuator 40 of the focus adjustment mechanism 32. The lens driver 56 is, for example, composed of circuitry. For example, when a user performs an operation related to composition adjustment or focus adjustment on the touchscreen 72 of the portable terminal 70, the portable terminal 70 sends a corresponding signal to the central control unit 50 via the wireless communication module 58. The controller 62 of the central control unit 50 sends a control signal to the lens driver 56 based on the received signal to perform composition adjustment or focus adjustment. In addition, for example, the controller 62 calculates the control amount of the actuator 36 or 40 required for composition adjustment or focus adjustment based on the tooth image from the imaging element 14, and sends a control signal corresponding to the calculated control amount to the lens driver 56.

[0066] The wireless communication module 58, for example, is mounted on the handle 10b of the intraoral camera 10 and communicates wirelessly with the portable terminal 70 based on control signals from the controller 62. The wireless communication module 58 performs wireless communication with the portable terminal 70 using existing communication standards such as WiFi (registered trademark) and Bluetooth (registered trademark). Through the wireless communication module 58, the intraoral camera 10 transmits an image of the dental arch D to the portable terminal 70, or the portable terminal 70 transmits operation signals to the intraoral camera 10.

[0067] In this embodiment, the power control unit 60 is mounted on the handle 10b of the intraoral camera 10, distributing power from the battery 66 to the central control unit 50, the LED control unit 54, the lens driver 56, and the wireless communication module 58. The power control unit 60 is, for example, composed of circuitry. Furthermore, in this embodiment, the battery 66 is a rechargeable secondary battery, wirelessly charged via a coil 68 mounted on the intraoral camera 10 and connected to an external charger 69 connected to a commercial power source.

[0068] Position sensor 90 is a sensor used to detect the posture and position of intraoral camera 10, such as a multi-axis (here, x, y, and z axes) accelerometer. For example, position sensor 90 could also be a six-axis sensor with a three-axis accelerometer and a three-axis gyroscope. For example, as... Figure 1 As shown, the z-axis coincides with the optical axis LA. The y-axis is parallel to the imaging plane and extends along the length of the intraoral camera 10. Additionally, the x-axis is parallel to the imaging plane and orthogonal to the y-axis. The outputs of each axis of the position sensor 90 can also be transmitted to the portable terminal 70 via the central control unit 50 and the wireless communication module 58.

[0069] As the position sensor 90, piezoelectric, capacitive, or thermal sensing MEMS (Micro Electro Mechanical Systems) sensors can also be used. Furthermore, although not specifically illustrated, a correction circuit can be provided to balance the sensitivity of the sensors on each axis, correct for the temperature characteristics of the sensitivity, or correct for temperature drift. Additionally, a bandpass filter (low-pass filter) can be provided to remove motion acceleration components or noise. Furthermore, noise can be reduced by smoothing the output waveform of the accelerometer.

[0070] Next, the operation of the intraoral camera system will be explained. Figure 4 This is a diagram illustrating the workflow of an intraoral camera system. Additionally, Figure 4 The processing shown is, for example, real-time processing, performed whenever one or more frames of image data are obtained.

[0071] The user generates image data by taking pictures of their teeth and gums inside their mouth using an intraoral camera 10 (S101). This image data can be, for example, obtained by capturing images of teeth that fluoresce when illuminated with light in the blue light band. The intraoral camera 10 then sends the captured image data to a portable terminal 70 (S102). The image data can be a moving image or one or more still images. Furthermore, if the image data is a moving image or multiple still images, sensor data can be sent for each frame of the moving image or each still image. Alternatively, if the image data is a moving image, sensor data can be sent for multiple frames.

[0072] In addition, image data can be transmitted in real time or in a concentrated manner after a series of images (such as images of all the teeth inside the mouth).

[0073] The portable terminal 70 performs image processing on the received image data (S103), generates an image representing the tartar area using the image data after image processing (S104), and displays the generated image (S105).

[0074] By using such an intraoral camera system, users can capture images of their own mouths using the intraoral camera 10 and check the state of their mouths displayed on the portable terminal 70. Furthermore, by showing areas of tartar in the displayed images, users can easily identify areas missed during brushing.

[0075] Furthermore, the portable terminal 70 can, for example, generate a three-dimensional model of multiple teeth inside the oral cavity based on multiple captured image data. Additionally, the portable terminal 70 can display images based on the generated three-dimensional model.

[0076] Furthermore, this example describes the processing of dental images by the portable terminal 70, but this processing can also be partially or entirely performed by the intraoral camera 10. The portable terminal 70 is an example of an image processing device (dental plaque detection device).

[0077] Figure 5 This is a functional block diagram of a portable terminal 70. The portable terminal 70 includes an acquisition unit 101, a generation unit 102, and a display unit 103.

[0078] The acquisition unit 101 acquires the image data (first RGB image) transmitted from the intraoral camera 10. The acquisition unit 101 may also acquire sensor data from the intraoral camera 10 in addition to the image data. The first RGB image is an image obtained by the intraoral camera 10 by photographing teeth that have undergone a fluorescence reaction by irradiating the teeth with light in a wavelength band containing blue light. Here, the blue light is an example of irradiation light having a prescribed wavelength that excites the fluorescent substance contained in dental plaque. In addition, the fluorescent substance is, for example, porphyrin.

[0079] The generation unit 102 uses the image data (first RGB image) acquired by the acquisition unit 101 to generate a display image indicating the dental plaque region. The generation unit 102 includes an image processing unit 111, a natural tooth region detection unit 112, a dental plaque region detection unit 113, and an image generation unit 114.

[0080] The image processing unit 111 performs image processing on the first RGB image as preprocessing. Specifically, the image processing unit 111 generates a second RGB image by performing inverse gamma transformation (inverse gamma correction) on the first RGB image. Next, the image processing unit 111 generates a third RGB image by performing white balance adjustment processing on the second RGB image. Next, the image processing unit 111 generates a fourth RGB image by performing gamma transformation (gamma correction) on the third RGB image.

[0081] The white balance adjustment processing is processing for adjusting the hue of white in the image. For example, the image processing unit 111 extracts a plurality of pixels within the plurality of second RGB pixels constituting the second RGB image that is the processing target, where the RGB values satisfy the following formula 1 and formula 2.

[0082] min(R, G, B) ≤ Ths, and max(R, G, B) < Thmax (formula 1) Thl ≤ Y ≤ Thu (formula 2) In formula 1, min(R, G, B) represents the minimum value among the pixel values of the three sub-pixels of RGB (that is, the red pixel value, the green pixel value, and the blue pixel value) possessed by the second RGB pixel.

[0083] Ths is a threshold value for excluding a region (such as a gloss region) that is strongly affected by the reflection of the irradiated light in the first RGB image. Ths is, for example, 900 in 10-bit representation.

[0084] max(R, G, B) represents the maximum value among the pixel values of the three sub-pixels of RGB (that is, the red pixel value, the green pixel value, and the blue pixel value) possessed by the second RGB pixel.

[0085] Thmax represents the maximum possible pixel value. For example, Thmax is represented as 1023 in a 10-bit representation.

[0086] In Equation 2, Thl is the threshold representing the lower limit of the tooth region, and Thu is the threshold representing the upper limit of the tooth region.

[0087] The tooth region in the second RGB image is extracted using Equation 2. That is, the multiple pixels extracted using Equations 1 and 2 are the multiple pixels that constitute the tooth region.

[0088] Furthermore, the image processing unit 111 calculates the average value of multiple red pixel values ​​in the tooth region satisfying Equations 1 and 2, namely the first red pixel average value Rave, the average value of multiple green pixel values ​​in the tooth region, namely the first green pixel average value Gave, and the average value of multiple blue pixel values ​​in the tooth region, namely the first blue pixel average value Bave. The image processing unit 111 also adjusts the gain of at least two of the red, green, and blue components of the RGB image being processed, so that the first red pixel average value Rave, the first green pixel average value Gave, and the first blue pixel average value Bave are equal.

[0089] Specifically, the image processing unit 111 calculates the gain (red pixel gain) of multiple red pixel values ​​by dividing the average value of the first green pixel, Gave, by the average value of the first red pixel, Rave. Similarly, the image processing unit 111 calculates the gain (blue pixel gain) of multiple blue pixel values ​​by dividing the average value of the first green pixel, Gave, by the average value of the first blue pixel, Bave. Furthermore, the image processing unit 111 generates a third RGB image by multiplying the red pixels of each of the multiple second RGB pixels constituting the second RGB image by the red pixel gain and by multiplying the blue pixels of each of the multiple second RGB pixels by the blue pixel gain. In other words, the pixel values ​​of the multiple third RGB pixels constituting the third RGB image are pixel values ​​calculated by multiplying the red pixel values ​​of the multiple second RGB pixels constituting the second RGB image by the red pixel gain and by multiplying the blue pixel values ​​of the multiple second RGB pixels by the blue pixel gain. In addition, the image processing unit 111 calculates the gain for red pixels and the gain for blue pixels based on the average value of green pixels, and multiplies each gain by the pixel value of the corresponding color component to perform white balance adjustment processing. However, it is not limited to this. It can also calculate the gain for green pixels and the gain for blue pixels based on the average value of red pixels, or it can calculate the gain for red pixels and the gain for green pixels based on the average value of blue pixels.

[0090] In addition, if the pixel value exceeds the maximum value (1023 in the case of 10-bit display) by multiplying by the gain, the image processing unit 111 replaces the pixel value with 1023.

[0091] Furthermore, the white balance adjustment process shown here is one example; other methods can also be used. Similarly, the image processing described here is one example; other processing methods can also be used. For instance, image processing can also include exposure control processing, etc. Additionally, only a portion of the above processing can be performed, or no image processing may be performed at all. That is, the first RGB image can also be used for natural tooth region detection processing, etc., as described later.

[0092] The natural tooth region detection unit 112 detects the natural tooth region within the tooth region of the 4RGB image, based on the 4RGB image. The plaque region detection unit 113 detects the plaque-containing region within the 4RGB image, based on the 4RGB image. The image generation unit 114 generates a display image representing the plaque region. Specifically, the image generation unit 114 generates a display image that displays the plaque region but is limited to the natural tooth region. For example, the display image is an image in which the plaque region is overlaid on the image data (4RGB image).

[0093] The display unit 103 is a display device provided by the portable terminal 70, which displays the display image generated by the image generation unit 114.

[0094] The following uses Figures 6-14 Image generation processing ( Figure 4 The details of step S104 shown will be explained. Figure 6 It is a flowchart showing the details of image generation and processing.

[0095] Figure 7 This is a diagram representing an example of an RGB image (the 4th RGB image). Figure 7 The RGB image shown is an image obtained by photographing a natural tooth 201, an artificial tooth 202, and a gum 203 (dental ridge). The artificial tooth 202 is, for example, a restoration made of metals such as gold or silver. The natural tooth 201 is a natural tooth, the part of a tooth excluding the artificial tooth 202.

[0096] First, the natural tooth region detection unit 112 detects the first natural tooth region using an RGB image (the fourth RGB image) (S201). Specifically, the natural tooth region detection unit 112 detects regions in the RGB image that satisfy both a first condition and a second condition as the first natural tooth region. The first condition is that the green pixel value (G) is greater than or equal to a preset first threshold, and the second condition is that the difference between the green pixel value (G), the red pixel value (R), and the blue pixel value (B) is less than a preset third threshold.

[0097] Here, when natural teeth are irradiated with excitation light (blue light), excitation fluorescence is emitted from the dentin. This excitation fluorescence passes through the enamel. As a result, natural teeth fluoresce green. Furthermore, the filling material of caries treatment marks, when irradiated with blue light, differs from its state when irradiated with white light, appearing darker (low brightness) in the image captured by the camera. On the other hand, natural teeth covered by enamel appear brighter (high brightness) in the image. Therefore, by extracting this green fluorescence according to the first condition, it is possible to identify areas of natural teeth and exclude areas of artificial teeth.

[0098] Furthermore, the second condition specifically includes the following conditions: (1) the absolute value of the difference between the red pixel value (R) and the green pixel value (G) (abs(RG)) is less than the threshold rg_th, and (2) the absolute value of the difference between the green pixel value (G) and the blue pixel value (B) (abs(GB)) is less than the threshold gb_th, and (3) the absolute value of the difference between the blue pixel value (B) and the red pixel value (R) (abs(BR)) is less than the threshold br_th. In addition, the thresholds rg_th, gb_th, and br_th can be the same value or different values.

[0099] Alternatively, in condition 1, the luminance value (Y) can be used instead of the green pixel value (G). The luminance value (Y) is calculated using Equation 3 below.

[0100] Y=0.21 R+0.72 G+0.07 B (Equation 3) In Equation 3, Y is the luminance value, R is the red pixel value, G is the green pixel value, and B is the blue pixel value. As shown in Equation 3, the proportion of green pixel values ​​is large in the luminance value, so the same detection can be performed even when using the luminance value as when using the green pixel value.

[0101] Furthermore, this example shows the use of both condition 1 and condition 2, but it is also possible to use only either condition 1 or condition 2. For example, it is also possible to use only condition 1.

[0102] Figure 8 This is a diagram showing an example of the first natural tooth region 211 that was examined. Additionally, in Figure 8 In the illustration, the shape of teeth, etc., is shown in addition to the first natural tooth region 211 for illustrative purposes; however, the shape of teeth, etc., may not be included in the information obtained from the examination. Regarding this point, Figures 9-13 The same.

[0103] like Figure 8As shown, the region including the area of ​​natural teeth 201 and the area of ​​gingiva 203 is detected as the first natural tooth region 211. That is, the first natural tooth region 211 does not include the area of ​​artificial teeth 202.

[0104] Furthermore, an example is shown here in which the first natural tooth region 211 includes the entire region of the gingiva 203. However, depending on the photographic conditions, there may be cases where only a portion of the region of the gingiva 203 is included in the first natural tooth region 211, or cases where the region of the gingiva 203 is not included in the first natural tooth region 211.

[0105] Next, the natural tooth region detection unit 112 generates an HSV image by converting the color space of the RGB image (the 4th RGB image) to the HSV space (S202). Then, the natural tooth region detection unit 112 uses the HSV image to detect the gingival region (S203). Specifically, the natural tooth region detection unit 112 detects regions whose hue (H) falls within a predetermined range as gingival regions.

[0106] Figure 9 This is a diagram showing an example of the detected gingival region 212. (See diagram below.) Figure 9 As shown, the area of ​​the gingiva 203 is detected as gingival region 212. Furthermore, an example of detecting the gum area is shown here, but in addition to the gum area, the lips and other areas can also be detected as gingival region 212.

[0107] Next, the natural tooth region detection unit 112 determines the region obtained from the first natural tooth region 211 by excluding the gingival region 212 as the second natural tooth region 213 (S204). Figure 10 This is a diagram showing an example of region 213 of the second natural tooth. (See diagram.) Figure 10 As shown, the second natural tooth region 213 includes the region of natural teeth 201, but excludes the regions of artificial teeth 202 and gingiva 203.

[0108] Next, the natural tooth region detection unit 112 determines the third natural tooth region by excluding small regions from the second natural tooth region 213 (S205). Here, a small region is, for example, a region with an area smaller than a preset value. As a result, regions with high brightness caused by reflected light from areas other than the natural tooth region (such as the gum region) are incorrectly identified as natural tooth regions.

[0109] Next, the natural tooth region detection unit 112 determines the fourth natural tooth region by expanding the third natural tooth region (S206). Specifically, the natural tooth region detection unit 112 determines the fourth natural tooth region as the area obtained by expanding the boundary of the third natural tooth region outward by a predetermined amount. As a result, the boundary between the tooth and the gum can be included in the natural tooth region, and excessive obscuration of tartar areas can be prevented.

[0110] Next, the plaque area detection unit 113 detects the first plaque area (S207). Specifically, the plaque area detection unit 113 detects the area containing one or more pixels among the plurality of pixels in the HSV image generated in step S202 that satisfy at least one of the following conditions as the first plaque area: chroma is within a first predetermined range (e.g., 30 or more and 80 or less in 8-bit representation); hue is within a second predetermined range (e.g., 140 or more and 170 or less in 8-bit representation); and lightness is within a third predetermined range (e.g., 100 or more and 180 or less in 8-bit representation). Furthermore, the first, second, and third predetermined ranges can be determined by comparing the actual plaque area, the tooth area, and the HSV image, and are not limited to the numerical ranges described above.

[0111] In addition, the range of values ​​for chroma, hue, and lightness can be determined by applying a staining agent to tartar and comparing it with the degree of staining by the tartar staining agent.

[0112] Figure 11 This is a diagram representing an example of the first detected tartar region 214. (See diagram for example.) Figure 11 As shown, the area containing dental plaque was detected as the first plaque area 214. Furthermore, in this example, the first plaque area 214 includes the area of ​​the artificial tooth 202. That is, the area of ​​the artificial tooth 202 was mistakenly detected as a plaque area.

[0113] Furthermore, while an example of detecting the presence of dental plaque in the first plaque region 214 (binary) is shown here, the amount of plaque can also be detected (multi-valued). For example, by comparing the intensity of red fluorescence per unit area of ​​the first plaque region 214, the accumulation level (concentration or density) of the fluorescent substance can be detected. For example, the plaque region detection unit 113 can also detect the accumulation level based on the brightness V value of the HSV image of the first plaque region 214. In addition, the plaque region detection unit 113 can also perform grayscale display corresponding to the accumulation level.

[0114] Next, the image generation unit 114 generates a display image by extracting the first tartar region 214 within the region of the fourth natural tooth (S208). Specifically, the image generation unit 114 generates a mask image that sets the region of the fourth natural tooth to a value of 1 (white) and sets all other regions to a value of 0 (black). Figure 12 This is a diagram representing an example of mask image 215. For example... Figure 12 As shown, in the mask image 215, the area of ​​the natural tooth 201 has a value of 1 (white), and the area outside the natural tooth 201 has a value of 0 (black).

[0115] Next, the image generation unit 114 calculates the mask image 215 and... Figure 11 The second tartar region is calculated by logically multiplying the image representing the first tartar region 214. That is, the image generation unit 114 extracts the region within the fourth natural tooth region from the first tartar region 214 as the second tartar region. In other words, the image generation unit 114 determines the second tartar region by excluding regions within the first tartar region 214 that are not included in the fourth natural tooth region.

[0116] Figure 13 This is a diagram showing an example of the second tartar region. (See diagram below.) Figure 13 As shown, the area obtained by excluding the area of ​​artificial tooth 202 from the first tartar area 214 is determined as the second tartar area 216.

[0117] Next, the image generation unit 114 generates the first RGB image ( Figure 7 The image overlaid the second tartar region 216. Figure 14 This is a diagram showing an example of an image. For example... Figure 14 As shown, by displaying information indicating tartar areas on the image of the teeth, users can easily identify areas that have been missed during brushing. Furthermore, the aforementioned processing helps to prevent the misdetection of restorations as tartar.

[0118] also, Figure 6 The process shown is an example, and this disclosure is not limited thereto. For example, the timing of performing tartar detection treatment (S207) can be any timing prior to step S208. Furthermore, it is not necessary to perform... Figure 6 All of the processes shown may be omitted, or some processes may be omitted. For example, at least some of the processes in steps S203 to S206 may be omitted.

[0119] Furthermore, while the above description illustrates examples of artificial teeth made of metals such as gold or silver, artificial teeth are not limited to metals and can also be made of ceramics or zirconia. As described above, when natural teeth are irradiated with excitation light (blue light), the natural tooth region is detected by utilizing the green fluorescence of the natural teeth. Therefore, artificial teeth other than those made of metal can also be excluded from the natural tooth region. On the other hand, the above method may not be able to exclude artificial teeth made of all materials from the natural tooth region. The above method can exclude artificial teeth made of at least some materials (e.g., metals) from the natural tooth region, thereby achieving the effect of suppressing false detections compared to not using this method.

[0120] Furthermore, in the above embodiment, the intraoral camera 10 sends a first RGB image to the portable terminal 70, where processing of the first RGB image is performed, but this is not limited to this. Alternatively, the structure could be as follows: the first RGB image is sent to the cloud server 80, the cloud server 80 performs the aforementioned processing, and sends the resulting second RGB image or fourth RGB image to the portable terminal 70. In this case, the first RGB image can be sent from the intraoral camera 10 to the cloud server 80 without going through the portable terminal 70, or it can be sent from the intraoral camera 10 to the cloud server 80 via the portable terminal 70.

[0121] As described above, the image processing method of this embodiment displays an image (e.g., a first RGB image) of teeth and plaque that fluoresces when illuminated with light in the blue light band, based on an image obtained by capturing such an image. The plaque area is then displayed, and natural tooth areas within the tooth area are detected based on the image (e.g., natural tooth areas within the tooth area). Figure 4 S104, Figure 6 (S201~S206), and detect tartar areas based on images (e.g., Figure 6 S207), limited to areas of natural teeth, showing areas of tartar (e.g., Figure 4 (S105). Therefore, this image processing method can suppress the misdetection of repairs and other materials as dental plaque. Thus, this image processing method can improve the detection accuracy of dental plaque.

[0122] For example, in the detection of the natural tooth region, this image processing method detects a first region (e.g., a first natural tooth region) and detects other natural tooth regions based on the first region (e.g., Figure 6 In S201 of the image, the first region is (i) a region in the entire pixel region of the image whose brightness value is above a preset first threshold, or (ii) a region in the entire pixel region of the image whose green pixel value is above a preset second threshold. Thus, this image processing method can detect natural tooth regions with high precision using either brightness value or green pixel value.

[0123] For example, in the detection of natural tooth regions, this image processing method detects regions in the first region where the difference between the green pixel value, red pixel value, and blue pixel value is less than a pre-set third threshold as natural tooth regions (e.g. Figure 6 (S201). Thus, this image processing method can use the differences between green pixel values, red pixel values, and blue pixel values ​​to detect natural tooth regions with high precision.

[0124] For example, in the detection of the natural tooth region, this image processing method detects the gingival region based on the hue of the image (e.g., Figure 6 In S203), the region obtained by excluding the gingival region from the first region is detected as the natural tooth region (e.g., Figure 6 (S204 in the image processing method). Therefore, this image processing method can suppress the gingival region from being incorrectly identified as a natural tooth region, thus improving the detection accuracy of natural tooth regions.

[0125] For example, in the detection of natural tooth regions, this image processing method identifies regions obtained by excluding areas smaller than a preset area from the first region as natural tooth regions (e.g., Figure 6 (S205). Therefore, this image processing method can suppress the misidentification of high-brightness areas caused by reflected light, etc., outside the natural tooth area as natural tooth areas, thus improving the detection accuracy of natural tooth areas.

[0126] For example, in the detection of the natural tooth region, this image processing method detects the region obtained by extending the boundary of the first region outward by a predetermined amount as the natural tooth region (e.g., Figure 6 (S206). Thus, this image processing method can suppress the excessive underdisplay of tartar areas by expanding the natural tooth area.

[0127] Furthermore, the image processing apparatus of this embodiment (e.g., portable terminal 70) displays a plaque area based on an image (e.g., a first RGB image) obtained by capturing a tooth and plaque that fluoresces when illuminated with light in the blue light band. It includes: a natural tooth area detection unit 112 that detects natural tooth areas within a tooth area based on the image; a plaque area detection unit 113 that detects plaque areas based on the image; and a display unit 103 that displays the plaque area only within the natural tooth area. Therefore, this image processing apparatus can suppress the misdetection of repairs or the like as plaque. Thus, this image processing apparatus can improve the detection accuracy of plaque.

[0128] The intraoral camera system according to the embodiments of the present disclosure has been described above, but the present disclosure is not limited to this embodiment.

[0129] For example, the above description illustrates an example using an intraoral camera 10 primarily for photographing teeth, but the intraoral camera 10 could also be an intraoral care device equipped with a camera. For example, the intraoral camera 10 could also be an intraoral cleaning machine equipped with a camera, etc.

[0130] Furthermore, the processing units included in the intraoral camera system of the above embodiments are typically implemented as LSIs (Liquid Crystal Sensors) of integrated circuits. They can be implemented individually on a single chip, or in a manner that includes some or all of them on a single chip.

[0131] Furthermore, integrated circuitry is not limited to LSIs; it can also be achieved through dedicated circuits or general-purpose processors. Alternatively, FPGAs (Field Programmable Gate Arrays) that can be programmed after LSI manufacturing, or reconfigurable processors that can reconfigure the connections and settings of the internal circuitry units of the LSI, can be used.

[0132] Furthermore, in the above embodiments, each component may be constructed using dedicated hardware, or implemented by executing software programs suitable for each component. Each component may also be implemented by a program execution unit such as a CPU or processor reading and executing software programs recorded on a recording medium such as a hard disk or semiconductor memory.

[0133] Furthermore, this disclosure can also be implemented as an image processing method performed by an intraoral camera system. Additionally, this disclosure can also be implemented as an intraoral camera, a portable terminal, or a cloud server included in an intraoral camera system.

[0134] Furthermore, the segmentation of functional blocks in the block diagram is one example. Multiple functional blocks can also be implemented as a single functional block, or a single functional block can be divided into multiple functional blocks, or some functionality can be transferred to other functional blocks. Additionally, the functionality of multiple functional blocks with similar functions can be processed in parallel or time-sharing by a single piece of hardware or software.

[0135] Furthermore, the order of the steps in the execution flowchart is illustrative for the purpose of explaining this disclosure, and may be in a different order than described above. Additionally, some of the steps described above may be executed simultaneously (in parallel) with other steps.

[0136] The above description describes one or more intraoral camera systems based on embodiments, but this disclosure is not limited to these embodiments. Various modifications conceived by those skilled in the art to these embodiments, and forms constructed by combining constituent elements from different embodiments, can also be included within the scope of these one or more embodiments, provided they do not depart from the spirit of this disclosure.

[0137] Industrial applicability This disclosure can be applied to intraoral camera systems.

[0138] Explanation of reference numerals in the attached figures 10 Intraoral Camera 10a Head 10b Handle section 10c Neck 12. Photographic Optical System 12a Inlet 14. Camera components 16 lenses 18 mirrors 20 Blue Light Cut-off Filter 24 aperture 26A, 26B, 26C, 26D LEDs 28 masks 30 Composition Adjustment Mechanism 32. Focus adjustment mechanism 34 Frame 36, 40 actuators 38 Lens holder 50 Central Control Department 54 LED Control Department 56 Lens Driver 58 Wireless Communication Module 60 Power Control Unit 62 controllers 64 Memory 66 batteries 68 coils 69 Charger 70 Portable Terminals 72 Touchscreen 80 cloud servers 90 Position Sensor 101 Obtained Department 102 Production Department 103 Display Department 111 Image Processing Department 112 Natural Tooth Area Examination Department 113 Dental Plaque Area Detection Department 114 Image Generation Unit 201 Natural Teeth 202 Artificial Teeth 203 Gum 211 First natural tooth region 212 Gingival region 213 Second natural tooth region 214 First tartar area 215 Mask Image 216 Second tartar region

Claims

1. An image processing method based on an image obtained by photographing a tooth and tartar exhibiting fluorescence, displaying a tartar area, wherein the fluorescence is induced by irradiating the tooth with light containing a blue light wavelength, wherein... Based on the image, detect the natural tooth region within the area of ​​the tooth. Based on the image, areas of dental plaque are detected. Limited to the area of ​​the natural teeth, showing the tartar area.

2. The image processing method according to claim 1, wherein, In the detection of the natural tooth region, a first region is detected, and the natural tooth region is detected based on the first region. The first region is (i) a region in the entire pixel region of the image whose brightness value is above a preset first threshold, or (ii) a region in the entire pixel region of the image whose green pixel value is above a preset second threshold.

3. The image processing method according to claim 2, wherein, In the detection of the natural tooth region, the region in the first region where the difference between the green pixel value, red pixel value and blue pixel value is less than a preset third threshold is detected as the natural tooth region.

4. The image processing method according to claim 2 or 3, wherein, In the detection of the natural tooth area, Based on the hue of the image, the gingival region is detected. The region obtained by excluding the gingival region from the first region is detected as the natural tooth region.

5. The image processing method according to claim 2 or 3, wherein, In the detection of the natural tooth region, the region obtained by excluding regions with an area smaller than a preset area from the first region is detected as the natural tooth region.

6. The image processing method according to claim 2 or 3, wherein, In the detection of the natural tooth region, the region obtained by extending the boundary of the first region outward by a predetermined amount is detected as the natural tooth region.

7. An image processing apparatus, based on an image obtained by photographing a tooth and plaque exhibiting fluorescence, displays a plaque area, wherein the fluorescence is induced by irradiating the tooth with light containing a blue light wavelength, wherein... have: The natural tooth region detection unit detects natural tooth regions within the tooth region based on the image. The plaque area detection unit detects plaque areas based on the image. as well as The display section, limited to the area of ​​the natural teeth, displays the area of ​​tartar.

8. A program for causing a computer to perform the image processing method of claim 1.

Images