Plaque detection device, plaque detection method and program

Abstract

This dental plaque detection device comprises: an acquisition unit (101) for acquiring a first RGB image from reflected light and fluorescence from teeth, dental plaque, and tartar in an oral cavity irradiated with irradiation light of a predetermined wavelength; and a detection unit (102) for generating a second RGB image by performing image processing including first image processing on the first RGB image, and detecting the amount contained, per unit area, of a fluorescent substance contained in tartar and dental plaque attached to the teeth, on the basis of the second RGB image. The first image processing is processing for: extracting a natural tooth region to which dental plaque and tartar are not attached from the first RGB image; and adjusting the gain of at least two color components from among the red, green, and blue components of the first RGB image such that a first red pixel average value of a plurality of red pixel values of a plurality of first pixels constituting the natural tooth region, a first green pixel average value of a plurality of green pixel values of the plurality of first pixels, and a first blue pixel average value of a plurality of blue pixel values of the plurality of first pixels are made equal to each other.

Description

[Technical Field]

[0001] The present disclosure relates to a plaque detection device, a plaque detection method, and a program. [Background technology]

[0002] Patent Document 1 discloses a device for detecting dental plaque based on photographs of teeth in the oral cavity. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-248220 Summary of the Invention [Problem to be solved by the invention]

[0004] It is desirable for such a plaque detection device to be able to detect the state of plaque in more detail.

[0005] Therefore, the present disclosure provides a plaque detection device or a plaque detection method that can detect the condition of teeth in detail. [Means for solving the problem]

[0006] A plaque detection device according to one aspect of the present disclosure includes an acquisition unit that acquires a first RGB image from reflected light and fluorescence from teeth, plaque, and tartar in an oral cavity that is irradiated with irradiation light of a predetermined wavelength that excites fluorescent substances contained in the plaque and tartar; and a detection unit that generates a second RGB image by performing image processing including first image processing on the first RGB image, and detects the amount of fluorescent substance contained in the plaque and tartar attached to the teeth per unit area based on the fluorescence intensity value of the fluorescent reaction of the fluorescent substance in the second RGB image, wherein the first image processing is a process of extracting a natural tooth region free of plaque and tartar from the first RGB image, and adjusting gains of at least two color components of the red, green, and blue components of the first RGB image so that a first red pixel average value of multiple red pixel values ​​of multiple first pixels constituting the natural tooth region, a first green pixel average value of multiple green pixel values ​​of the multiple first pixels, and a first blue pixel average value of multiple blue pixel values ​​of the multiple first pixels are equal. [Effects of the Invention]

[0007] The present disclosure can provide a plaque detection device or a plaque detection method that can detect the condition of teeth in detail. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a perspective view of an intraoral camera in an intraoral camera system according to an embodiment. [Figure 2] FIG. 2 is a cross-sectional view schematically showing a photographing optical system incorporated in the intraoral camera in the intraoral camera system according to the embodiment. [Figure 3] FIG. 3 is a schematic configuration diagram of an intraoral camera system according to an embodiment. [Figure 4] FIG. 4 is a diagram showing the flow of operations in the intraoral camera system according to the embodiment. [Figure 5] FIG. 5 is a functional block diagram of the mobile terminal according to the embodiment. [Figure 6] FIG. 6 is a diagram showing an example of teeth in an oral cavity according to an embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of a layered model according to the embodiment. [Figure 8] FIG. 8 is a diagram for explaining the illuminance of the fluorescent light at a depth D according to the embodiment. [Figure 9] FIG. 9 is a diagram for explaining the illuminance of fluorescent light observed from dental plaque having a thickness D0 according to the embodiment. [Figure 10] FIG. 10 is a flowchart of a process for detecting the concentration distribution of a fluorescent material according to the embodiment. [Figure 11] FIG. 11 is a diagram showing an example of a fourth RGB image according to the embodiment. [Figure 12] FIG. 12 is a diagram illustrating an example of a fourth RGB image according to the embodiment. [Figure 13] FIG. 13 is a diagram showing the relationship between the fluorescence intensity, MIN, and k according to the embodiment. [Figure 14] FIG. 14 is a diagram showing the relationship between the fluorescence intensity, S (saturation), and k according to the embodiment. [Figure 15] FIG. 15 is a diagram showing the relationship between the fluorescence intensity, L (brightness), and k according to the embodiment. [Figure 16] FIG. 16 is a diagram showing an example of pixel values ​​in each image when the blue light region is attenuated by signal processing according to the embodiment. [Figure 17] FIG. 17 is a diagram showing an example of pixel values ​​in each image when the blue light region is attenuated by signal processing according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Furthermore, a plaque detection device according to an aspect of the present disclosure includes an acquisition unit that acquires a first RGB image from reflected light and fluorescence from teeth, plaque, and tartar in an oral cavity that is irradiated with irradiation light of a predetermined wavelength that excites fluorescent substances contained in the plaque and tartar; and a detection unit that generates a second RGB image by performing image processing including a first image processing on the first RGB image, and detects the amount of fluorescent substance contained in the plaque and tartar attached to the teeth per unit area based on the fluorescence intensity value of the fluorescent reaction of the fluorescent substance in the second RGB image, wherein the first image processing is a process of extracting a natural tooth region free of plaque and tartar from the first RGB image, and adjusting gains of at least two color components of the red, green, and blue components of the first RGB image so that a first red pixel average value of a plurality of red pixel values ​​of a plurality of first pixels constituting the natural tooth region, a first green pixel average value of a plurality of green pixel values ​​of the plurality of first pixels, and a first blue pixel average value of a plurality of blue pixel values ​​of the plurality of first pixels are equal.

[0010] This allows the plaque detection device to detect the amount of fluorescent substance contained in plaque and tartar attached to the teeth per unit area, thereby enabling detailed detection of the condition of the teeth. Furthermore, by performing the first image processing, the plaque detection device can adjust the white balance of the first RGB image in which the fluorescently reacting teeth are captured. Therefore, the plaque detection device can generate a second RGB image in which plaque regions, which are areas of the teeth where plaque is attached, can be easily distinguished. Therefore, the plaque detection device can improve the detection accuracy of the amount of fluorescent substance contained in plaque and tartar attached to the teeth per unit area. Furthermore, the plaque detection device can improve the accuracy of the white balance adjustment process by performing the first image processing using pixels of natural tooth regions where plaque and tartar are not attached.

[0011] For example, in the extraction of the natural tooth region, a first region may be detected as follows: (i) a region in which a luminance value is equal to or greater than a predetermined first threshold value in the entire pixel region of the first RGB image, or (ii) a region in which a green pixel value is equal to or greater than a predetermined second threshold value in the entire pixel region of the first RGB image, and the natural tooth region may be extracted based on the first region. This allows the plaque detection device to accurately detect the natural tooth region using the luminance value or the green pixel value.

[0012] For example, in extracting the natural tooth region, an area obtained by excluding areas of plaque and tartar from the first area may be extracted as the natural tooth region, thereby improving the accuracy of the white balance adjustment process in the plaque detection device.

[0013] For example, the first RGB image may be an image in which at least a part of the blue light region is attenuated from the reflected light and fluorescence from the teeth and plaque in the oral cavity. In this way, the plaque detection device can improve the detection accuracy of the content per unit area of ​​the fluorescent substance contained in the plaque and tartar attached to the teeth by using the first image in which at least a part of the blue light region is attenuated from the reflected light and fluorescence from the teeth, plaque, and tartar in the oral cavity that is irradiated with irradiation light of a predetermined wavelength that excites the fluorescent substance contained in the plaque and tartar.

[0014] For example, the detection unit may generate an HSV image from the second RGB image and detect the amount of fluorescent material contained in the plaque and tartar attached to the teeth per unit area from the brightness value of the HSV image. In this way, the plaque detection device can accurately detect the amount of fluorescent material contained in the plaque and tartar attached to the teeth per unit area based on the brightness value of the HSV image.

[0015] For example, the detection unit may identify a specific pixel area in which one or more fourth pixels of the HSV image that satisfy at least one of a first predetermined range for saturation, a second predetermined range for hue, and a third predetermined range for brightness are located, and detect the amount of the fluorescent substance contained in the plaque and tartar attached to the teeth per unit area from the brightness value in the specific pixel area.In this way, the plaque detection device can improve the detection accuracy of the amount of the fluorescent substance contained in the plaque and tartar attached to the teeth per unit area by identifying the plaque area in the tooth image and then detecting the amount of the fluorescent substance contained in the plaque and tartar attached to the teeth per unit area.

[0016] For example, the detection unit may generate an HSL image from the second RGB image and detect the amount of fluorescent material contained in the plaque and tartar attached to the teeth per unit area from the brightness value of the HSL image. In this way, the plaque detection device can accurately detect the amount of fluorescent material contained in the plaque and tartar attached to the teeth per unit area based on the brightness value of the HSL image.

[0017] For example, the detection unit may identify a specific pixel area in which one or more fifth pixels of the plurality of fifth pixels in the HSL image are located, the fifth pixels having saturation within a fourth predetermined range, hue within a fifth predetermined range, and luminance within a sixth predetermined range, and detect the amount of the fluorescent substance contained in the plaque and tartar attached to the teeth per unit area from the luminance value in the specific pixel area. In this way, the plaque detection device can improve the detection accuracy of the amount of the fluorescent substance contained in the plaque and tartar attached to the teeth per unit area by identifying the plaque area in the tooth image and then detecting the amount of the fluorescent substance contained in the plaque and tartar attached to the teeth per unit area.

[0018] For example, the fluorescent substance may be porphyrin. For example, the detection unit may assign the content per unit area of ​​the fluorescent substance contained in the plaque and tartar attached to the tooth to three or more gradations, and generate a third image by superimposing the content per unit area of ​​the fluorescent substance contained in the plaque and tartar attached to the tooth, displayed in gradations, on a second image based on the first RGB image. In this way, for example, the generated third image can notify the user of the content per unit area of ​​the fluorescent substance contained in the plaque and tartar attached to the tooth.

[0019] For example, the plaque detection device may further include an identification unit that identifies the type of photographed tooth, and a memory unit that stores the amount of fluorescent substance contained in plaque and tartar attached to the tooth per unit area detected from the photographed tooth in association with the identified tooth type. This allows the plaque detection device to manage the amount of fluorescent substance contained in plaque and tartar attached to the tooth per unit area for each tooth.

[0020] Furthermore, a plaque detection method according to one aspect of the present disclosure includes: acquiring a first RGB image from reflected light and fluorescence from teeth, plaque, and tartar in an oral cavity that is irradiated with irradiation light of a predetermined wavelength that excites fluorescent substances contained in the plaque and tartar; generating a second RGB image by performing image processing including first image processing on the first RGB image; and detecting the amount of fluorescent substance contained in the plaque and tartar attached to the teeth per unit area based on the fluorescence intensity value of the fluorescent reaction of the fluorescent substance in the second RGB image; the first image processing is a process of extracting a natural tooth region free of plaque and tartar from the first RGB image; and adjusting gains of at least two color components of the red, green, and blue components of the first RGB image so that a first red pixel average value of multiple red pixel values ​​of multiple first pixels constituting the natural tooth region, a first green pixel average value of multiple green pixel values ​​of the multiple first pixels, and a first blue pixel average value of multiple blue pixel values ​​of the multiple first pixels are equal.

[0021] According to this, the plaque detection method can detect the amount of fluorescent substance contained in plaque and tartar attached to the teeth per unit area, thereby enabling detailed detection of the condition of the teeth. Furthermore, the plaque detection method can adjust the white balance of the first RGB image in which the fluorescently reacting teeth are photographed by performing the first image processing. Therefore, the plaque detection method can generate a second RGB image in which the plaque region, which is the region of the tooth where plaque is attached, can be easily distinguished. Therefore, the plaque detection method can improve the detection accuracy of the amount of fluorescent substance contained in plaque and tartar attached to the teeth per unit area. Furthermore, the plaque detection method can improve the accuracy of the white balance adjustment process by performing the first image processing using pixels of the natural tooth region where plaque and tartar are not attached.

[0022] Furthermore, a program according to one aspect of the present disclosure is a program for causing a computer to execute the dental plaque detection method.

[0023] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

[0024] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of well-known matters or redundant explanation of substantially the same configuration may be omitted. This is to avoid unnecessary redundancy in the following explanation and to facilitate understanding by those skilled in the art.

[0025] The inventors have provided the accompanying drawings and the following description to enable those skilled in the art to fully understand the present disclosure, and do not intend for them to limit the subject matter described in the claims.

[0026] (Embodiment) Fig. 1 is a perspective view of an intraoral camera in an intraoral camera system according to the present embodiment. As shown in Fig. 1, the intraoral camera 10 has a toothbrush-shaped housing that can be handled with one hand, and the housing includes a head 10a that is placed in the user's oral cavity when photographing the dentition, a handle 10b that the user holds, and a neck 10c that connects the head 10a and the handle 10b.

[0027] 2 is a cross-sectional view showing the imaging optical system 12 incorporated in the intraoral camera 10. In this embodiment, the imaging optical system 12 of the intraoral camera 10 is incorporated in the head portion 10a and the neck portion 10c as shown in FIG. 2. The imaging optical system 12 includes an image sensor 14 and a lens 16 arranged on its optical axis LA.

[0028] The imaging element 14 is an imaging device such as a C-MOS (Complementary Metal-Oxide-Semiconductor) sensor or a CCD (Charge Coupled Device) element, and an image of the tooth D is formed by the lens 16. The imaging element 14 outputs a signal (image data) corresponding to the formed image to the outside.

[0029] The lens 16 is, for example, a condenser lens, and forms an image of the tooth D incident thereon on the imaging element 14. The lens 16 may be a single lens or a lens group made up of a plurality of lenses.

[0030] In this embodiment, the photographing optical system 12 further includes a mirror 18 that reflects the image of the tooth D toward the lens 16, a blue light cut filter (blue blocking element) 20 arranged between the mirror 18 and the lens 16, and an aperture 24 arranged between the lens 16 and the image sensor 14.

[0031] The mirror 18 is disposed on the optical axis LA of the photographing optical system 12 so as to reflect the image of the tooth D that has passed through the entrance 12 a of the photographing optical system 12 toward the lens 16 .

[0032] The blue light cut filter 20 is a filter that cuts out light components of blue wavelengths contained in light that enters the image sensor 14. When light including a wavelength range of blue light is irradiated onto teeth to detect dental plaque, if the light including a wavelength range of blue light is increased to strengthen the excitation fluorescence of the plaque, the entire first RGB image will take on a blue tint. In this state, blue pixel values ​​become dominant over red and green pixel values, and the effect of making plaque regions easier to distinguish by performing image processing (exposure control processing and white balance adjustment processing) described below may be reduced. To address this issue, the blue light cut filter 20 cuts out light including a wavelength range of blue light from the light before it enters the image sensor 14.

[0033] The diaphragm 24 is a plate-like member with a through-hole on the optical axis LA of the imaging optical system 12, and realizes a deep focal depth. This allows the depth direction of the oral cavity to be brought into focus, and an image of the row of teeth with clear contours can be obtained.

[0034] The intraoral camera 10 is also equipped with a plurality of first to fourth LEDs 26A to 26D as lighting devices that irradiate light onto the tooth D to be photographed during photography. The first to fourth LEDs 26A to 26D are, for example, blue light-emitting diodes (LEDs). As shown in FIG. 1, in this embodiment, the first to fourth LEDs 26A to 26D are arranged to surround the entrance 12a. To prevent the gums G or the like from coming into contact with the first to fourth LEDs 26A to 26D and resulting in insufficient illumination light, a translucent cover 28 is provided on the head unit 10a to cover the first to fourth LEDs 26A to 26D and the entrance 12a. Some of the first to fourth LEDs 26A to 26D may be white LEDs. Using white LEDs for some of the first to fourth LEDs 26A to 26D can brighten the first RGB image and improve the balance of blue pixel values ​​relative to red and green pixel values.

[0035] Furthermore, in this embodiment, the intraoral camera 10 has a composition adjustment mechanism 30 and a focus adjustment mechanism 32, as shown in FIG.

[0036] The composition adjustment mechanism 30 is composed of a housing 34 that holds the image sensor 14 and the lens 16, and an actuator 36 that moves the housing 34 in the direction of extension of the optical axis LA. The actuator 36 adjusts the position of the housing 34 to adjust the angle of view, i.e., the size of the row of teeth imaged on the image sensor 14. The composition adjustment mechanism 30 automatically adjusts the position of the housing 34 so that, for example, one entire tooth is captured in the captured image. The composition adjustment mechanism 30 also adjusts the position of the housing 34 based on a user's operation so that the angle of view desired by the user is achieved.

[0037] The focus adjustment mechanism 32 is held within the housing 34 of the composition adjustment mechanism 30 and is composed of a lens holder 38 that holds the lens 16 and an actuator 40 that moves the lens holder 38 in the direction of extension of the optical axis LA. The actuator 40 adjusts the relative position of the lens holder 38 with respect to the image sensor 14, thereby adjusting the focus, i.e., the focal point. The focus adjustment mechanism 32 automatically adjusts the position of the lens holder 38 so that, for example, a tooth located at the center of the captured image is in focus. The focus adjustment mechanism 32 also adjusts the position of the lens holder 38 based on a user operation.

[0038] The components of the imaging optical system 12 except for the mirror 18 may be provided on the handle portion 10b of the intraoral camera 10.

[0039] The image output by the imaging element 14 is an RGB image in which each of the multiple pixels that make up the image has RGB sub-pixels.

[0040] The intraoral camera 10 is also equipped with a plurality of first to fourth LEDs 26A to 26D as lighting devices that irradiate light onto the teeth to be photographed during photography. The first to fourth LEDs 26A to 26D are, for example, blue LEDs that irradiate blue light having a wavelength with a peak at 405 nm. Note that the first to fourth LEDs 26A to 26D are not limited to blue LEDs and may be any light source that irradiates light including the wavelength range of blue light.

[0041] Fig. 3 is a schematic diagram of the intraoral camera system according to the present embodiment. As shown in Fig. 3, the intraoral camera system according to the present embodiment is generally configured to photograph the row of teeth using an intraoral camera 10 and perform image processing on the photographed image.

[0042] As shown in Fig. 3, the intraoral camera system includes an intraoral camera 10, a mobile terminal 70, and a cloud server 80. The mobile terminal 70 is, for example, a smartphone or tablet terminal capable of wireless communication. The mobile terminal 70 includes, as an input device and an output device, for example, a touch screen 72 capable of displaying an image of a row of teeth. The mobile terminal 70 functions as a user interface for the intraoral camera system.

[0043] The cloud server 80 is a server that can communicate with the mobile terminal 70 via the Internet or the like, and provides the mobile terminal 70 with an application for using the intraoral camera 10. For example, a user downloads the application from the cloud server 80 and installs it on the mobile terminal 70. The cloud server 80 also acquires dentition images captured by the intraoral camera 10 via the mobile terminal 70.

[0044] The intraoral camera 10 includes a central control unit 50 as the main part for controlling the system, an LED control unit 54 that controls multiple LEDs 26A to 26D, a lens driver 56 that controls the actuator 36 of the composition adjustment mechanism 30 and the actuator 40 of the focus adjustment mechanism 32, and a position sensor 90.

[0045] The intraoral camera 10 also includes a wireless communication module 58 that performs wireless communication with the mobile terminal 70, and a power supply control unit 60 that supplies power to the central control unit 50 and the like.

[0046] The central control unit 50 of the intraoral camera 10 is mounted, for example, on the handle unit 10b of the intraoral camera 10. For example, the central control unit 50 also includes a controller 62 such as a CPU (Central Processing Unit) or MPU (Micro Processing Unit) that executes various processes described below, and a memory 64 such as a RAM (Random Access Memory) or ROM (Read Only Memory) that stores programs for causing the controller 62 to execute the various processes. In addition to the programs, the memory 64 also stores a row-of-teeth image (image data) captured by the image sensor 14, various setting data, and the like. The row-of-teeth image captured by the image sensor 14 is an example of a first RGB image.

[0047] The controller 62 transmits the row-of-teeth image output from the imaging element 14 to the mobile terminal 70 via the wireless communication module 58. The mobile terminal 70 displays the transmitted row-of-teeth image on the touch screen 72, thereby presenting the row-of-teeth image to the user.

[0048] The LED control unit 54 is mounted on, for example, the handle portion 10b of the intraoral camera 10, and turns on and off the first to fourth LEDs 26A to 26D based on a control signal from the controller 62. The LED control unit 54 is configured, for example, by a circuit. For example, when a user performs an operation on the touch screen 72 of the mobile terminal 70 to activate the intraoral camera 10, a corresponding signal is transmitted from the mobile terminal 70 to the controller 62 via the wireless communication module 58. Based on the received signal, the controller 62 transmits a control signal to the LED control unit 54 to turn on the first to fourth LEDs 26A to 26D.

[0049] The lens driver 56 is mounted on, for example, the handle portion 10b of the intraoral camera 10, and controls the actuator 36 of the composition adjustment mechanism 30 and the actuator 40 of the focus adjustment mechanism 32 based on a control signal from a controller 62 of the central control unit 50. The lens driver 56 is configured, for example, by a circuit. For example, when a user performs an operation related to composition adjustment or focus adjustment on the touch screen 72 of the mobile terminal 70, a corresponding signal is transmitted from the mobile terminal 70 to the central control unit 50 via the wireless communication module 58. Based on the received signal, the controller 62 of the central control unit 50 transmits a control signal to the lens driver 56 to adjust the composition or focus. Furthermore, for example, the controller 62 calculates the control amount of the actuator 36 or 40 required for the composition adjustment or focus adjustment based on the row-of-teeth image from the imaging element 14, and transmits a control signal corresponding to the calculated control amount to the lens driver 56.

[0050] The wireless communication module 58 is mounted on, for example, the handle portion 10b of the intraoral camera 10, and performs wireless communication with the mobile terminal 70 based on a control signal from the controller 62. The wireless communication module 58 performs wireless communication with the mobile terminal 70 in accordance with an existing communication standard such as WiFi (registered trademark) or Bluetooth (registered trademark). Via the wireless communication module 58, the intraoral camera 10 transmits a row-of-dentition image showing the teeth D to the mobile terminal 70, and operation signals are transmitted from the mobile terminal 70 to the intraoral camera 10.

[0051] In this embodiment, the power supply control unit 60 is mounted on the handle unit 10b of the intraoral camera 10 and distributes power from a battery 66 to the central control unit 50, the LED control unit 54, the lens driver 56, and the wireless communication module 58. The power supply control unit 60 is configured, for example, by a circuit. In this embodiment, the battery 66 is a rechargeable secondary battery, and is wirelessly charged by an external charger 69 connected to a commercial power source via a coil 68 mounted on the intraoral camera 10.

[0052] The position sensor 90 is a sensor for detecting the posture and position of the intraoral camera 10, and is, for example, a multi-axis (here, three axes: x, y, and z) acceleration sensor. For example, the position sensor 90 may be a six-axis sensor having a three-axis acceleration sensor and a three-axis gyro sensor. For example, as shown in FIG. 1 , the z-axis coincides with the optical axis LA. The y-axis is parallel to the imaging surface and extends in the longitudinal direction of the intraoral camera 10. The x-axis is parallel to the imaging surface and perpendicular to the y-axis. Outputs of each axis of the position sensor 90 may be transmitted to the mobile terminal 70 via the central control unit 50 and the wireless communication module 58.

[0053] The position sensor 90 may be a piezo-resistive, capacitive, or thermal-detection MEMS (Micro Electro Mechanical Systems) sensor. Although not shown, a correction circuit may be provided to correct the balance of sensitivity of the sensors for each axis, the temperature characteristics of sensitivity, or temperature drift. A band-pass filter (low-pass filter) may also be provided to remove dynamic acceleration components or noise. Noise may also be reduced by smoothing the output waveform of the acceleration sensor.

[0054] Next, the operation of the intraoral camera system will be described. Fig. 4 is a diagram showing the flow of operation in the intraoral camera system. Note that the processing shown in Fig. 4 is, for example, processing performed in real time, and is performed every time one frame or multiple frames of image data are obtained.

[0055] A user uses the intraoral camera 10 to capture images of the teeth and gums in their oral cavity, generating image data (S101). This image data is obtained by, for example, irradiating the teeth with light including a wavelength range of blue light and capturing images of the teeth that are undergoing a fluorescent reaction. Next, the intraoral camera 10 transmits the captured image data to the mobile terminal 70 (S102). Note that the image data may be a video or one or more still images. If the image data is a video or multiple still images, the sensor data may be transmitted for each frame of the video or for each still image. If the image data is a video, the sensor data may be transmitted for each multiple frame.

[0056] The image data may be transmitted in real time, or may be transmitted all at once after a series of photographs (for example, photographs of all teeth in the oral cavity) have been taken.

[0057] The mobile terminal 70 performs image processing on the received image data (S103), and detects the concentration distribution of the fluorescent substance using the processed image data (S104). Next, the mobile terminal 70 generates an image in which the detected concentration distribution of the fluorescent substance is superimposed on an image of the oral cavity (S105), and displays the generated image (S106).

[0058] By using such an intraoral camera system, a user can take an image of the inside of their own oral cavity with the intraoral camera 10 and check the condition of the inside of their oral cavity displayed on the mobile terminal 70. Furthermore, the concentration distribution of the fluorescent substance is shown in the displayed image, so that the user can easily check the health condition of their own teeth.

[0059] Furthermore, the mobile terminal 70 may generate a three-dimensional model of a plurality of teeth in the oral cavity from a plurality of captured image data, and may display an image based on the generated three-dimensional model.

[0060] Here, an example in which the mobile terminal 70 processes an image of teeth is described, but part or all of this processing may be performed by the intraoral camera 10. The mobile terminal 70 is an example of a dental plaque detection device.

[0061] FIG. 5 is a functional block diagram of the mobile terminal 70. The mobile terminal 70 includes an acquisition unit 101, a detection unit 102, a display unit 103, an identification unit 104, and a storage unit 105.

[0062] The acquisition unit 101 acquires the image data (first RGB image) transmitted from the intraoral camera 10. The acquisition unit 101 may acquire sensor data in addition to the image data from the intraoral camera 10. The first RGB image is an image obtained by the intraoral camera 10 photographing teeth that fluoresce by irradiating the teeth with light including the wavelength range of blue light. Here, the blue light is an example of irradiation light having a predetermined wavelength that excites the fluorescent substance contained in dental plaque. The fluorescent substance is, for example, porphyrin.

[0063] The detection unit 102 may generate a third RGB image by performing exposure control processing (second image processing) on the first RGB image, and generate a second RGB image by performing white balance adjustment processing (first image processing) on the third RGB image.

[0064] (Exposure control processing) In the exposure control processing, the detection unit 102 first extracts a plurality of pixels among the plurality of first RGB pixels (third pixels) constituting the first RGB image whose RGB values satisfy the following equations 1 and 2.

[0065] min(R,G,B)≦Ths, and, max(R,G,B)<Thmax (Equation 1)

[0066] Gmax - G≦Thb (Equation 2)

[0067] min(R,G,B) indicates the minimum value among the pixel values of the three sub-pixels of RGB respectively possessed by the first RGB pixel (that is, the red pixel value, the green pixel value, and the blue pixel value).

[0068] Ths is a threshold value for excluding areas in the first RGB image that are strongly affected by reflection of the irradiating light (for example, glossy areas), and is, for example, 900 in 10-bit representation.

[0069] max(R, G, B) indicates the maximum value among the pixel values ​​of the three RGB sub-pixels (that is, the red pixel value, green pixel value, and blue pixel value) of the first RGB pixel.

[0070] Thmax indicates the maximum value that a pixel value can take. Thmax is expressed as 1023 in 10-bit notation, for example. Thmax is an example of a first threshold value.

[0071] Gmax is the maximum value of the multiple green pixel values ​​in the first RGB image. In other words, it is the pixel value of the green pixel with the maximum pixel value among the multiple green pixels of the first RGB pixels that make up the first RGB image. Thb is a threshold value used to extract the second green pixel from the first RGB pixels. Since a large Thb value makes the image too bright, it is set to a value of 10 or less in 10-bit representation, for example.

[0072] Glossy regions are excluded using equation 1, and tooth regions in the first RGB image are extracted using equation 2. That is, the multiple pixels extracted using equations 1 and 2 are multiple second pixels that make up the tooth region. In this way, the multiple second pixels are pixels that, among the multiple first RGB pixels (third pixels) that make up the first RGB image, satisfy the following conditions: the pixel value max(R,G,B) of the maximum color component is smaller than the first threshold value (Thmax), and the pixel value min(R,G,B) of the minimum color component is equal to or smaller than the second threshold value (Ths).

[0073] The detection unit 102 calculates an average value of the green pixels of the plurality of second pixels, and determines a gain by which the pixel values ​​of the three RGB sub-pixels are multiplied according to the calculated average value of the green pixels. 、RThe detection unit 102 determines a gain to be multiplied by the pixel values ​​of the three subpixels of GB. The gain is obtained by dividing the target pixel value by the average value of the green pixels. The detection unit 102 generates a third RGB image by multiplying each of the multiple first RGB pixels constituting the first RGB image by the determined gain. More specifically, the detection unit 102 generates the third RGB image by multiplying the pixel values ​​of the three subpixels of each of the multiple first RGB pixels by the determined gain. In other words, the pixel values ​​of the multiple third RGB pixels constituting the third RGB image are pixel values ​​calculated by multiplying the pixel values ​​of the multiple first RGB pixels constituting the first RGB image by the determined gain. Note that if the pixel value exceeds the maximum value (1023 in the case of 10-bit representation) after multiplication by the gain, the detection unit 102 replaces the pixel value with 1023.

[0074] In the above description, the average value of the green pixels of the second pixels extracted from the first RGB pixels is calculated using Equation 2, and the gain by which the pixel values ​​of the three RGB subpixels are multiplied is determined in accordance with the calculated average value of the green pixels. However, this is not limiting. The average value of the red pixels of the second pixels extracted from the first RGB pixels may be calculated, and the gain by which the pixel values ​​of the three RGB subpixels are multiplied may be determined in accordance with the calculated average value of the red pixels. Similarly, the average value of the blue pixels of the second pixels extracted from the first RGB pixels may be calculated, and the gain by which the pixel values ​​of the three RGB subpixels are multiplied may be determined in accordance with the calculated average value of the blue pixels.

[0075] As described above, the exposure control process is a process for generating a third RGB image by determining gains for a plurality of second pixel values ​​calculated from a plurality of second pixel values ​​of a plurality of second pixels (pixels corresponding to the tooth region) included in an RGB image to be processed (here, the first RGB image) so that the average of the plurality of index values ​​calculated from the plurality of second pixel values ​​of each of the plurality of second pixels is a predetermined value, and applying the determined gains to a plurality of first RGB pixel values ​​of a plurality of first RGB pixels included in the first RGB image. Note that the index value may be a value calculated from the pixel values ​​of the three RGB subpixels constituting one pixel, or may be the pixel value of any one of the three subpixels. Here, the average of the plurality of index values ​​is the average of the color component having the largest pixel value among the red, green, and blue components of the first RGB image. The color component having the maximum average value is the color component having the maximum average value among three average values: a first red pixel average value of the multiple red pixel values ​​of the multiple first pixels constituting the first RGB image, a first green pixel average value of the multiple green pixel values ​​of the multiple first pixels, and a first blue pixel average value of the multiple blue pixel values ​​of the multiple first pixels. Note that the color component having the maximum average value does not need to be determined by calculating and comparing the first red pixel average value, the first green pixel average value, and the first blue pixel average value, and may be fixed to the green component.

[0076] In the above description, the detection unit 102 calculates the average value of the green pixels of the plurality of second pixels and determines the gain based on the calculated average value of the green pixels. However, this is not limited to this. The detection unit 102 may calculate the average value of the luminance values ​​of the plurality of second pixels as the average value of the plurality of index values ​​and determine the gain based on the calculated average value of the luminance values. In this manner, the index value may be the pixel value of any one of the three RGB subpixels that constitute one pixel, or may be a value calculated from the pixel values ​​of the three subpixels. Specifically, for each of the plurality of second pixels, the detection unit 102 calculates the luminance value of the second pixel using the subpixel values ​​of the three subpixels of the second pixel. For example, the detection unit 102 calculates the luminance value using the following Equation 3:

[0077] Y=0.21*R+0.72*G+0.07*B (Equation 3)

[0078] In Equation 3, Y is the luminance value, R is the red pixel value, G is the green pixel value, and B is the blue pixel value.

[0079] In this way, the plurality of luminance values ​​may be obtained by calculating, for each of the plurality of pixel values, the luminance values ​​based on the red pixel value, the green pixel value, and the blue pixel value included in the pixel value.

[0080] (White balance adjustment processing) In the white balance adjustment process, the detection unit 102 extracts a plurality of pixels whose RGB values ​​satisfy Expression 1 and Expression 4 below from among a plurality of third RGB pixels constituting the third RGB image to be processed.

[0081] Thl≦Y≦Thu (Formula 4)

[0082] In Equation 4, Thl is a threshold value indicating the lower limit of the tooth region, and Thu is a threshold value indicating the upper limit of the tooth region.

[0083] The tooth region in the third RGB image is extracted by equation 4. That is, the multiple pixels extracted by equations 1 and 4 are the multiple second pixels that make up the tooth region.

[0084] Then, detection unit 102 calculates a first red pixel average value Rave, which is the average value of multiple red pixel values ​​in the tooth region that satisfy Equations 1 and 4, a first green pixel average value Gave, which is the average value of multiple green pixel values ​​in the tooth region, and a first blue pixel average value Bave, which is the average value of multiple blue pixel values ​​in the tooth region. Detection unit 102 then adjusts the gains of at least two color components, the red component, the green component, and the blue component, of the RGB image to be processed so that the first red pixel average value Rave, the first green pixel average value Gave, and the first blue pixel average Bave are equal.

[0085] Specifically, the detection unit 102 calculates the gains of the multiple red pixel values ​​(red pixel gains) by dividing the first average green pixel value Gave by the first average red pixel value Rave. The detection unit 102 also calculates the gains of the multiple blue pixel values ​​(blue pixel gains) by dividing the first average green pixel value Gave by the first average blue pixel value Bave. The detection unit 102 then generates the second RGB image by multiplying each red pixel of the multiple third RGB pixels constituting the third RGB image by the red pixel gain and by multiplying each blue pixel of the multiple third RGB pixels by the blue pixel gain. In other words, the pixel values ​​of the multiple second RGB pixels constituting the second RGB image are calculated by multiplying the red pixel values ​​of the multiple third RGB pixels constituting the third RGB image by the red pixel gain and by multiplying the blue pixel values ​​of the multiple third RGB pixels by the blue pixel gain. It should be noted that the detection unit 102 performs white balance adjustment by calculating the red pixel gain and the blue pixel gain based on the green pixel average value and multiplying each gain by the pixel value of the corresponding color component, but this is not limiting. The green pixel gain and the blue pixel gain may be calculated based on the red pixel average value, or the red pixel gain and the green pixel gain may be calculated based on the blue pixel average value.

[0086] If the pixel value exceeds the maximum value (1023 in the case of 10-bit representation) as a result of multiplying the pixel value by the gain, the detection unit 102 replaces the pixel value with 1023.

[0087] Furthermore, the detection unit 102 may perform the following third image processing on the second RGB image to enhance the plaque region within the tooth region in the second RGB image. Specifically, the detection unit 102 generates an HSV image by converting the color space of the second RGB image into an HSV space. The detection unit 102 then identifies, as the plaque region, a specific pixel region in which one or more fourth pixels of the HSV image are located, the fourth pixels having at least one of a saturation within a first predetermined range (e.g., 30 to 80 in 8-bit representation), a hue within a second predetermined range (e.g., 140 to 170 in 8-bit representation), and a lightness within a third predetermined range (e.g., 100 to 180 in 8-bit representation). Note that the first, second, and third predetermined ranges are not limited to the above-mentioned numerical ranges, and may be identified by comparing the actual plaque region and tooth region with the HSV image.

[0088] The ranges of saturation, hue, and brightness values ​​can be determined by administering a plaque staining agent and comparing the degree of staining by the plaque staining agent.

[0089] (Fluorescent substance concentration distribution detection process) The detecting unit 102 detects the concentration distribution of the fluorescent material using the HSV image. Specifically, the detecting unit 102 detects the concentration distribution of the fluorescent material using the value of brightness V of the HSV image.

[0090] Fig. 6 is a diagram showing an example of teeth in an oral cavity. Fig. 6 shows teeth 301, gums 302, and dental plaque 303. Fig. 7 is a diagram showing an example of a layered model of the structure of region 304 shown in Fig. 6. As shown in Fig. 7, dental plaque 303 is made up of layers of mature dental plaque (tartar) 305 and young dental plaque 306.

[0091] As blue light passes through each layer, porphyrins in the plaque are excited, generating red fluorescence. Furthermore, the intensity of the fluorescence is thought to indicate the accumulation of fluorescent substances (porphyrins), rather than reflecting the current bacterial flora. In other words, the greater the accumulation of fluorescent substances, the stronger the red fluorescence. In other words, the level of porphyrin accumulation increases as the plaque matures. Therefore, the fluorescence intensity of mature plaque 305 is stronger than that of young plaque 306.

[0092] The detection unit 102 detects the accumulation level (concentration or density) of the fluorescent substance by comparing the intensity of red fluorescence per unit area of ​​the plaque region. In other words, the detection unit 102 detects the amount of fluorescent substance contained in the plaque and tartar attached to the teeth per unit area.

[0093] As described above, a plaque region is extracted from one or more fourth pixels from the HSV image that satisfy at least one of the following conditions: saturation S within a first predetermined range, hue H within a second predetermined range, and brightness V within a third predetermined range.

[0094] Furthermore, if the maximum value of the three values ​​R, G, and B is MAX and the minimum value is MIN, then the following formulas 5, 6, and 7 hold in the cylindrical model of the HSV space.

[0095]

number

[0096] Here, since the second RGB image is an image after white balance adjustment processing, MAX=R, and MIN=G or B. That is, in the plaque region, the lightness V is determined by the value of R, regardless of the saturation S and hue H.

[0097] It is known that the fluorescent wavelength of porphyrin in dental plaque is 600 nm to 740 nm, with a peak fluorescence wavelength of 630 nm. In other words, the concentration of porphyrin accumulated in the dental plaque region can be evaluated by detecting the brightness V value of the HSV image of the dental plaque region.

[0098] Furthermore, a biofilm layering model can be used for the layering model shown in Fig. 7. Fig. 8 is a diagram for explaining the illuminance of fluorescence at a depth D.

[0099] In the biofilm model, the LED light and plaque fluorescence attenuate as they travel through the plaque. Attenuation of the LED light and plaque fluorescence outside the plaque (i.e., in the air) can be ignored. The brightness of the plaque fluorescence is proportional to the illuminance of the LED light that hits the plaque. The fluorescence of the plaque is emitted from the surface that the LED light hits in the direction from which the LED light came (the direction of reflection). For simplicity in considering the depth direction, we will assume that the LED light and plaque fluorescence are a uniform surface light source (i.e., brightness = illuminance).

[0100] Specifically, the LED light has a brightness of E λ1 The density of dental plaque is uniformly distributed, and the LED light attenuation rate due to dental plaque is σ λ1 is constant regardless of the depth D. In this case, the illuminance E of the LED light at the depth D is λ1 (D) is expressed by Equation 8. Here, the spectral transmittance T dλ1 is expressed by Equation 9, and Equation 10 is obtained from Equation 8 and Equation 9.

[0101]

number

[0102] Also, the fluorescence intensity E of dental plaque at depth D λ2 is expressed by Equation 11 using a proportionality constant k.

[0103]

number

[0104] In addition, the brightness of the fluorescence emitted by dental plaque, E λ2 is assumed to be equal regardless of location if the depth D is the same. Dental plaque has a uniform density distribution, and the decay rate of fluorescence due to plaque is σ λ2 is assumed to be constant regardless of the depth D. In this case, the illuminance E of the fluorescence observed from the dental plaque at the depth D is λ2(D) is expressed by Equation 12. Here, the spectral transmittance T dλ2 is expressed by Equation 13, so Equation 14 is obtained from Equations 12 and 13. Furthermore, Equation 15 is obtained from Equations 10, 11, and 14.

[0105]

number

[0106] Figure 9 shows the illuminance E of fluorescence observed from dental plaque with a thickness of D0. λ2_t (D0). The illuminance E of fluorescence observed from dental plaque with a thickness of D0. λ2_t (D0) is E λ2 (D) can be expressed as the integral of E λ2_t (D0) is expressed by Equation 16.

[0107]

number

[0108] Thus, the intensity of red fluorescence from a dental plaque layer (biofilm) with a thickness of D0 reflects the level of porphyrin accumulation in the dental plaque layer (biofilm). While the above explanation assumes a constant porphyrin concentration within the dental plaque layer, this model can also explain the fluorescence response when the lower layer in the biofilm model is dental calculus with a high porphyrin concentration and the upper layer is young dental plaque with a low porphyrin concentration. Specifically, as shown in Figure 7, when the lower layer of mature dental plaque 305 (dental calculus) is covered by the upper layer of young dental plaque 306, the red fluorescence in the area with the lower layer of mature dental plaque 305 is stronger than the red fluorescence in the area without the lower layer of mature dental plaque 305.

[0109] Furthermore, from Equation 16, it can be seen that the fluorescence changes depending on the thickness D (the greater the thickness D, the stronger the fluorescence). In other words, the intensity of the fluorescence indicates the accumulation level, which is proportional to the concentration and amount (thickness) of the fluorescent substance.

[0110] Fig. 10 is a flowchart of the process of detecting the concentration distribution of a fluorescent substance by the detection unit 102. For example, the detection unit 102 detects the accumulation level for each pixel by performing the process shown in Fig. 10 on each pixel in the plaque region included in the HSV image. Note that the detection unit 102 may also detect the accumulation level for each unit pixel by performing the process shown in Fig. 10 for each unit pixel made up of multiple pixels. In this case, for example, the average value of the brightness of multiple pixels included in the unit pixel may be used.

[0111] First, the detection unit 102 determines whether the brightness of the target pixel is less than the first threshold (S121). If the brightness of the target pixel is less than the first threshold (Yes in S121), the detection unit 102 determines the accumulation level of the target pixel to be accumulation level 0 (e.g., no plaque) (S122).

[0112] On the other hand, if the brightness of the target pixel is equal to or greater than the first threshold (No in S121), it is determined whether the brightness of the target pixel is less than a second threshold (S123). Here, the second threshold is greater than the first threshold. If the brightness of the target pixel is less than the second threshold (Yes in S123), that is, if the brightness of the target pixel is greater than or equal to the first threshold and less than the second threshold, the detection unit 102 determines that the accumulation level of the target pixel is accumulation level 1 (e.g., young plaque) (S124).

[0113] On the other hand, if the brightness of the target pixel is equal to or greater than the second threshold (No in S123), the detection unit 102 determines the accumulation level of the target pixel to be accumulation level 2 (for example, mature plaque (tartar)) (S125).

[0114] In this way, the detection unit 102 detects the distribution of the accumulation levels (concentration of the fluorescent substance) by determining the accumulation level for each pixel. Here, the distribution of the accumulation levels is information indicating the accumulation level for each two-dimensional position (e.g., pixel) on the xy plane.

[0115] Next, the detection unit 102 assigns the three accumulation levels 0 to 2 to different gradations, and generates, for example, a fourth RGB image by superimposing the distribution of accumulation levels displayed in gradations on the second RGB image. Fig. 11 is a diagram showing an example of the fourth RGB image. For example, as shown in Fig. 11, a pattern of a first gradation (e.g., 0.5) is superimposed on the area of ​​young plaque 306, and a pattern of a second gradation (e.g., 1.0) is superimposed on the area of ​​mature plaque 305.

[0116] In the above description, an example in which there are three accumulation levels is shown, but there may be four or more accumulation levels. Furthermore, the second image onto which the accumulation level distribution is superimposed may be something other than the second RGB image. For example, the second image may be the first RGB image, or an image generated by performing image processing on the first RGB image or the second RGB image.

[0117] Fig. 12 is a diagram showing an example of teeth after oral care has been performed on the teeth in the state shown in Fig. 11. As shown in Fig. 12, by performing oral care (tooth brushing, etc.), young plaque 306 has been removed, but mature plaque 305 has not been removed.

[0118] Therefore, the detection unit 102 may determine the score of oral care (whether there is any area left unbrushed) based on the state of the young dental plaque 306. Furthermore, the detection unit 102 may recommend that the user have a checkup at a dentist based on the state of the mature dental plaque 305.

[0119] For example, the detection unit 102 calculates the ratio of the area of ​​the young plaque 306 region to the area of ​​the tooth region. That is, the detection unit 102 calculates (area of ​​the young plaque 306 region) / (area of ​​the tooth region)×100{%} as a first area ratio of the young plaque 306. The detection unit 102 also calculates the ratio of the area of ​​the mature plaque 305 region to the area of ​​the tooth region. That is, the detection unit 102 calculates (area of ​​the mature plaque 305 region) / (area of ​​the tooth region)×100{%} as a second area ratio of the mature plaque 305.

[0120] The detection unit 102 may determine an oral care score using the calculated first area ratio. The detection unit 102 may also recommend that the user see a dentist using the calculated second area ratio. For example, the detection unit 102 may display a message recommending that the user see a dentist when the second area ratio is greater than a predetermined threshold.

[0121] The calculation and determination of the area ratio may be performed collectively for all teeth in the oral cavity. That is, the area ratio may be the ratio between the total area of ​​the dental regions of all teeth and the total area of ​​plaque (young plaque 306 or mature plaque 305). Alternatively, the calculation and determination of the area ratio may be performed individually for each of multiple teeth in the oral cavity. That is, the area ratio may be the ratio between the area of ​​the dental region of one tooth and the area of ​​plaque of one tooth. Alternatively, the calculation and determination of the area ratio may be performed individually for each of the dental regions obtained by dividing multiple teeth in the oral cavity. A dental region is, for example, a region that includes two or more teeth, such as the right rear of the upper jaw or the left front of the lower jaw.

[0122] 5 identifies the types of multiple teeth in the image data based on the image data. Here, the type of tooth is information that can uniquely identify a tooth in the oral cavity, such as a maxillary right central incisor or a mandibular left lateral incisor.

[0123] For example, the identification unit 104 acquires reference data corresponding to multiple tooth types from the cloud server 80, and uses the image data and the acquired reference data to identify the types of each of the multiple teeth included in the image data by comparing features, etc.

[0124] The identification unit 104 stores the concentration distribution (accumulation level) of the fluorescent substance detected by the detection unit 102 in association with a plurality of tooth types in the storage unit 105. That is, the storage unit 105 stores the concentration distribution (accumulation level) of the fluorescent substance for each tooth. Furthermore, the area ratio and determination process for each tooth described above may be performed using this information for each tooth.

[0125] The display unit 103 is a display device included in the mobile terminal 70, and displays a fourth RGB image in which the accumulation level distribution is superimposed on the image data. The display unit 103 also displays the above-mentioned determination result and a message based on the determination result. The display unit 103 may also display the above-mentioned area ratio, etc.

[0126] A modification of the above-described embodiment will now be described.

[0127] (Variation 1) In the above embodiment, the detection unit 102 performs exposure control processing on the first RGB image and white balance adjustment processing on the third RGB image generated by the exposure control processing, but this is not limiting and the exposure control processing may not be performed. For example, if a first RGB image with reduced variations in luminance distribution is obtained, the exposure control processing may not be performed. For example, the variations in luminance distribution of the obtained first RGB image may be reduced by controlling lighting so that the shooting conditions are constant.

[0128] (Variation 2) In the above embodiment, the accumulation level is detected using image data after image processing (exposure control processing, white balance adjustment processing, etc.), but some or all of the image processing may not be performed. For example, an HSV image may be generated from the first RGB image, and the accumulation level may be detected using the HSV image.

[0129] (Variation 3) In the above embodiment, the accumulation level is detected using the brightness of the HSV image, but the method for determining the accumulation level is not limited to this. For example, the saturation or hue of the HSV image may be used, or a combination of brightness and at least one of saturation and hue may be used. For example, an evaluation value calculated from brightness and at least one of saturation and hue may be compared with a threshold value.

[0130] Alternatively, the accumulation level may be detected using an RGB image. For example, the R value of the RGB image may be used, or a combination of the R value and at least one of the G value and B value may be used. For example, an evaluation value calculated from the R value and at least one of the G value and B value may be compared with a threshold value.

[0131] (Variation 4) In the above embodiment, the intraoral camera 10 transmits the first RGB image to the mobile terminal 70, and the mobile terminal 70 processes the first RGB image, but this is not limiting. The first RGB image may be transmitted to the cloud server 80, the cloud server 80 performs the above processing, and the second RGB image or the fourth RGB image resulting from the processing may be transmitted to the mobile terminal 70. In this case, the first RGB image may be transmitted from the intraoral camera 10 to the cloud server 80 without passing through the mobile terminal 70, or may be transmitted from the intraoral camera 10 to the cloud server 80 via the mobile terminal 70.

[0132] (Variation 5) In the above embodiment, the accumulation level is detected using an HSV image, but the method for determining the accumulation level is not limited to this. For example, an HSL image may be used instead of an HSV image.

[0133] For example, in the third image processing described above, the detection unit 102 generates an HSL image by converting the color space of the second RGB image into an HSL space. Then, the detection unit 102 identifies, as a plaque region, a specific pixel region in which one or more fifth pixels of the HSL image that satisfy at least one of a fourth predetermined range for saturation, a fifth predetermined range for hue, and a sixth predetermined range for luminance are located.

[0134] In the above-described process for detecting the concentration distribution of the fluorescent substance, the detection unit 102 detects the concentration distribution of the fluorescent substance using the HSL image. Specifically, the detection unit 102 detects the concentration distribution of the fluorescent substance using the value of the luminance L of the HSL image.

[0135] Here, the HSL color space (also called the HLS color space) is a color space consisting of three components: H (hue), S (saturation), and L (luminance), and is obtained by nonlinear conversion of the RGB color space.

[0136] If the maximum of the three R, G, and B values ​​is MAX and the minimum is MIN, then H (hue) of the HSL image is calculated using the above-mentioned formula 5. Furthermore, L (brightness) is calculated using the following formula 17. S (saturation) is calculated using the following formula 18 when a cylindrical model is used, and is calculated using the following formula 19 when a biconical model is used.

[0137]

number

[0138] In this way, in HSL space, H, S, and L are calculated from MAX(max(R,G,B)) and MIN(min(R,G,B)), but in the fluorescent region of plaque, if MIN=k×MAX, then k=approximately 0 to 0.3.

[0139] Fig. 13 is a diagram showing the relationship between fluorescence intensity, MIN, and k. Fig. 14 is a diagram showing the relationship between fluorescence intensity, S (saturation), and k. Fig. 15 is a diagram showing the relationship between fluorescence intensity, L (luminance), and k. Note that this example is an example when a cylindrical model is used.

[0140] The characteristics for k=0 to 0.3 shown by solid lines in Figures 13 to 15 correspond to the characteristics of the fluorescent region of dental plaque. As shown in Figure 14, when k=0 to 0.3, S=0.5 to 1. As shown in Figure 15, when k=0 to 0.3, L=0 to 0.65.

[0141] Therefore, when S is within the range of 0.5 to 1 in plaque detection, L changes linearly between 0 and 0.65, and the value of L is proportional to the amount of porphyrin. Therefore, the concentration distribution of the fluorescent substance (the amount of fluorescent substance contained in plaque and tartar attached to the teeth per unit area) can be detected based on the value of L.

[0142] (Variation 6) In the above explanation, an example was given in which the blue light cut filter 20 is used as a method for generating a first RGB image in which at least a portion of the blue light region is attenuated from the reflected light and fluorescence from the teeth, plaque, and tartar in the oral cavity, which is irradiated with irradiation light of a predetermined wavelength that excites fluorescent substances contained in plaque and tartar. However, at least a portion of the blue light region may also be attenuated by signal processing without using the blue light cut filter 20.

[0143] 16 is a diagram showing an example of pixel values ​​in each image when the blue light region is attenuated by signal processing. For example, the detection unit 102 generates a fifth RGB image by performing blue-cut processing on the first RGB image, which is image data obtained by the image sensor 14, as shown in FIG. 16. For example, the detection unit 102 generates the fifth RGB image by multiplying the blue pixel values ​​of the first RGB image by a gain of 0. Note that the detection unit 102 may multiply the blue pixel values ​​by a predetermined gain smaller than 1, rather than by a gain of 0. Alternatively, the detection unit 102 may replace the blue pixel values ​​with a predetermined value (for example, 0), or may clip them to a predetermined value or less.

[0144] The detection unit 102 generates a third RGB image by performing the exposure control process described above on the fifth RGB image generated in this manner. For example, the detection unit 102 multiplies R, G, and B by equal gains so that max(R, G, B) reaches a predetermined level.

[0145] Next, the detection unit 102 generates a second RGB image by performing the above-described white balance adjustment process on the third RGB image. For example, the detection unit 102 multiplies R and B individually by gains so that R and B have the same level as G. Note that, when blue cut processing is performed, the detection unit 102 does not need to multiply B by a gain in the white balance adjustment process. Furthermore, the above-described fluorescent material concentration distribution detection process is performed using this second RGB image.

[0146] Even when blue light cutting processing is performed by signal processing in this way, an image (fifth RGB image) in which at least a part of the blue light region is attenuated can be generated, similar to the case where the blue light cutting filter 20 is used.

[0147] Note that the blue color cutting process may be performed on the second RGB image instead of the first RGB image. Fig. 17 shows an example of pixel values ​​in each image when the blue light region is attenuated by signal processing in this case.

[0148] 17, the detection unit 102 generates a sixth RGB image by performing the same blue cut processing as above on the second RGB image after the white balance adjustment processing. The sixth RGB image is used to perform the above-described fluorescent material concentration distribution detection processing.

[0149] Comparing the second RGB image shown in FIG. 16 with the sixth RGB image shown in FIG. 17, the second RGB image shown in FIG. 16 is closer to the state in which blue has been cut using an optical filter, and the remaining R and G levels are also higher, making the image brighter.

[0150] The blue color filtering process may be digital or analog. Although the example above describes the detection unit 102 (mobile terminal 70) performing the blue color filtering process, it may also be performed by the intraoral camera 10. In a typical image sensor used in an RGB camera, the output order of R, G, and B pixel values ​​is determined according to the RGB color filter array. Therefore, the B pixel value can be identified in the blue color filtering process.

[0151] (Variation 7) In the white balance adjustment process (first image processing), the detection unit 102 may extract a natural tooth region free of plaque from the first RGB image and perform the white balance adjustment process using the pixels of the extracted natural tooth region. That is, the first image processing may be a process of extracting a natural tooth region free of plaque and tartar from the first RGB image, and adjusting the gains of at least two color components of the red, green, and blue components of the first RGB image so that a first red pixel average value of the multiple red pixel values ​​of the multiple first pixels constituting the natural tooth region, a first green pixel average value of the multiple green pixel values ​​of the multiple first pixels, and a first blue pixel average value of the multiple blue pixel values ​​of the multiple first pixels become equal.

[0152] Here, the natural tooth region refers to the tooth region excluding the artificial tooth region. The artificial tooth region is an artificial tooth or prosthesis made of, for example, metal (gold, silver, etc.), ceramic, zirconia, etc.

[0153] In this way, by performing white balance adjustment processing using information on pixels in the natural tooth region excluding the artificial tooth region, the accuracy of the white balance adjustment processing can be improved.

[0154] Specifically, the detection unit 102 detects a first natural tooth region in the first RGB image where the green pixel value (G) is equal to or greater than a predetermined first threshold value.

[0155] When excitation light (blue light) is irradiated onto a natural tooth, excitation fluorescence is emitted from the dentin. This excitation fluorescence passes through the enamel, causing the natural tooth to fluoresce green. Furthermore, when blue light is irradiated, fillings from caries treatments appear dark (low brightness) in images captured by a camera, unlike when white light is irradiated. On the other hand, natural teeth covered with enamel appear bright (high brightness) in images. Therefore, by extracting areas where the green pixel value (G) is equal to or greater than a predetermined first threshold and extracting this green fluorescence, it is possible to identify natural tooth areas and exclude artificial tooth areas.

[0156] Note that a luminance value (Y) may be used instead of the green pixel value (G). The luminance value (Y) is calculated using the above formula 3. As shown in formula 3, the ratio of green pixel values ​​to luminance values ​​is high, so even when luminance values ​​are used, the same detection as when green pixel values ​​are used can be performed.

[0157] The detection unit 102 may perform white balance adjustment processing using information on the first natural tooth area detected in this manner, or may detect a second natural tooth area by further excluding areas of plaque and tartar from the first natural tooth area, and perform white balance adjustment processing using information on the detected second natural tooth area.

[0158] Specifically, the detection unit 102 generates an HSV image from the first RGB image, and extracts a region where the values ​​of H, S, and V of the HSV image fall within a predetermined range as a plaque or tartar region. The detection unit 102 may also generate an HSL image from the first RGB image, and extract a region where the values ​​of H, S, and L of the HSL image fall within a predetermined range as a plaque or tartar region.

[0159] Next, the detection unit 102 detects the second natural tooth region by excluding the plaque or tartar region from the first natural tooth region.

[0160] As described above, the plaque detection device (e.g., mobile terminal 70) according to this embodiment includes an acquisition unit 101 that acquires a first image (e.g., a first RGB image) in which at least a part of the blue light region is attenuated from the reflected light and fluorescence from the teeth, plaque, and tartar in the oral cavity irradiated with irradiation light of a predetermined wavelength that excites the fluorescent substances contained in the plaque and tartar, and a detection unit 102 that detects from the first image (e.g., based on the fluorescence intensity value of the fluorescence reaction of the fluorescent substance in the first image) the amount of fluorescent substance contained in the plaque and tartar per unit area (e.g., the concentration distribution of the fluorescent substance). In this way, the plaque detection device can detect the amount of fluorescent substance contained in the plaque and tartar per unit area, thereby enabling detailed detection of the condition of the teeth. Furthermore, the plaque detection device can improve the detection accuracy of the content per unit area of ​​fluorescent substances contained in plaque and tartar attached to the teeth by using a first image in which at least a portion of the blue light region is attenuated from the reflected light and fluorescence from the teeth, plaque, and tartar in the oral cavity, which is irradiated with light of a predetermined wavelength that excites fluorescent substances contained in plaque and tartar.

[0161] For example, the detection unit 102 generates an HSV image from the first image and detects the amount of fluorescent material contained in plaque and tartar attached to the teeth per unit area from the brightness value of the HSV image. This allows the plaque detection device to accurately detect the amount of fluorescent material contained in plaque and tartar attached to the teeth per unit area based on the brightness value of the HSV image.

[0162] For example, the first image is a first RGB image, and the detection unit 102 generates the second RGB image by performing image processing including first image processing on the first RGB image. The first image processing is processing for adjusting the gains of at least two color components of the red, green, and blue components of the RGB image to be processed so that a first average red pixel value of multiple red pixel values ​​of multiple first pixels constituting a tooth region in the RGB image to be processed, a first average green pixel value of multiple green pixel values ​​of the multiple first pixels, and a first average blue pixel value of multiple blue pixel values ​​of the multiple first pixels are equal. The detection unit 102 generates the HSV image by converting the color space of the second RGB image into an HSV space.

[0163] According to this, by performing the first image processing, the plaque detection device can adjust the white balance of the first RGB image in which the fluorescently reacting teeth are captured. As a result, the plaque detection device can generate a second RGB image in which plaque regions, which are regions of the teeth where plaque is attached, can be easily distinguished. Therefore, the plaque detection device can improve the detection accuracy of the amount of fluorescent substance contained in plaque and tartar attached to the teeth per unit area.

[0164] For example, the detection unit 102 identifies a specific pixel area in which one or more fourth pixels of the multiple fourth pixels in the HSV image are located, the fourth pixels having at least one of a first predetermined range for saturation, a second predetermined range for hue, and a third predetermined range for brightness, and detects the amount of fluorescent substance contained in plaque and tartar attached to the teeth per unit area from the brightness value in the specific pixel area of ​​the HSV image.In this way, the plaque detection device can improve the detection accuracy of the amount of fluorescent substance contained in plaque and tartar attached to the teeth per unit area by identifying the plaque area in the tooth image and then detecting the amount of fluorescent substance contained in the plaque and tartar attached to the teeth.

[0165] For example, the detection unit 102 generates an HSL image from the first image and detects the amount of fluorescent material contained in plaque and tartar attached to the teeth per unit area from the brightness value of the HSL image. This allows the plaque detection device to accurately detect the amount of fluorescent material contained in plaque and tartar attached to the teeth per unit area based on the brightness value of the HSL image.

[0166] For example, the first image is a first RGB image, and the detection unit 102 generates the second RGB image by performing image processing including first image processing on the first RGB image. The first image processing is processing for adjusting the gains of at least two color components of the red, green, and blue components of the RGB image to be processed so that a first average red pixel value of multiple red pixel values ​​of multiple first pixels constituting a tooth region in the RGB image to be processed, a first average green pixel value of multiple green pixel values ​​of the multiple first pixels, and a first average blue pixel value of multiple blue pixel values ​​of the multiple first pixels are equal. The detection unit 102 generates the HSL image by converting the color space of the second RGB image into an HSL space.

[0167] According to this, by performing the first image processing, the plaque detection device can adjust the white balance of the first RGB image in which the fluorescently reacting teeth are captured. As a result, the plaque detection device can generate a second RGB image in which plaque regions, which are regions of the teeth where plaque is attached, can be easily distinguished. Therefore, the plaque detection device can improve the detection accuracy of the amount of fluorescent substance contained in plaque and tartar attached to the teeth per unit area.

[0168] For example, the detection unit 102 identifies a specific pixel area in which one or more fifth pixels of the plurality of fifth pixels in the HSL image are located, the fifth pixels having saturation within a fourth predetermined range, hue within a fifth predetermined range, and luminance within a sixth predetermined range, and detects the amount of fluorescent substance contained in plaque and tartar attached to the teeth per unit area from the luminance value in the specific pixel area. In this way, the plaque detection device can improve the detection accuracy of the amount of fluorescent substance contained in plaque and tartar attached to the teeth per unit area by identifying the plaque area in the tooth image and then detecting the amount of fluorescent substance contained in the plaque and tartar attached to the teeth.

[0169] For example, the fluorescent substance is porphyrin. For example, the detection unit 102 assigns the content per unit area of ​​the fluorescent substance contained in the plaque and tartar attached to the teeth to three or more gradations, and generates a third image (e.g., a fourth RGB image) by superimposing the content per unit area of ​​the fluorescent substance contained in the plaque and tartar attached to the teeth, displayed in gradations, on a second image (e.g., the first RGB image or the second RGB image) based on the first image. In this way, for example, the generated third image can notify the user of the content per unit area of ​​the fluorescent substance contained in the plaque and tartar attached to the teeth.

[0170] For example, the plaque detection device further includes an identification unit 104 that identifies the type of photographed tooth, and a storage unit 105 that stores the amount of fluorescent substance contained in plaque and tartar attached to the tooth per unit area detected from the photographed tooth in association with the identified tooth type. This allows the plaque detection device to manage the amount of fluorescent substance contained in plaque and tartar attached to the tooth per unit area for each tooth.

[0171] Furthermore, the plaque detection device (for example, the mobile terminal 70) according to this embodiment includes an acquisition unit 101 that acquires a first RGB image from reflected light and fluorescence from the teeth, plaque, and tartar in the oral cavity that is irradiated with irradiation light of a predetermined wavelength that excites fluorescent substances contained in the plaque and tartar, and a detection unit 102 that generates a second RGB image by performing image processing including first image processing on the first RGB image, and detects the content per unit area of ​​the fluorescent substances contained in the plaque and tartar attached to the teeth based on the fluorescence intensity value of the fluorescent reaction of the fluorescent substances in the second RGB image. The first image processing is a process of extracting a natural tooth region free of plaque and tartar from the first RGB image, and adjusting the gain of at least two color components of the red, green, and blue components of the first RGB image so that a first red pixel average value of the multiple red pixel values ​​possessed by the multiple first pixels constituting the natural tooth region, a first green pixel average value of the multiple green pixel values ​​possessed by the multiple first pixels, and a first blue pixel average value of the multiple blue pixel values ​​possessed by the multiple first pixels are equal.

[0172] This allows the plaque detection device to detect the amount of fluorescent substance contained in plaque and tartar attached to the teeth per unit area, thereby enabling detailed detection of the condition of the teeth. Furthermore, by performing the first image processing, the plaque detection device can adjust the white balance of the first RGB image in which the fluorescently reacting teeth are captured. Therefore, the plaque detection device can generate a second RGB image in which plaque regions, which are areas of the teeth where plaque is attached, can be easily distinguished. Therefore, the plaque detection device can improve the detection accuracy of the amount of fluorescent substance contained in plaque and tartar attached to the teeth per unit area. Furthermore, the plaque detection device can improve the accuracy of the white balance adjustment process by performing the first image processing using pixels of natural tooth regions where plaque and tartar are not attached.

[0173] For example, in extracting the natural tooth region, a first region (e.g., a first natural tooth region) is detected as (i) a region in the entire pixel region of the first RGB image where the luminance value is equal to or greater than a predetermined first threshold, or (ii) a region in the entire pixel region of the first RGB image where the green pixel value is equal to or greater than a predetermined second threshold, and the natural tooth region is extracted based on the first region. This allows the plaque detection device to accurately detect the natural tooth region using the luminance value or the green pixel value.

[0174] For example, in extracting the natural tooth region, the region obtained by excluding the regions of plaque and tartar from the first region is extracted as the natural tooth region. This allows the plaque detection device to improve the accuracy of the white balance adjustment process.

[0175] For example, the first RGB image is an image in which at least a part of the blue light region is attenuated from the reflected light and fluorescence from the teeth and plaque in the oral cavity. By using this first image in which at least a part of the blue light region is attenuated from the reflected light and fluorescence from the teeth, plaque, and tartar in the oral cavity that are irradiated with irradiation light of a predetermined wavelength that excites the fluorescent substances contained in the plaque and tartar, the plaque detection device can improve the detection accuracy of the content per unit area of ​​the fluorescent substances contained in the plaque and tartar attached to the teeth.

[0176] For example, the detection unit 102 generates an HSV image from the second RGB image and detects the amount of fluorescent material contained in plaque and tartar attached to the teeth per unit area from the brightness value of the HSV image. This allows the plaque detection device to accurately detect the amount of fluorescent material contained in plaque and tartar attached to the teeth per unit area based on the brightness value of the HSV image.

[0177] For example, the detection unit 102 identifies a specific pixel area in which one or more fourth pixels of the multiple fourth pixels in the HSV image are located, the fourth pixels having at least one of a first predetermined range for saturation, a second predetermined range for hue, and a third predetermined range for brightness, and detects the amount of fluorescent substance contained in plaque and tartar attached to the teeth per unit area from the brightness value in the specific pixel area.In this way, the plaque detection device can improve the detection accuracy of the amount of fluorescent substance contained in plaque and tartar attached to the teeth by identifying the plaque area in the tooth image and then detecting the amount of fluorescent substance contained in the plaque and tartar attached to the teeth per unit area.

[0178] For example, the detection unit 102 generates an HSL image from the second RGB image and detects the amount of fluorescent material contained in plaque and tartar attached to the teeth per unit area from the brightness value of the HSL image. This allows the plaque detection device to accurately detect the amount of fluorescent material contained in plaque and tartar attached to the teeth per unit area based on the brightness value of the HSL image.

[0179] For example, the detection unit 102 identifies a specific pixel area in which one or more fifth pixels of the plurality of fifth pixels in the HSL image are located, the fifth pixels having saturation within a fourth predetermined range, hue within a fifth predetermined range, and luminance within a sixth predetermined range, and detects the amount of fluorescent substance contained in plaque and tartar attached to the teeth per unit area from the luminance value in the specific pixel area. In this way, the plaque detection device can improve the detection accuracy of the amount of fluorescent substance contained in plaque and tartar attached to the teeth per unit area by identifying the plaque area in the tooth image and then detecting the amount of fluorescent substance contained in the plaque and tartar attached to the teeth.

[0180] For example, the fluorescent substance is porphyrin. For example, the detection unit 102 assigns the content per unit area of ​​the fluorescent substance contained in the plaque and tartar attached to the teeth to three or more gradations, and generates a third image (e.g., a fourth RGB image) by superimposing the content per unit area of ​​the fluorescent substance contained in the plaque and tartar attached to the teeth, displayed in gradations, on a second image (e.g., the first RGB image or the second RGB image) based on the first RGB image. In this way, for example, the generated third image can notify the user of the content per unit area of ​​the fluorescent substance contained in the plaque and tartar attached to the teeth.

[0181] For example, the plaque detection device further includes an identification unit 104 that identifies the type of photographed tooth, and a storage unit 105 that stores the amount of fluorescent substance contained in plaque and tartar attached to the tooth per unit area detected from the photographed tooth in association with the identified tooth type. This allows the plaque detection device to manage the amount of fluorescent substance contained in plaque and tartar attached to the tooth per unit area for each tooth.

[0182] Although the intraoral camera system according to the embodiment of the present disclosure has been described above, the present disclosure is not limited to this embodiment.

[0183] For example, although the above description has been given of an example in which the intraoral camera 10 is primarily intended to photograph teeth, the intraoral camera 10 may also be an oral care device equipped with a camera. For example, the intraoral camera 10 may also be an oral irrigator equipped with a camera.

[0184] Furthermore, each processing unit included in the intraoral camera system according to the above embodiment is typically realized as an LSI, which is an integrated circuit. These may be individually implemented as single chips, or some or all of them may be integrated into a single chip.

[0185] Furthermore, the integration is not limited to LSI, but may be realized by dedicated circuits or general-purpose processors. FPGAs (Field Programmable Gate Arrays), which can be programmed after LSI fabrication, or reconfigurable processors, which allow the connections and settings of circuit cells within LSIs to be reconfigured, may also be used.

[0186] In each of the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0187] The present disclosure may also be realized as an image display method or the like executed by an intraoral camera system. The present disclosure may also be realized as an intraoral camera included in the intraoral camera system, a mobile terminal, or a cloud server.

[0188] The division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to another functional block.Furthermore, the functions of multiple functional blocks having similar functions may be processed in parallel or in time-sharing by a single piece of hardware or software.

[0189] The order in which the steps in the flowchart are executed is merely an example for specifically explaining the present disclosure, and other orders may be used. Some of the steps may be executed simultaneously (in parallel) with other steps.

[0190] Although the intraoral camera system according to one or more aspects has been described based on the embodiments, the present disclosure is not limited to these embodiments. As long as it does not deviate from the spirit of the present disclosure, various modifications conceivable by a person skilled in the art to the present embodiments and forms constructed by combining components of different embodiments may also be included within the scope of one or more aspects. [Industrial Applicability]

[0191] The present disclosure is applicable to intraoral camera systems. [Explanation of symbols]

[0192] 10 Intraoral Camera 10a head part 10b Handle 10c neck 12. Shooting optical system 12a Inlet 14 Image sensor 16 lenses 18. Mirror 20 Blue light blocking filter 24 aperture 26A 1st LED 26B Second LED 26C Third LED 26D 4th LED 28 Cover 30 Composition adjustment mechanism 32 Focus adjustment mechanism 34 Case 36, 40 Actuator 38 Lens holder 50 Central Control Unit 54 LED control unit 56 Lens Driver 58 Wireless Communication Module 60 Power supply control unit 62 Controller 64 memory 66 Batteries 68 Coil 69 Charger 70 Mobile Devices 72 Touchscreen 80 Cloud Servers 90 Position Sensor 101 Acquisition Department 102 Detection unit 103 Display section 104 Identification unit 105 Storage section 301 Teeth 302 Gums 303 Dental Plaque 304 area 305 Mature plaque 306 Young Plaque

Claims

1. an acquisition unit that acquires a first RGB image from reflected light and fluorescence from teeth, plaque, and tartar in an oral cavity irradiated with irradiation light of a predetermined wavelength that excites fluorescent substances contained in plaque and tartar; a detection unit that generates a second RGB image by performing image processing including a first image processing on the first RGB image, and detects the content per unit area of ​​the fluorescent substance contained in the plaque and tartar attached to the tooth based on the fluorescence intensity value of the fluorescent reaction of the fluorescent substance in the second RGB image, The first image processing is a process of extracting a natural tooth region free from plaque and tartar from the first RGB image, and adjusting gains of at least two color components of the red, green, and blue components of the first RGB image so that a first red pixel average value of a plurality of red pixel values ​​of a plurality of first pixels constituting the natural tooth region, a first green pixel average value of a plurality of green pixel values ​​of the plurality of first pixels, and a first blue pixel average value of a plurality of blue pixel values ​​of the plurality of first pixels become equal. Plaque detection device.

2. In the extraction of the natural tooth region, (i) a region in which a luminance value is equal to or greater than a predetermined first threshold value in the entire pixel region of the first RGB image, or (ii) a region in which a green pixel value is equal to or greater than a predetermined second threshold value in the entire pixel region of the first RGB image, is detected, and the natural tooth region is extracted based on the first region. The dental plaque detection device of claim 1.

3. In the extraction of the natural tooth region, A region excluding the plaque and tartar regions from the first region is extracted as the natural tooth region.

3. The dental plaque detection device according to claim 2.

4. The first RGB image is an image in which at least a part of the blue light region is attenuated from the reflected light and fluorescence from the teeth and dental plaque in the oral cavity. The plaque detection device according to any one of claims 1 to 3.

5. The detection unit generates an HSV image from the second RGB image, and detects the content per unit area of ​​fluorescent substances contained in the plaque and tartar attached to the teeth from a brightness value of the HSV image. The plaque detection device according to any one of claims 1 to 3.

6. The detection unit identifying a specific pixel area in which one or more fourth pixels of the HSV image are located, the fourth pixels satisfying at least one of a first predetermined range for saturation, a second predetermined range for hue, and a third predetermined range for brightness; The content of the fluorescent substance per unit area contained in the plaque and tartar attached to the tooth is detected from the brightness value in the specific pixel region. The dental plaque detection device according to claim 5.

7. The detection unit generates an HSL image from the second RGB image, and detects the content per unit area of ​​fluorescent substances contained in the plaque and tartar attached to the teeth from a luminance value of the HSL image. The plaque detection device according to any one of claims 1 to 3.

8. The detection unit identifying a specific pixel area in which one or more fifth pixels of the HSL image are located, the fifth pixels satisfying at least one of a fourth predetermined range for saturation, a fifth predetermined range for hue, and a sixth predetermined range for luminance; The content of the fluorescent substance per unit area contained in the plaque and tartar attached to the tooth is detected from the brightness value in the specific pixel region. The dental plaque detection device according to claim 7.

9. The fluorescent substance is a porphyrin The plaque detection device according to any one of claims 1 to 3.

10. The detection unit allocating the content per unit area of ​​the fluorescent substance contained in the plaque and tartar attached to the tooth to three or more gradations; A third image is generated by superimposing the content per unit area of ​​the fluorescent substance contained in the plaque and tartar attached to the teeth, which is displayed in gradation, on a second image based on the first RGB image. The plaque detection device according to any one of claims 1 to 3.

11. The plaque detection device further comprises: an identification unit for identifying the type of photographed tooth; a storage unit that stores the content per unit area of ​​fluorescent substances contained in plaque and tartar attached to the teeth detected from the photographed teeth in association with the identified type of the teeth. The plaque detection device according to any one of claims 1 to 3.

12. acquiring a first RGB image from reflected light and fluorescence from the teeth, plaque, and tartar in the oral cavity irradiated with irradiation light of a predetermined wavelength that excites fluorescent substances contained in the plaque and tartar; generating a second RGB image by performing image processing including a first image processing on the first RGB image, and detecting a content per unit area of ​​the fluorescent substance contained in the plaque and tartar attached to the tooth based on a fluorescent intensity value of a fluorescent reaction of the fluorescent substance in the second RGB image; The first image processing is a process of extracting a natural tooth region free from plaque and tartar from the first RGB image, and adjusting gains of at least two color components of the red, green, and blue components of the first RGB image so that a first red pixel average value of a plurality of red pixel values ​​of a plurality of first pixels constituting the natural tooth region, a first green pixel average value of a plurality of green pixel values ​​of the plurality of first pixels, and a first blue pixel average value of a plurality of blue pixel values ​​of the plurality of first pixels become equal. Plaque detection methods.

13. A program for causing a computer to execute the dental plaque detection method according to claim 12.

Images