Tartar detection device, tartar detection method, and program

Abstract

A tartar detection device is provided with: an acquisition unit (101) for acquiring a first RGB image from reflected light and fluorescent light from teeth, tartar, and tartar in an oral cavity irradiated with irradiation light having a predetermined wavelength; and a detection unit (102) that generates a second RGB image by performing image processing including the first image processing on the first RGB image, and that detects the content per unit area of a fluorescent substance contained in tartar and tartar adhering to the teeth on the basis of the second RGB image. The first image processing is a process in which a natural tooth region to which tartar and tartar are not attached is extracted from the first RGB image, and gains of at least two color components among a red component, a green component, and a blue component of the first RGB image are adjusted. A first red pixel average value of a plurality of red pixel values of a plurality of first pixels constituting the natural tooth region, a first green pixel average value of a plurality of green pixel values of the plurality of first pixels, and a first blue pixel average value of a plurality of blue pixel values of the plurality of first pixels are equal.

Description

Technical Field

[0001] The present disclosure relates to a dental plaque detection device, a dental plaque detection method, and a program. Background Art

[0002] Patent Document 1 discloses a device for detecting dental plaque based on an image of teeth in the oral cavity.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2013-248220 Summary of the invention

[0006] Problem that the invention aims to solve

[0007] It is hoped that such a dental plaque detection device can detect the state of dental plaque in more detail.

[0008] Therefore, the present disclosure provides a dental plaque detection device or a dental plaque detection method capable of detecting the state of teeth in detail.

[0009] Means used to solve problems

[0010] A dental plaque detection device according to one embodiment of the present disclosure comprises: an acquisition unit, which acquires a first RGB image from reflected light and fluorescence from teeth, dental plaque and tartar in an oral cavity irradiated with irradiation light of a specified wavelength, wherein the irradiation light of the specified wavelength excites fluorescent substances contained in the dental plaque and tartar; and a detection unit, which generates a second RGB image by performing image processing including a first image processing on the first RGB image, and detects the content per unit area of ​​fluorescent substances contained in the dental plaque and tartar attached to the teeth based on the value of the fluorescence intensity of the fluorescent reaction of the fluorescent substances in the second RGB image, wherein the first image processing is the following processing: a natural tooth area to which no dental plaque and tartar are attached is extracted from the first RGB image, and the gains of at least two color components among the red component, the green component and the blue component of the first RGB image are adjusted so that a first red pixel average value of a plurality of red pixel values ​​of a plurality of first pixels constituting the natural tooth area, a first green pixel average value of a plurality of green pixel values ​​of the plurality of first pixels, and a first blue pixel average value of a plurality of blue pixel values ​​of the plurality of first pixels are equal.

[0011] Effects of the Invention

[0012] The present disclosure can provide a dental plaque detection device or a dental plaque detection method that can detect the state of teeth in detail. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1It is a stereoscopic view of the intraoral camera in the intraoral camera system of the embodiment.

[0014] Figure 2 This is a cross-sectional view schematically showing a photographing optical system incorporated in the intraoral camera in the intraoral camera system according to the embodiment.

[0015] Figure 3 This is a schematic structural diagram of an intraoral camera system according to an embodiment.

[0016] Figure 4 It is a diagram showing the flow of operations in the intraoral camera system according to the embodiment.

[0017] Figure 5 This is a functional block diagram of a portable terminal according to an embodiment.

[0018] Figure 6 This is a diagram showing an example of teeth in the oral cavity according to the embodiment.

[0019] Figure 7 It is a diagram showing an example of a stacking model according to the embodiment.

[0020] Figure 8 It is a diagram for explaining the illumination of fluorescence at a depth D according to the embodiment.

[0021] Fig. 9 This is a diagram for explaining the illumination of fluorescence observed from dental plaque having a thickness D0 according to the embodiment.

[0022] Fig.10 This is a flowchart of the fluorescent substance concentration distribution detection process according to the embodiment.

[0023] Fig.11 It is a diagram showing an example of a fourth RGB image according to the embodiment.

[0024] Fig.12 It is a diagram showing an example of a fourth RGB image according to the embodiment.

[0025] Fig.13 It is a diagram showing the relationship among fluorescence intensity, MIN, and k according to the embodiment.

[0026] Fig.14 It is a diagram showing the relationship among the fluorescence intensity, S (chroma), and k according to the embodiment.

[0027] Fig.15 It is a diagram showing the relationship among fluorescence intensity, L (brightness), and k according to the embodiment.

[0028] Fig.16 This is a diagram showing an example of pixel values ​​in each image when the blue light region is attenuated by signal processing according to the embodiment.

[0029] Fig.17 This is a diagram showing an example of pixel values ​​in each image when the blue light region is attenuated by signal processing according to the embodiment. DETAILED DESCRIPTION

[0030] In addition, a dental plaque detection device of one embodiment of the present disclosure comprises: an acquisition unit, which acquires a first RGB image from reflected light and fluorescence from teeth, dental plaque and tartar in an oral cavity irradiated with irradiation light of a specified wavelength, wherein the irradiation light of the specified wavelength excites fluorescent substances contained in the dental plaque and tartar; and a detection unit, which generates a second RGB image by performing image processing including first image processing on the first RGB image, and detects the content per unit area of ​​fluorescent substances contained in the dental plaque and tartar attached to the teeth based on the value of the fluorescence intensity of the fluorescent reaction of the fluorescent substances in the second RGB image, wherein the first image processing is the following processing: a natural tooth area to which no dental plaque and tartar are attached is extracted from the first RGB image, and the gains of at least two color components among the red component, the green component and the blue component of the first RGB image are adjusted so that a first red pixel average value of a plurality of red pixel values ​​of a plurality of first pixels constituting the natural tooth area, a first green pixel average value of a plurality of green pixel values ​​of the plurality of first pixels, and a first blue pixel average value of a plurality of blue pixel values ​​of the plurality of first pixels are equal.

[0031] Thus, the dental plaque detection device can detect the content per unit area of ​​fluorescent substances contained in dental plaque and tartar attached to teeth, and thus can detect the state of teeth in detail. In addition, the dental plaque detection device can adjust the white balance of the first RGB image of teeth undergoing a fluorescent reaction by performing the first image processing. Therefore, the dental plaque detection device can generate a second RGB image that can easily distinguish the area on which dental plaque is attached in the teeth, i.e., the dental plaque area. Therefore, the dental plaque detection device can improve the detection accuracy of the content per unit area of ​​fluorescent substances contained in dental plaque and tartar attached to teeth. In addition, the dental plaque detection device can improve the accuracy of the white balance adjustment processing by performing the first image processing using pixels of the natural tooth area to which dental plaque and tartar are not attached.

[0032] For example, in the extraction of the natural tooth region, a first region may be detected, and the natural tooth region may be extracted based on the first region, wherein the first region is (i) a region whose brightness value is greater than a predetermined first threshold value in the full pixel region of the first RGB image, or (ii) a region whose green pixel value is greater than a predetermined second threshold value in the full pixel region of the first RGB image. Thus, the dental plaque detection device can detect the natural tooth region with good accuracy using the brightness value or the green pixel value.

[0033] For example, in the extraction of the natural tooth region, a region excluding regions of dental plaque and calculus may be extracted from the first region as the natural tooth region. Thus, the dental plaque detection device can improve the accuracy of the white balance adjustment process.

[0034] For example, the first RGB image may be an image obtained by attenuating at least a portion of a blue light region from reflected light and fluorescence from the teeth and the dental plaque in the oral cavity. Thus, the dental plaque detection device can improve the detection accuracy of the content per unit area of ​​fluorescent substances contained in dental plaque and tartar attached to teeth by using the first image, which is an image obtained by attenuating at least a portion of a blue light region from reflected light and fluorescence from the teeth, dental plaque and dental plaque in the oral cavity irradiated with irradiation light of a specified wavelength, wherein the specified wavelength is a wavelength that excites fluorescent substances contained in dental plaque and dental plaque.

[0035] For example, the detection unit may generate an HSV image based on the second RGB image, and detect the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth based on the brightness value of the HSV image. Thus, the dental plaque detection device can accurately detect the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth based on the brightness value of the HSV image.

[0036] For example, the detection unit may determine a specific pixel area where at least one fourth pixel that satisfies the conditions that the chroma is within the first specified range, the hue is within the second specified range, and the lightness is within the third specified range among a plurality of fourth pixels of the HSV image is located, and detect the content per unit area of ​​the fluorescent substance contained in the plaque and tartar attached to the teeth according to the value of the lightness in the specific pixel area. Thus, the plaque detection device detects the content per unit area of ​​the fluorescent substance contained in the plaque and tartar attached to the teeth on the basis of determining the plaque area in the image of the teeth, thereby improving the detection accuracy of the content.

[0037] For example, the detection unit may generate an HSL image based on the second RGB image, and detect the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth based on the brightness value of the HSL image. Thus, the dental plaque detection device can accurately detect the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth based on the brightness value of the HSL image.

[0038] For example, the detection unit may determine a specific pixel region where at least one of the fifth pixels satisfying at least one of the chroma within the fourth prescribed range, the hue within the fifth prescribed range, and the brightness within the sixth prescribed range among a plurality of fifth pixels of the HSL image is located, and detect the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth according to the value of the brightness in the specific pixel region. Thus, the dental plaque detection device detects the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth on the basis of determining the dental plaque region in the image of the teeth, thereby being able to improve the detection accuracy of the content.

[0039] For example, the fluorescent substance may be porphyrin. For example, the detection unit may distribute the content per unit area of ​​the fluorescent substance contained in the dental plaque and tartar attached to the teeth into three or more shades, and generate a third image, wherein the third image is an image obtained by superimposing the content per unit area of ​​the fluorescent substance contained in the dental plaque and tartar attached to the teeth showing the shades on the second image based on the first RGB image. Thus, for example, based on the generated third image, the user can be informed of the content per unit area of ​​the fluorescent substance contained in the dental plaque and tartar attached to the teeth.

[0040] For example, the dental plaque detection device may further include: an identification unit for identifying the type of the photographed tooth; and a storage unit for storing the content per unit area of ​​the fluorescent substance contained in the dental plaque and tartar attached to the tooth detected from the photographed tooth in correspondence with the identified type of the tooth. Thus, the dental plaque detection device can manage the content per unit area of ​​the fluorescent substance contained in the dental plaque and tartar attached to the tooth for each tooth.

[0041] In addition, a dental plaque detection method of one embodiment of the present invention obtains a first RGB image based on reflected light and fluorescence from teeth, dental plaque and tartar in an oral cavity irradiated with irradiation light of a specified wavelength, wherein the irradiation light of the specified wavelength excites fluorescent substances contained in dental plaque and tartar, and the dental plaque detection method generates a second RGB image by performing image processing including first image processing on the first RGB image, and detects the content per unit area of ​​fluorescent substances contained in the dental plaque and tartar attached to the teeth based on the value of the fluorescence intensity of the fluorescent reaction of the fluorescent substances in the second RGB image, wherein the first image processing is the following processing: a natural tooth area to which no dental plaque and tartar are attached is extracted from the first RGB image, and the gains of at least two color components among the red component, green component and blue component of the first RGB image are adjusted so that a first red pixel average value of multiple red pixel values ​​of multiple first pixels constituting the natural tooth area, a first green pixel average value of multiple green pixel values ​​of the multiple first pixels, and a first blue pixel average value of multiple blue pixel values ​​of the multiple first pixels are equal.

[0042] Thus, the dental plaque detection method can detect the content per unit area of ​​fluorescent substances contained in dental plaque and tartar attached to teeth, and thus can detect the state of teeth in detail. In addition, the dental plaque detection method can adjust the white balance of a first RGB image of teeth undergoing a fluorescent reaction by performing a first image processing. Therefore, the dental plaque detection method can generate a second RGB image that can easily distinguish an area of ​​teeth to which dental plaque is attached, namely, a dental plaque area. Therefore, the dental plaque detection method can improve the detection accuracy of the content per unit area of ​​fluorescent substances contained in dental plaque and tartar attached to teeth. In addition, the dental plaque detection method can improve the accuracy of the white balance adjustment processing by performing the first image processing using pixels of a natural tooth area to which dental plaque and tartar are not attached.

[0043] Furthermore, a program according to one aspect of the present disclosure is a program for causing a computer to execute the dental plaque detection method.

[0044] In addition, these general or specific methods can be implemented by systems, methods, integrated circuits, computer programs, or computer-readable recording media such as CD-ROMs, or by any combination of systems, methods, integrated circuits, computer programs, and recording media.

[0045] Hereinafter, the embodiments are described in detail with reference to the accompanying drawings as appropriate. However, sometimes a more detailed description than necessary is omitted. For example, sometimes a detailed description of a known matter or a repeated description of a substantially identical structure is omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art.

[0046] In addition, the inventor provides the accompanying drawings and the following description so that those skilled in the art can fully understand the present disclosure, and does not intend to limit the subject matter described in the claims by them.

[0047] (Implementation Method)

[0048] Figure 1 FIG. 2 is a stereoscopic diagram of an intraoral camera in an intraoral camera system according to the present embodiment. Figure 1 As shown, the intraoral camera 10 has a toothbrush-shaped frame that can be held with one hand, and includes a head 10a that is placed in the user's mouth when photographing the dentition, a handle 10b that the user holds, and a neck 10c that connects the head 10a and the handle 10b.

[0049] Figure 2 1 is a cross-sectional view schematically showing the photographing optical system 12 assembled in the intraoral camera 10. Figure 2 As shown, in the present embodiment, the imaging optical system 12 of the intraoral camera 10 is assembled to the head 10a and the neck 10c. The imaging optical system 12 includes an imaging element 14 and a lens 16 arranged on the optical axis LA.

[0050] The imaging element 14 is an imaging device such as a C-MOS (Complementary Metal-Oxide-Semiconductor) sensor or a CCD (Charge Coupled Device) element, and forms an image of the tooth D through a lens 16. The imaging element 14 outputs a signal (image data) corresponding to the formed image to the outside.

[0051] The lens 16 is, for example, a condenser lens, and forms an image of the incident tooth D on the imaging element 14. The lens 16 may be a single lens or a lens group composed of a plurality of lenses.

[0052] In the present embodiment, the photographing optical system 12 further includes: a reflector 18 that reflects the image of the tooth D toward the lens 16; a blue light cutoff filter (blue blocking element) 20 that is disposed between the reflector 18 and the lens 16; and an aperture 24 that is disposed between the lens 16 and the imaging element 14.

[0053] The reflecting mirror 18 is disposed on the optical axis LA of the photographing optical system 12 so as to reflect the image of the tooth D having passed through the entrance port 12 a of the photographing optical system 12 toward the lens 16 .

[0054] The blue light cut filter 20 is a filter that cuts off the light component of the blue wavelength contained in the light incident on the imaging element 14. When light in the wavelength region containing blue light is irradiated onto teeth to detect dental plaque, when the light in the wavelength region containing blue light is enhanced in order to enhance the excitation fluorescence of dental plaque, the first RGB image appears blue as a whole. Since the blue pixel value is dominant compared to the red pixel value and the green pixel value in this state, the effect of easily distinguishing the dental plaque area by performing the image processing (exposure control processing and white balance adjustment processing) described later is sometimes reduced. As a countermeasure, the blue light cut filter 20 cuts off the light in the wavelength region containing blue light from the light before it is incident on the imaging element 14.

[0055] The aperture 24 is a plate-shaped member having a through hole on the optical axis LA of the imaging optical system 12, and realizes a deep focal depth. Thus, the depth direction in the oral cavity can be focused, and a dental image with a clear outline can be obtained.

[0056] In addition, the intraoral camera 10 is equipped with a plurality of first to fourth LEDs 26A to 26D as lighting devices for irradiating light onto the teeth D of the photographed object during photographing. The first to fourth LEDs 26A to 26D are, for example, blue LEDs (Light Emitting Diodes). Figure 1 As shown, in the case of the present embodiment, the first to fourth LEDs 26A to 26D are arranged around the incident port 12a. In addition, a translucent cover 28 covering the first to fourth LEDs 26A to 26D and the incident port 12a is provided on the head 10a so that the gums G and the like do not come into contact with the first to fourth LEDs 26A to 26D and the illumination light is insufficient. In addition, a portion of the first to fourth LEDs 26A to 26D can be set as white LEDs. By setting a portion of the first to fourth LEDs 26A to 26D as white LEDs, the first RGB image can be made brighter, and the balance of the blue pixel value relative to the red pixel value and the green pixel value can be improved.

[0057] In addition, in the case of this embodiment, if Figure 2 As shown, the intraoral camera 10 has a composition adjustment mechanism 30 and a focus adjustment mechanism 32 .

[0058] The composition adjustment mechanism 30 is composed of a frame 34 that holds the imaging element 14 and the lens 16, and an actuator 36 that moves the frame 34 in the extension direction of the optical axis LA. The actuator 36 adjusts the position of the frame 34 to adjust the angle of view, that is, the size of the tooth row imaged on the imaging element 14 is adjusted. In addition, the composition adjustment mechanism 30 automatically adjusts the position of the frame 34 so that, for example, one tooth is reflected as a whole in the captured image. In addition, the composition adjustment mechanism 30 adjusts the position of the frame 34 based on the user's operation to achieve the angle of view desired by the user.

[0059] The focus adjustment mechanism 32 is held in the frame 34 of the composition adjustment mechanism 30, and is composed of a lens holder 38 that holds the lens 16 and an actuator 40 that moves the lens holder 38 in the extending direction of the optical axis LA. The actuator 40 adjusts the relative position of the lens holder 38 with respect to the imaging element 14, so that the focus is adjusted, that is, the focusing is adjusted. In addition, the focus adjustment mechanism 32 automatically adjusts the position of the lens holder 38, for example, so that the focus is on the tooth located at the center position of the captured image. In addition, the focus adjustment mechanism 32 adjusts the position of the lens holder 38 based on the user's operation.

[0060] In addition, the components of the imaging optical system 12 other than the reflecting mirror 18 may be provided in the handle portion 10 b of the intraoral camera 10 .

[0061] The image output from the imaging element 14 is an RGB image in which each of the plurality of pixels constituting the image has RGB sub-pixels.

[0062] In addition, the intraoral camera 10 is equipped with a plurality of first to fourth LEDs 26A to 26D as lighting devices for irradiating light to the teeth of the photographed object during photographing. The first to fourth LEDs 26A to 26D are, for example, blue LEDs that irradiate blue light having a wavelength with a peak value of 405 nm. In addition, the first to fourth LEDs 26A to 26D may be light sources that irradiate light in a wavelength region including blue light, and are not limited to blue LEDs.

[0063] Figure 3 FIG. 2 is a schematic diagram of the structure of the intraoral camera system of this embodiment. Figure 3 As shown, the intraoral camera system of the present embodiment is schematically configured to image a tooth row using the intraoral camera 10 and perform image processing on the imaged image.

[0064] like Figure 3 As shown, the intraoral camera system includes an intraoral camera 10, a portable terminal 70, and a cloud server 80. The portable terminal 70 is, for example, a smart phone or a tablet terminal capable of wireless communication. The portable terminal 70 has, for example, a touch screen 72 capable of displaying a dentition image as an input device and an output device. The portable terminal 70 functions as a user interface of the intraoral camera system.

[0065] The cloud server 80 is a server that can communicate with the portable terminal 70 via the Internet or the like, and provides the portable terminal 70 with an application for using the intraoral camera 10. For example, the user downloads the application from the cloud server 80 and installs it in the portable terminal 70. In addition, the cloud server 80 obtains the dental image captured by the intraoral camera 10 via the portable terminal 70.

[0066] The intraoral camera 10 includes: a central control unit 50 as a main part for controlling the system; an LED controller 54 for controlling multiple LEDs 26A to 26D; a lens driver 56 for controlling the actuator 36 of the composition adjustment mechanism 30 and the actuator 40 of the focus adjustment mechanism 32; and a position sensor 90.

[0067] In addition, the intraoral camera 10 includes a wireless communication module 58 for wirelessly communicating with a portable terminal 70 , and a power supply control unit 60 for supplying power to the central control unit 50 and the like.

[0068] The central control unit 50 of the intraoral camera 10 is mounted on the handle portion 10b of the intraoral camera 10, for example. For example, the central control unit 50 includes a controller 62 such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) that executes various processes described later, and a memory 64 such as a RAM (Random Access Memory) or a ROM (Read-Only Memory) that stores programs for causing the controller 62 to execute various processes. In addition to the programs, the memory 64 also stores a dental image (image data) captured by the imaging element 14 and various setting data. The dental image captured by the imaging element 14 is an example of a first RGB image.

[0069] The controller 62 transmits the dental image output from the imaging element 14 to the portable terminal 70 via the wireless communication module 58. The portable terminal 70 displays the transmitted dental image on the touch screen 72, thereby presenting the dental image to the user.

[0070] The LED control unit 54 is mounted on the handle portion 10b of the intraoral camera 10, for example, and turns on and off the first to fourth LEDs 26A to 26D based on a control signal from the controller 62. The LED control unit 54 is composed of, for example, a circuit. For example, when a user performs an operation to activate the intraoral camera 10 on the touch screen 72 of the portable terminal 70, a corresponding signal is sent from the portable terminal 70 to the controller 62 via the wireless communication module 58. Based on the received signal, the controller 62 sends a control signal to the LED control unit 54 to light up the first to fourth LEDs 26A to 26D.

[0071] The lens driver 56 is mounted on the handle portion 10b of the intraoral camera 10, for example, and controls the actuator 36 of the composition adjustment mechanism 30 and the actuator 40 of the focus adjustment mechanism 32 based on the control signal from the controller 62 of the central control unit 50. The lens driver 56 is composed of, for example, a circuit. For example, when a user performs an operation related to composition adjustment or focus adjustment on the touch screen 72 of the portable terminal 70, a corresponding signal is sent from the portable terminal 70 to the central control unit 50 via the wireless communication module 58. The controller 62 of the central control unit 50 sends a control signal to the lens driver 56 based on the received signal to perform composition adjustment or focus adjustment. In addition, for example, the controller 62 calculates the control amount of the actuator 36 or 40 required for composition adjustment or focus adjustment based on the dental image from the imaging element 14, and sends a control signal corresponding to the calculated control amount to the lens driver 56.

[0072] The wireless communication module 58 is mounted on the handle portion 10b of the intraoral camera 10, for example, and performs wireless communication with the portable terminal 70 based on a control signal from the controller 62. Wireless communication based on existing communication standards such as WiFi (registered trademark) and Bluetooth (registered trademark) is performed between the wireless communication module 58 and the portable terminal 70. Through the wireless communication module 58, a dentition image showing the teeth D is sent from the intraoral camera 10 to the portable terminal 70, or an operation signal is sent from the portable terminal 70 to the intraoral camera 10.

[0073] In the case of the present embodiment, the power control unit 60 is mounted on the handle portion 10b of the intraoral camera 10, and distributes the power of the battery 66 to the central control unit 50, the LED control unit 54, the lens driver 56, and the wireless communication module 58. The power control unit 60 is composed of, for example, a circuit. In addition, in the case of the present embodiment, the battery 66 is a rechargeable secondary battery, and is wirelessly charged by an external charger 69 connected to a commercial power source via a coil 68 mounted on the intraoral camera 10.

[0074] The position sensor 90 is a sensor for detecting the posture and position of the intraoral camera 10, for example, a multi-axis (here, x, y, z three-axis) acceleration sensor. For example, the position sensor 90 may be a six-axis sensor having a three-axis acceleration sensor and a three-axis gyroscope sensor. Figure 1 As shown, the z-axis coincides with the optical axis LA. The y-axis is parallel to the imaging plane and extends in the long-side direction of the intraoral camera 10. In addition, the x-axis is parallel to the imaging plane and is orthogonal to the y-axis. The output of each axis of the position sensor 90 can be sent to the portable terminal 70 via the central control unit 50 and the wireless communication module 58.

[0075] As the position sensor 90, a piezoresistive, capacitive or thermal detection type MEMS (Micro ElectroMechanical Systems) sensor can be used. In addition, although not specifically shown, a correction circuit for correcting the balance of the sensitivity of the sensors of each axis, the temperature characteristics of the sensitivity, or the temperature drift can also be provided. In addition, a bandpass filter (low-pass filter) for removing dynamic acceleration components or noise can be provided. In addition, the noise can be reduced by smoothing the output waveform of the acceleration sensor.

[0076] Next, the operation of the intraoral camera system will be described. Figure 4 is a diagram showing the flow of actions in the intraoral camera system. Figure 4 The processing shown is, for example, processing performed in real time, and is performed each time one or more frames of image data are obtained.

[0077] Image data is generated by the user using the intraoral camera 10 to photograph the teeth and gums in his or her oral cavity (S101). The image data is, for example, image data obtained by irradiating the teeth with light in a wavelength region including blue light to photograph teeth undergoing a fluorescent reaction. Next, the intraoral camera 10 sends the captured image data to the portable terminal 70 (S102). In addition, here, the image data can be a moving image or one or more still images. In addition, in the case where the image data is a moving image or a plurality of still images, sensor data can be sent for each frame of the moving image or each still image. In addition, in the case where the image data is a moving image, sensor data can be sent for each of the plurality of frames.

[0078] Furthermore, the image data may be transmitted in real time or may be transmitted in batches after a series of images are taken (eg, images of all teeth in the oral cavity).

[0079] The portable terminal 70 performs image processing on the received image data (S103), and uses the image data after image processing to detect the concentration distribution of the fluorescent substance (S104). Next, the portable terminal 70 generates an image in which the detected concentration distribution of the fluorescent substance is superimposed on the image in the oral cavity (S105), and displays the generated image (S106).

[0080] By using this intraoral camera system, the user can take an image of the user's own oral cavity with the intraoral camera 10 and confirm the state of the oral cavity displayed on the portable terminal 70. In addition, since the displayed image shows the concentration distribution of the fluorescent substance, the user can easily confirm the health status of his or her own teeth.

[0081] In addition, the portable terminal 70 can generate, for example, a three-dimensional model of a plurality of teeth in the oral cavity based on a plurality of captured image data. Further, the portable terminal 70 can display an image based on the generated three-dimensional model.

[0082] Here, an example in which the portable terminal 70 processes an image of teeth is described, but a part or all of this processing can be performed by the intraoral camera 10. The portable terminal 70 is an example of a dental calculus detection device.

[0083] Figure 5 is a functional block diagram of the portable terminal 70. The portable terminal 70 includes an acquisition unit 101, a detection unit 102, a display unit 103, an identification unit 104, and a storage unit 105.

[0084] The acquisition unit 101 acquires image data (first RGB image) transmitted from the intraoral camera 10. In addition to acquiring image data from the intraoral camera 10, the acquisition unit 101 can also acquire sensor data. The first RGB image is an image obtained by irradiating teeth that are undergoing a fluorescence reaction with light in a wavelength region including blue light using the intraoral camera 10. Here, blue light is an example of irradiation light having a specified wavelength that excites the fluorescent substance contained in dental calculus. Further, the fluorescent substance is, for example, porphyrin.

[0085] The detection unit 102 generates a third RGB image by performing an exposure control process (second image process) on the first RGB image, and can also generate a second RGB image by performing a white balance adjustment process (first image process) on the third RGB image.

[0086] (Exposure control process)

[0087] In the exposure control process, the detection unit 102 first extracts a plurality of pixels whose RGB values satisfy the following Expression 1 and Expression 2 from among a plurality of first RGB pixels (third pixels) constituting the first RGB image.

[0088] min(R, G, B) ≤ Ths, and max(R, G, B) < Thmax (Expression 1)

[0089] Gmax - G ≤ Thb (Expression 2)

[0090] min(R, G, B) represents the minimum value among the pixel values (i.e., red pixel value, green pixel value, and blue pixel value) of the three sub-pixels of RGB each possessed by the first RGB pixel.

[0091] Ths is a threshold value for excluding a region (e.g., a gloss region) in the first RGB image that is strongly affected by reflection from the irradiation light. For example, Ths is 900 in 10-bit representation.

[0092] max(R, G, B) represents the maximum value among the pixel values ​​of the three RGB sub-pixels of the first RGB pixel (ie, the red pixel value, the green pixel value, and the blue pixel value).

[0093] Thmax indicates the maximum value that a pixel value can take. For example, Thmax is 1023 in 10-bit representation. Thmax is an example of a first threshold value.

[0094] Gmax is the maximum value of the multiple green pixel values ​​in the first RGB image. In other words, it is the pixel value of the green pixel with the maximum pixel value among the green pixels of the multiple first RGB pixels constituting the first RGB image. Thb is a threshold value for extracting the green pixel of the second pixel from the first RGB pixels. If the value of Thb increases, the image becomes too bright, so it is set to a value below 10 in 10-bit representation, for example.

[0095] According to Formula 1, the glossy area is excluded, and the area of ​​the teeth in the first RGB image is extracted by Formula 2. That is, the multiple pixels extracted by Formulas 1 and 2 are multiple second pixels constituting the area of ​​the teeth. In this way, the multiple second pixels are pixels that satisfy the maximum color component pixel value max(R, G, B) less than the first threshold value (Thmax) and the minimum color component pixel value min(R, G, B) less than the second threshold value (Ths) among the multiple first RGB pixels (third pixels) constituting the first RGB image.

[0096] The detection unit 102 calculates the average value of the green pixels of the plurality of second pixels, and determines the gain to be multiplied by the pixel values ​​of the three sub-pixels of RGB based on the calculated average value of the green pixels. The detection unit 102 determines the gain to be multiplied by the pixel values ​​of the three sub-pixels of RGB, for example, using the following formula 3. The gain is obtained by dividing the target pixel value by the average value of the green pixels. The detection unit 102 generates a third RGB image by multiplying the plurality of first RGB pixels constituting the first RGB image by the determined gain. More specifically, for each of the plurality of first RGB pixels, the detection unit 102 generates a third RGB image by multiplying the pixel values ​​of the three sub-pixels possessed by the first RGB pixel by the determined gain. In other words, the pixel values ​​of the plurality of third RGB pixels constituting the third RGB image are pixel values ​​calculated by multiplying the pixel values ​​of the plurality of first RGB pixels constituting the first RGB image by the determined gain. In addition, when the pixel value exceeds the maximum value (1023 in the case of 10-bit representation) by multiplying the gain, the detection unit 102 replaces the pixel value with 1023.

[0097] In the above description, the average value of the green pixels of the multiple second pixels extracted from the first RGB pixels is calculated by Formula 2, and the gain multiplied by the pixel values ​​of the three sub-pixels of RGB is determined based on the calculated average value of the green pixels, but it is not limited to this. The average value of the red pixels of the multiple second pixels extracted from the first RGB pixels can also be calculated, and the gain multiplied by the pixel values ​​of the three sub-pixels of RGB is determined based on the calculated average value of the red pixels. Similarly, the average value of the blue pixels of the multiple second pixels extracted from the first RGB pixels can also be calculated, and the gain multiplied by the pixel values ​​of the three sub-pixels of RGB is determined based on the calculated average value of the blue pixels.

[0098] As described above, the exposure control process determines the gain for the plurality of second pixel values ​​in such a manner that the average value of the plurality of index values ​​calculated from the plurality of second pixel values ​​respectively possessed by the plurality of second pixels (pixels corresponding to the region of the teeth) contained in the RGB image (here, the first RGB image) to be processed becomes a predetermined value, and applies the determined gain to the plurality of first RGB pixel values ​​possessed by the plurality of first RGB pixels possessed by the first RGB image, thereby generating a third RGB image. In addition, the index value may be a value calculated from the pixel values ​​of the three sub-pixels constituting the RGB of one pixel, or may be the pixel value of one of the three sub-pixels. Here, the average value of the plurality of index values ​​is the average value of the color component having the maximum pixel value among the red component, the green component, and the blue component of the first RGB image. In addition, the color component having the maximum average value is the color component having the maximum average value among the first red pixel average value of the plurality of red pixel values ​​possessed by the plurality of first pixels constituting the first RGB image, the first green pixel average value of the plurality of green pixel values ​​possessed by the plurality of first pixels, and the first blue pixel average value of the plurality of blue pixel values ​​possessed by the plurality of first pixels. In addition, the color component having the maximum average value may not be calculated and compared with the first red pixel average value, the first green pixel average value, and the first blue pixel average value, but may be fixedly determined as the green component.

[0099] In addition, the detection unit 102 calculates the average value of the green pixels of the plurality of second pixels in determining the gain, and determines the gain based on the calculated average value of the green pixels, but is not limited to this. The detection unit 102 may also calculate the average value of the brightness values ​​of the plurality of second pixels as the average value of the plurality of index values, and determine the gain based on the calculated average value of the brightness values. In this way, the index value may be the pixel value of one of the three sub-pixels constituting the RGB of a pixel, or may be a value calculated based on the pixel values ​​of the three sub-pixels. Specifically, the detection unit 102 uses the sub-pixel values ​​of the three sub-pixels of the second pixel for each of the plurality of second pixels to calculate the brightness value of the second pixel. For example, the detection unit 102 calculates the brightness value using the following formula 3.

[0100] Y=0.21*R+0.72*G+0.07*B(Formula 3)

[0101] In Formula 3, Y is the brightness value, R is the red pixel value, G is the green pixel value, and B is the blue pixel value.

[0102] In this way, for each of the multiple pixel values, multiple brightness values ​​can also be obtained by calculating based on the red pixel value, green pixel value and blue pixel value included in the pixel value.

[0103] (White balance adjustment processing)

[0104] In the white balance adjustment process, the detection unit 102 extracts a plurality of pixels whose RGB values ​​satisfy Expression 1 and Expression 4 described below from among a plurality of third RGB pixels constituting the third RGB image to be processed.

[0105] Thl≤Y≤Thu(Formula 4)

[0106] In Formula 4, Th1 is a threshold value representing the lower limit value of the tooth region, and Thu is a threshold value representing the upper limit value of the tooth region.

[0107] The region of the teeth in the third RGB image is extracted according to Formula 4. That is, the plurality of pixels extracted according to Formula 1 and Formula 4 are a plurality of second pixels constituting the region of the teeth.

[0108] Then, the detection unit 102 calculates the first red pixel average value Rave, which is the average value of multiple red pixel values ​​in the tooth region, the first green pixel average value Gave, which is the average value of multiple green pixel values ​​in the tooth region, and the first blue pixel average value Bave, which is the average value of multiple blue pixel values ​​in the tooth region, which satisfy Formula 1 and Formula 4. Then, the detection unit 102 adjusts the gain of at least two color components among the red component, the green component, and the blue component of the RGB image to be processed so that the first red pixel average value Rave, the first green pixel average value Gave, and the first blue pixel average value Bave are equal.

[0109] Specifically, the detection unit 102 calculates the gain of multiple red pixel values ​​(gain for red pixels) by dividing the first green pixel average value Gave by the first red pixel average value Rave. In addition, the detection unit 102 calculates the gain of multiple blue pixel values ​​(gain for blue pixels) by dividing the first green pixel average value Gave by the first blue pixel average value Bave. Furthermore, the detection unit 102 generates a second RGB image by multiplying each red pixel of a plurality of third RGB pixels constituting the third RGB image by a gain for red pixels, and multiplying each blue pixel of a plurality of third RGB pixels by a gain for blue pixels. In other words, the pixel values ​​of the plurality of second RGB pixels constituting the second RGB image are pixel values ​​calculated by multiplying the red pixel values ​​of the plurality of third RGB pixels constituting the third RGB image by a gain for red pixels, and multiplying the blue pixel values ​​of the plurality of third RGB pixels by a gain for blue pixels. In addition, the detection unit 102 uses the average value of green pixels as a reference to calculate the gain for red pixels and the gain for blue pixels, and multiplies each gain by the pixel value of the corresponding color component to adjust the white balance, but is not limited to this. The average value of red pixels may be used as a reference to calculate the gain for green pixels and the gain for blue pixels, and the average value of blue pixels may be used as a reference to calculate the gain for red pixels and the gain for green pixels.

[0110] In addition, when the pixel value exceeds the maximum value (1023 in the case of 10-bit expression) by multiplying the gain, the detection unit 102 replaces the pixel value with 1023.

[0111] In addition, the detection unit 102 can emphasize the dental plaque area in the area of ​​the teeth in the second RGB image by performing the following third image processing on the second RGB image. Specifically, the detection unit 102 generates an HSV image by converting the color space of the second RGB image into an HSV space. Then, the detection unit 102 determines the determined pixel area where at least one of the plurality of fourth pixels of the HSV image that satisfies the chroma within the first prescribed range (for example, 30 or more and 80 or less in 8-bit representation) and the hue within the second prescribed range (for example, 140 or more and 170 or less in 8-bit representation) and the lightness within the third prescribed range (for example, 100 or more and 180 or less in 8-bit representation) is located as the dental plaque area. In addition, the first prescribed range, the second prescribed range and the third prescribed range can be determined by comparing the actual dental plaque area, the area of ​​the teeth and the HSV image, and are not limited to the above-mentioned numerical ranges.

[0112] The ranges of the values ​​of saturation, hue, and lightness can be determined by applying a plaque stain and comparing the staining degree of the plaque stain.

[0113] (Concentration distribution detection processing of fluorescent material)

[0114] The detection unit 102 detects the concentration distribution of the fluorescent substance using the HSV image. Specifically, the detection unit 102 detects the concentration distribution of the fluorescent substance using the value of the brightness V of the HSV image.

[0115] Figure 6 is a diagram showing an example of teeth in the oral cavity. Figure 6 , teeth 301 , gums 302 , and plaque 303 are illustrated. Figure 7 It means that Figure 6 FIG. 3 is a diagram showing an example of a structural stack model of the region 304 shown. Figure 7 As shown, dental plaque 303 is layered with mature dental plaque (tartar) 305 and immature dental plaque 306 .

[0116] As the blue light passes through the layers, the porphyrins in the dental plaque are excited and produce red fluorescence. In addition, the fluorescence intensity does not reflect the current flora, but is considered to indicate the accumulation of fluorescent substances (porphyrins). That is, the more fluorescent substances accumulate, the stronger the red fluorescence. In other words, as the dental plaque matures, the accumulation level of porphyrins increases. Therefore, the fluorescence intensity of mature dental plaque 305 is stronger than that of immature dental plaque 306.

[0117] The detection unit 102 detects the accumulation level (concentration or density) of the fluorescent substance by comparing the intensity of red fluorescence per unit area of ​​the dental plaque region. That is, the detection unit 102 detects the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth.

[0118] As described above, from the HSV image, the dental plaque area is extracted from one or more fourth pixels that satisfy at least one of the conditions that the chroma S is within the first prescribed range, the hue H is within the second prescribed range, and the lightness V is within the third prescribed range.

[0119] In addition, if the largest of the three values ​​of R, G, and B is set to MAX and the smallest is set to MIN, then in the cylindrical model of the HSV space, the following equations 5, 6, and 7 are established.

[0120] [Mathematical formula 1]

[0121]

[0122] V=MAX…(Formula 6)

[0123]

[0124] Here, since the second RGB image is an image after white balance adjustment processing, MAX=R and MIN=G or B. That is, in the dental plaque area, the lightness V does not depend on the chroma S and the hue H, but is determined by the R value.

[0125] It is known that the fluorescence wavelength of porphyrin, a fluorescent substance in dental plaque, is 600 nm to 740 nm, and the peak fluorescence wavelength is 630 nm. That is, by detecting the value of brightness V of the HSV image of the dental plaque area, the concentration of porphyrin accumulated in the dental plaque area can be evaluated.

[0126] In addition, the biofilm stack model can be used to Figure 7 The stackup shown is modeled. Figure 8 It is a diagram for explaining the illumination of fluorescence at a depth D.

[0127] In the biofilm model, the LED light and the fluorescence of dental plaque attenuate as they advance through the dental plaque. The attenuation of the LED light and the fluorescence of dental plaque outside the dental plaque (i.e., in the atmosphere) can be ignored. The brightness of the fluorescence of dental plaque is proportional to the illuminance of the LED light irradiated on the dental plaque. The fluorescence of dental plaque emits light in the direction of arrival (reflection direction) of the LED light on the surface where the LED light arrives. As a study in the depth direction, for the sake of simplicity, it is assumed that the LED light and the fluorescence of dental plaque are uniform surface light sources (i.e., brightness = illuminance).

[0128] Specifically, suppose the LED light has a brightness of E λ1 Uniform illumination. The density of dental plaque is evenly distributed, and regardless of the depth D, the attenuation rate of LED light caused by dental plaque is λ1 In this case, the illuminance E of the LED light at depth D is λ1 (D) is expressed by equation 8. Here, since the spectral transmittance Td λ1 It is expressed by equation 9, so equation 10 is obtained according to equation 8 and equation 9.

[0129] [Mathematical formula 2]

[0130] E λ1 (D) = T dλ1 ·E λ1 …(Equation 8)

[0131] T dλ1 =E λ1 ·exp(-σ λ1 D)…(Formula 9)

[0132] E λ1 (D)=E λ1 ·exp(-σ λ1 ·D)…(Formula 10)

[0133] In addition, the fluorescence brightness E of dental plaque at the depth D λ2 It is expressed using equation 11 with a proportionality constant k.

[0134] [Mathematical formula 3]

[0135] Eλ2 = k·E λ1 (D)…(Formula 11)

[0136] Furthermore, if the depth D is the same, the brightness E of the fluorescence emitted by dental plaque is λ2 The density of dental plaque is uniformly distributed, and no matter what the depth D is, the decay rate of fluorescence caused by dental plaque is λ2 In this case, the fluorescence illumination E observed from the dental plaque at depth D is λ2 (D) is expressed by equation 12. Here, since the spectral transmittance Td λ2 It is expressed by equation 13, so equation 14 is obtained from equation 12 and equation 13. In addition, equation 15 is obtained from equation 10, equation 11 and equation 14.

[0137] [Formula 4]

[0138] E λ2 (D) = T dλ2 ·E λ2 …(Equation 12)

[0139] T dλ2 =E λ2 ·exp(-σ λ2 ·D)…(Formula 13)

[0140] E λ2 (D)=E λ2 ·exp(-σ λ2 ·D)…(Formula 14)

[0141] E λ2 (D) = k·E λ1 ·exp(-σ λ1 ·D-σ λ2 ·D)

[0142] = k·E λ1 ·exp{-(σ λ1 +σ λ2 )·D}…(Formula 15)

[0143] Fig. 9 is used to explain the illumination E of the fluorescence observed from the dental plaque with a thickness of D0. λ2_t (D0) The fluorescence intensity E observed from the dental plaque with a thickness of D0 λ2_t (D0) can be expressed as E λ2 Specifically, E λ2_t (D0) is expressed by formula 16.

[0144] [Mathematical formula 5]

[0145]

[0146] Thus, the red fluorescence brightness from the dental plaque layer (biofilm) of thickness D0 reflects the accumulation level of porphyrins accumulated in the dental plaque layer (biofilm). In addition, in the above description, it can be explained that the porphyrin concentration in the dental plaque layer is constant, but in the biofilm model, the lower layer is dental plaque with a high porphyrin concentration, and the upper layer is immature dental plaque with a low porphyrin concentration. That is, if Figure 7 As shown, when the lower layer of mature tartar 305 (dental plaque) is covered by the upper layer of immature tartar 306, the red fluorescence of the lower layer having the mature tartar 305 is stronger than the red fluorescence of the lower layer having no mature tartar 305.

[0147] In addition, it can be seen from Formula 16 that the fluorescence changes according to the thickness D (the larger the thickness D, the stronger the fluorescence). That is, the intensity of fluorescence represents a cumulative level proportional to the concentration and amount (thickness) of the fluorescent substance.

[0148] Fig.10 FIG. 1 is a flowchart of the concentration distribution detection process of the fluorescent substance performed by the detection unit 102. For example, the detection unit 102 detects each pixel of the dental plaque area included in the HSV image. Fig.10 The detection unit 102 can detect the accumulation level of each pixel by performing the process shown in FIG. Fig.10 The processing shown is used to detect the accumulation level of each unit pixel. In this case, for example, the average value of the brightness of a plurality of pixels included in the unit pixel can be used.

[0149] First, the detection unit 102 determines whether the brightness of the target pixel is less than a first threshold value (S121). If the brightness of the target pixel is less than the first threshold value (Yes in S121), the detection unit 102 determines the accumulation level of the target pixel to be accumulation level 0 (for example, no dental plaque) (S122).

[0150] On the other hand, when the brightness of the object pixel is greater than the first threshold value (No in S121), it is determined whether the brightness of the object pixel is less than the second threshold value (S123). Here, the second threshold value is greater than the first threshold value. When the brightness of the object pixel is less than the second threshold value (Yes in S123), that is, when the brightness of the object pixel is included in the range of greater than the first threshold value and less than the second threshold value, the detection unit 102 determines that the accumulation level of the object pixel is accumulation level 1 (for example, immature dental plaque) (S124).

[0151] On the other hand, when the brightness of the target pixel is equal to or greater than the second threshold value (No in S123), the detection unit 102 determines that the accumulation level of the target pixel is accumulation level 2 (for example, mature dental plaque (tartar)) (S125).

[0152] In this way, the detection unit 102 determines the accumulation level for each pixel to detect the distribution of the accumulation level (concentration of the fluorescent substance). Here, the distribution of the accumulation level is information indicating the accumulation level for each two-dimensional position (eg, pixel) on the xy plane.

[0153] Next, the detection unit 102 allocates the three accumulation levels 0 to 2 to different shades, and generates a fourth RGB image in which the distribution of the accumulation levels indicated by the shades overlaps with the second RGB image. Fig.11 is a diagram showing an example of a fourth RGB image. Fig.11 As shown, a pattern of a first shade level (eg, 0.5) is superimposed on the region of immature plaque 306 , and a pattern of a second shade level (eg, 1.0) is superimposed on the region of mature plaque 305 .

[0154] In addition, in the above description, an example in which the cumulative level is three shades is shown, but the cumulative level may be four or more. In addition, the second image of the object overlapping with the distribution of the cumulative level may be an image other than the second RGB image. For example, the second image may be the first RGB image, or may be an image generated by performing image processing on the first RGB image or the second RGB image.

[0155] Fig.12 Yes Fig.11 The state of the teeth shown is an example of the state of the teeth after oral care. Fig.12 As shown, by performing oral care (brushing teeth, etc.), immature plaque 306 is removed, while mature plaque 305 is not removed.

[0156] Therefore, the detection unit 102 can determine the score of oral care (whether it is not brushed clean) based on the state of the immature dental plaque 306. In addition, the detection unit 102 can recommend the user to go to the dentist for a checkup and diagnosis based on the state of the mature dental plaque 305.

[0157] For example, the detection unit 102 calculates the ratio of the area of ​​the region of the immature dental plaque 306 to the area of ​​the tooth region. That is, the detection unit 102 calculates (the area of ​​the region of the immature dental plaque 306) / (the area of ​​the tooth region)×100{%} as the first area ratio of the immature dental plaque 306. In addition, the detection unit 102 calculates the ratio of the area of ​​the region of the mature dental plaque 305 to the area of ​​the tooth region. That is, the detection unit 102 calculates (the area of ​​the region of the mature dental plaque 305) / (the area of ​​the tooth region)×100{%} as the second area ratio of the mature dental plaque 305.

[0158] The detection unit 102 may use the calculated first area ratio to determine the score of the oral care. In addition, the detection unit 102 may use the calculated second area ratio to recommend the user to go to the dentist for a checkup. For example, when the second area ratio is greater than a predetermined threshold, the detection unit 102 may display a message to the user recommending that the user go to the dentist for a checkup.

[0159] In addition, the calculation and determination of the above-mentioned area ratio can be performed at once for all teeth in the oral cavity. That is, the area ratio can be the ratio of the area of ​​the total tooth region of all teeth to the total area of ​​dental plaque (immature dental plaque 306 or mature dental plaque 305). Alternatively, the calculation and determination of the above-mentioned area ratio can be performed separately for multiple teeth in the oral cavity. That is, the area ratio can be the ratio of the area of ​​the tooth region of one tooth to the area of ​​dental plaque of one tooth. Alternatively, the calculation and determination of the above-mentioned area ratio can be performed separately for each tooth region obtained by dividing the multiple teeth in the oral cavity. The tooth region refers to, for example, the deep right side of the upper jaw, the front left side of the lower jaw, and other regions that include more than two teeth.

[0160] also, Figure 5 The identification unit 104 shown identifies the types of multiple teeth in the image data based on the image data. Here, the type of tooth is information that can uniquely identify a tooth in the oral cavity, such as the central incisor on the right side of the upper jaw or the lateral incisor on the lower left jaw.

[0161] For example, the recognition unit 104 obtains reference data corresponding to the types of multiple teeth from the cloud server 80, and uses the image data and the obtained reference data to recognize the types of each of the multiple teeth included in the image data by comparing feature amounts.

[0162] The recognition unit 104 stores the concentration distribution (cumulative level) of the fluorescent substance detected by the detection unit 102 in the storage unit 105 in correspondence with the types of the plurality of teeth. That is, the storage unit 105 stores the concentration distribution (cumulative level) of the fluorescent substance for each tooth. In addition, the area ratio and determination process for each tooth described above can be performed using the information for each tooth.

[0163] The display unit 103 is a display device provided in the portable terminal 70, and displays a fourth RGB image in which the distribution of the accumulation level is superimposed on the image data. In addition, the display unit 103 displays the above-mentioned determination result and a message based on the determination result, etc. In addition, the display unit 103 can display the above-mentioned area ratio, etc.

[0164] Hereinafter, modified examples of the above-described embodiment will be described.

[0165] (Variant 1)

[0166] In the above embodiment, the detection unit 102 performs exposure control processing on the first RGB image and performs white balance adjustment processing on the third RGB image generated by the exposure control processing, but the present invention is not limited thereto and the exposure control processing may not be performed. For example, if the first RGB image with reduced uneven brightness distribution is obtained, the exposure control processing may not be performed. For example, the uneven brightness distribution of the obtained first RGB image may be reduced by performing lighting control to make the shooting conditions constant.

[0167] (Variant 2)

[0168] In the above-mentioned embodiment, the accumulation level is detected using image data after image processing (exposure control processing and white balance adjustment processing, etc.), but part or all of the image processing may not be performed. For example, an HSV image may be generated from the first RGB image, and the accumulation level may be detected using the HSV image.

[0169] (Variant 3)

[0170] In the above embodiment, the brightness of the HSV image is used to detect the cumulative level, but the method for determining the cumulative level is not limited thereto. For example, the chroma or hue of the HSV image may be used, or a combination of at least one of the chroma and hue and the brightness may be used. For example, an evaluation value calculated based on at least one of the chroma and hue and the brightness may be compared with a threshold value.

[0171] In addition, the accumulation level can be detected using an RGB image. For example, the R value of the RGB image can be used, or a combination of the R value and at least one of the G value and the B value can be used. For example, an evaluation value calculated based on the R value and at least one of the G value and the B value can be compared with a threshold value.

[0172] (Variant 4)

[0173] In the above embodiment, the intraoral camera 10 sends the first RGB image to the portable terminal 70, and the first RGB image is processed in the portable terminal 70, but the present invention is not limited thereto. The first RGB image may be sent to the cloud server 80, and the cloud server 80 performs the above processing, and the second RGB image or the fourth RGB image of the processing result is sent to the portable terminal 70. In this case, the first RGB image may be sent from the intraoral camera 10 to the cloud server 80 without passing through the portable terminal 70, or may be sent from the intraoral camera 10 to the cloud server 80 via the portable terminal 70.

[0174] (Variant 5)

[0175] In the above embodiment, the HSV image is used to detect the accumulation level, but the method of determining the accumulation level is not limited thereto. For example, an HSL image may be used instead of an HSV image.

[0176] For example, in the third image processing described above, the detection unit 102 generates an HSL image by converting the color space of the second RGB image into the HSL space. Then, the detection unit 102 determines a specific pixel region where at least one of the plurality of fifth pixels of the HSL image that satisfies at least one of the chroma within the fourth prescribed range, the hue within the fifth prescribed range, and the brightness within the sixth prescribed range is located as a dental plaque region.

[0177] In the above-mentioned fluorescent substance concentration distribution detection process, the detection unit 102 detects the fluorescent substance concentration distribution using the HSL image. Specifically, the detection unit 102 detects the fluorescent substance concentration distribution using the value of the brightness L of the HSL image.

[0178] Here, the HSL color space (also referred to as the HLS color space) is a color space composed of three components of H (hue), S (chroma), and L (brightness), and is obtained by nonlinear transformation of the RGB color space.

[0179] If the maximum value of the three values ​​of R, G, and B is set to MAX and the minimum value is set to MIN, the H (hue) of the HSL image is calculated using the above formula 5. In addition, L (brightness) is calculated using the following formula 17. S (chroma) is calculated using the following formula 18 when using the cylindrical model, and the following formula 19 when using the double cone model.

[0180] [Mathematical formula 6]

[0181]

[0182] S = MAX-MIN ... (Formula 19)

[0183] Thus, in the HSL space, H, S, and L are calculated from MAX (max (R, G, B)) and MIN (min (R, G, B)). In the fluorescent area of ​​dental plaque, when MIN = k × MAX is set, k = 0 to about 0.3.

[0184] Fig.13 It is a graph showing the relationship among fluorescence intensity, MIN and k. Fig.14 This is a graph showing the relationship among fluorescence intensity, S (chroma), and k. Fig.15 : is a graph showing the relationship between fluorescence intensity, L (brightness), and k. In addition, this example is an example in which a cylindrical model is used.

[0185] Figure 13 to Figure 15 The characteristics of k = 0 to 0.3 shown by the solid line in FIG. 1 correspond to the characteristics of the fluorescent area of ​​dental plaque. Fig.14 As shown in , when k = 0 to 0.3, S = 0.5 to 1. Fig.15 As shown, when k=0-0.3, L=0-0.65.

[0186] Therefore, when detecting dental plaque, when S is in the range of 0.5 to 1, L changes linearly at 0 to 0.65, so the value of L is proportional to the amount of porphyrin. Therefore, based on the value of L, the concentration distribution of the fluorescent substance (the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth) can be detected.

[0187] (Variant 6)

[0188] In the above description, the first RGB image is an image obtained by attenuating at least a portion of the blue light region from reflected light and fluorescence from teeth, plaque and tartar in the oral cavity irradiated with irradiation light of a specified wavelength, wherein the specified wavelength is a wavelength that excites fluorescent substances contained in plaque and tartar. As a method for generating the first RGB image, an example of using a blue light cutoff filter 20 is shown, but the blue light cutoff filter 20 may be omitted and at least a portion of the blue light region may be attenuated by signal processing.

[0189] Fig.16 1 is a diagram showing an example of pixel values ​​in each image when the blue light region is attenuated by signal processing. Fig.16 As shown, the fifth RGB image is generated by performing a blue cutoff process on the image data obtained by the imaging element 14, that is, the first RGB image. For example, the detection unit 102 generates the fifth RGB image by multiplying the blue pixel value of the first RGB image by gain=0. In addition, the detection unit 102 is not limited to gain=0, and the blue pixel value may be multiplied by a predetermined gain less than 1. Alternatively, the detection unit 102 may replace the blue pixel value with a predetermined value (e.g., 0), or may clip the blue pixel value to be less than a predetermined value.

[0190] The detection unit 102 generates a third RGB image by performing the above-described exposure control process on the fifth RGB image generated in this way. For example, the detection unit 102 multiplies R, G, and B by equal gains so that max(R, G, B) becomes a predetermined level.

[0191] Next, the detection unit 102 generates a second RGB image by performing the above-mentioned white balance adjustment process on the third RGB image. For example, the detection unit 102 multiplies R and B by gains individually so that R, B and G are at the same level. In addition, when performing a blue cutoff process, the detection unit 102 may not multiply B by a gain in the white balance adjustment process. In addition, the above-mentioned fluorescent substance concentration distribution detection process is performed using the second RGB image.

[0192] Even when the blue cut process is performed by signal processing in this way, it is possible to generate an image (fifth RGB image) in which at least a part of the blue light region is attenuated, similarly to the case of using the blue light cut filter 20 .

[0193] Additionally, the blue cutoff process may be performed on the second RGB image instead of on the first RGB image. Fig.17 FIG. 4 is a diagram showing an example of pixel values ​​in each image when the blue light region is attenuated by signal processing in this case.

[0194] exist Fig.17 In the example shown, the detection unit 102 generates a sixth RGB image by performing the same blue cutoff process as described above on the second RGB image after the white balance adjustment process. The above-described fluorescent substance concentration distribution detection process is performed using the sixth RGB image.

[0195] When compared Fig.16 The second RGB image and Fig.17 The sixth RGB image shown is Fig.16 The second RGB image shown is close to a state where blue is cut off using an optical filter, and the levels of remaining R and G also increase, making the image brighter.

[0196] In addition, the blue cutoff process can be a digital process or an analog process. In addition, in the above, an example of the detection unit 102 (portable terminal 70) performing the blue cutoff process is described, but it can also be performed by the intraoral camera 10. In addition, in a general image sensor used for an RGB camera, the output order of each pixel value of R, G, and B is determined according to the RGB color filter arrangement. Therefore, in the blue cutoff process, the pixel value of B can be determined.

[0197] (Variant 7)

[0198] The detection unit 102 can extract a natural tooth region without plaque from the first RGB image in the white balance adjustment process (first image processing), and use multiple pixels of the extracted natural tooth region to perform white balance adjustment processing. That is, the first image processing can be the following processing: extracting a natural tooth region without plaque and tartar from the first RGB image, adjusting the gain of at least two color components among the red component, green component and blue component of the first RGB image, so that the first red pixel average value of the multiple red pixel values ​​of the multiple first pixels constituting the natural tooth region, the first green pixel average value of the multiple green pixel values ​​of the multiple first pixels, and the first blue pixel average value of the multiple blue pixel values ​​of the multiple first pixels are equal.

[0199] The natural tooth region here refers to the region of the tooth region excluding the region of the artificial tooth. The artificial tooth is an artificial tooth or restoration made of, for example, metal (gold or silver, etc.), ceramic, or zirconia.

[0200] In this way, by performing the white balance adjustment process using information on pixels in the natural tooth region other than the artificial tooth region, it is possible to improve the accuracy of the white balance adjustment process.

[0201] Specifically, the detection unit 102 detects a first natural tooth region having a green pixel value (G) equal to or greater than a predetermined first threshold value in the first RGB image.

[0202] Here, when excitation light (blue light) is irradiated onto natural teeth, excitation fluorescence is emitted from the dentin. This excitation fluorescence passes through the enamel. As a result, the natural teeth emit green fluorescence. In addition, in the state of irradiating blue light, the filling for treating caries marks becomes darker (low brightness) in the image taken by the camera, unlike the state of irradiating white light. On the other hand, the natural teeth covered with enamel become brighter (high brightness) in the image. Therefore, by extracting the area where the green pixel value (G) is above a predetermined first threshold and extracting the green fluorescence, it is possible to distinguish the area of ​​natural teeth and exclude the area of ​​artificial teeth.

[0203] In addition, the brightness value (Y) can be used instead of the green pixel value (G). The brightness value (Y) is calculated using the above formula 3. As shown in formula 3, since the brightness value accounts for a large proportion of the green pixel value, the brightness value can also be used to perform the same detection as the case of using the green pixel value.

[0204] In addition, the detection unit 102 can use the information of the first natural tooth area detected in this way to perform white balance adjustment processing, or it can detect the second natural tooth area by further excluding the area of ​​plaque and tartar from the first natural tooth area, and use the information of the detected second natural tooth area to perform white balance adjustment processing.

[0205] Specifically, the detection unit 102 generates an HSV image based on the first RGB image, and extracts an area where the values ​​of H, S, and V of the HSV image are within a predetermined range as a dental plaque or calculus area. In addition, the detection unit 102 may also generate an HSL image from the first RGB image, and extract an area where the values ​​of H, S, and L of the HSL image are within a predetermined range as a dental plaque or calculus area.

[0206] Next, the detection unit 102 detects the second natural tooth region by excluding the plaque or calculus region from the first natural tooth region.

[0207] As described above, the dental plaque detection device (e.g., portable terminal 70) of the present embodiment comprises: an acquisition unit 101 that acquires a first image (e.g., a first RGB image), which is an image obtained by at least a portion of a blue light region attenuated by reflected light and fluorescence from teeth, dental plaque, and tartar in the oral cavity irradiated with irradiation light of a specified wavelength, wherein the irradiation light of the specified wavelength excites fluorescent substances contained in dental plaque and tartar; and a detection unit 102 that detects the content per unit area of ​​fluorescent substances contained in dental plaque and tartar attached to the teeth (e.g., the concentration distribution of fluorescent substances) based on the first image (e.g., the value of the fluorescence intensity based on the fluorescence reaction of the fluorescent substances in the first image). Thus, the dental plaque detection device can detect the content per unit area of ​​fluorescent substances contained in dental plaque and tartar attached to the teeth, and thus can detect the state of the teeth in detail. In addition, the plaque detection device can improve the detection accuracy of the content per unit area of ​​fluorescent substances contained in plaque and tartar attached to teeth by using a first image, wherein the first image is an image obtained by attenuating at least a portion of the blue light region from reflected light and fluorescence from teeth, plaque and tartar in the oral cavity irradiated with irradiation light of a specified wavelength, wherein the irradiation light of the specified wavelength excites the fluorescent substances contained in plaque and tartar.

[0208] For example, the detection unit 102 generates an HSV image based on the first image, and detects the content per unit area of ​​the fluorescent substance contained in the dental plaque and tartar attached to the teeth based on the brightness value of the HSV image. Thus, the dental plaque detection device can accurately detect the content per unit area of ​​the fluorescent substance contained in the dental plaque and tartar attached to the teeth based on the brightness value of the HSV image.

[0209] For example, the first image is a first RGB image, and the detection unit 102 generates a second RGB image by performing image processing including first image processing on the first RGB image. The first image processing is a process of adjusting the gain of at least two color components among the red component, the green component, and the blue component of the RGB image of the processing object so that the first red pixel average value of the multiple red pixel values ​​of the multiple first pixels constituting the tooth region in the RGB image of the processing object, the first green pixel average value of the multiple green pixel values ​​of the multiple first pixels, and the first blue pixel average value of the multiple blue pixel values ​​of the multiple first pixels are equal. The detection unit 102 generates an HSV image by converting the color space of the second RGB image into an HSV space.

[0210] Thus, the dental plaque detection device can adjust the white balance of the first RGB image of the tooth undergoing the fluorescent reaction by performing the first image processing. Therefore, the dental plaque detection device can generate a second RGB image that can easily distinguish the area on which dental plaque is attached in the tooth, that is, the dental plaque area. Therefore, the dental plaque detection device can improve the detection accuracy of the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth.

[0211] For example, the detection unit 102 determines a specific pixel area where at least one fourth pixel that satisfies the conditions that the chroma is within the first specified range, the hue is within the second specified range, and the lightness is within the third specified range among a plurality of fourth pixels of the HSV image is located, and detects the content per unit area of ​​the fluorescent substance contained in the plaque and tartar attached to the teeth according to the value of the lightness in the specific pixel area of ​​the HSV image. Thus, the plaque detection device detects the content per unit area of ​​the fluorescent substance contained in the plaque and tartar attached to the teeth on the basis of determining the plaque area in the image of the teeth, thereby being able to improve the detection accuracy of the content.

[0212] For example, the detection unit 102 generates an HSL image based on the first image, and detects the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth based on the brightness value of the HSL image. Thus, the dental plaque detection device can accurately detect the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth based on the brightness value of the HSL image.

[0213] For example, the first image is a first RGB image, and the detection unit 102 generates a second RGB image by performing image processing including first image processing on the first RGB image. The first image processing is a process of adjusting the gain of at least two color components among the red component, the green component, and the blue component of the RGB image of the processing object so that the first red pixel average value of the multiple red pixel values ​​of the multiple first pixels constituting the tooth region in the RGB image of the processing object, the first green pixel average value of the multiple green pixel values ​​of the multiple first pixels, and the first blue pixel average value of the multiple blue pixel values ​​of the multiple first pixels are equal. The detection unit 102 generates the HSL image by converting the color space of the second RGB image into the HSL space.

[0214] Thus, the dental plaque detection device can adjust the white balance of the first RGB image of the tooth undergoing the fluorescent reaction by performing the first image processing. Therefore, the dental plaque detection device can generate a second RGB image that can easily distinguish the area on which dental plaque is attached in the tooth, that is, the dental plaque area. Therefore, the dental plaque detection device can improve the detection accuracy of the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth.

[0215] For example, the detection unit 102 determines a specific pixel region where at least one of the fifth pixels of the HSL image that satisfies at least one of the conditions that the chroma is within the fourth prescribed range, the hue is within the fifth prescribed range, and the brightness is within the sixth prescribed range is located, and detects the content per unit area of ​​the fluorescent substance contained in the dental plaque and tartar attached to the teeth according to the value of the brightness in the determined pixel region. Thus, the dental plaque detection device detects the content per unit area of ​​the fluorescent substance contained in the dental plaque and tartar attached to the teeth on the basis of determining the dental plaque region in the image of the teeth, thereby being able to improve the detection accuracy of the content.

[0216] For example, the fluorescent substance is porphyrin. For example, the detection unit 102 distributes the content per unit area of ​​the fluorescent substance contained in the dental plaque and tartar attached to the teeth into three or more shades, and generates a third image (e.g., a fourth RGB image), wherein the third image is an image obtained by superimposing the content per unit area of ​​the fluorescent substance contained in the dental plaque and tartar attached to the teeth displayed by the shades on the second image (e.g., the first RGB image or the second RGB image) based on the first image. Thus, for example, based on the generated third image, the user can be notified of the content per unit area of ​​the fluorescent substance contained in the dental plaque and tartar attached to the teeth.

[0217] For example, the dental plaque detection device further includes: an identification unit 104 for identifying the type of the photographed tooth; and a storage unit 105 for storing the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the tooth detected from the photographed tooth in correspondence with the identified type of the tooth. Thus, the dental plaque detection device can manage the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the tooth for each tooth.

[0218] In addition, the dental plaque detection device (for example, the portable terminal 70) of the present embodiment comprises: an acquisition unit 101, which acquires a first RGB image from reflected light and fluorescence from teeth, dental plaque and tartar in the oral cavity irradiated with irradiation light of a predetermined wavelength, wherein the irradiation light of the predetermined wavelength excites fluorescent substances contained in dental plaque and tartar; and a detection unit 102, which generates a second RGB image by performing image processing including a first image processing on the first RGB image, and detects the content per unit area of ​​fluorescent substances contained in dental plaque and tartar attached to the teeth based on the value of the fluorescence intensity of the fluorescent reaction of the fluorescent substances in the second RGB image. The first image processing is a process as follows: a natural tooth region to which dental plaque and tartar are not attached is extracted from the first RGB image, and the gains of at least two color components among the red component, the green component and the blue component of the first RGB image are adjusted so that the first red pixel average value of the multiple red pixel values ​​of the multiple first pixels constituting the natural tooth region, the first green pixel average value of the multiple green pixel values ​​of the multiple first pixels, and the first blue pixel average value of the multiple blue pixel values ​​of the multiple first pixels are equal.

[0219] Thus, the plaque detection device can detect the content per unit area of ​​fluorescent substances contained in plaque and tartar attached to teeth, and thus can detect the state of teeth in detail. In addition, the plaque detection device can adjust the white balance of the first RGB image of teeth undergoing a fluorescent reaction by performing the first image processing. Therefore, the plaque detection device can generate a second RGB image that can easily distinguish the area on which plaque is attached in the teeth, i.e., the plaque area. Therefore, the plaque detection device can improve the detection accuracy of the content per unit area of ​​fluorescent substances contained in plaque and tartar attached to teeth. In addition, the plaque detection device can improve the accuracy of the white balance adjustment processing by performing the first image processing using pixels of the natural tooth area to which plaque and tartar are not attached.

[0220] For example, in the extraction of the natural tooth region, a first region (e.g., a first natural tooth region) is detected as (i) a region in which the brightness value is greater than a predetermined first threshold value in the full pixel region of the first RGB image, or (ii) a region in which the green pixel value is greater than a predetermined second threshold value in the full pixel region of the first RGB image, and the natural tooth region is extracted based on the first region. Thus, the dental plaque detection device can detect the natural tooth region with good accuracy using the brightness value or the green pixel value.

[0221] For example, in the extraction of the natural tooth region, the region after excluding the region of dental plaque and calculus is extracted from the first region as the natural tooth region. Thus, the dental plaque detection device can improve the accuracy of the white balance adjustment process.

[0222] For example, the first RGB image is an image obtained by attenuating at least a portion of the blue light region from reflected light and fluorescence from teeth and dental plaque in the oral cavity. Thus, the dental plaque detection device can improve the detection accuracy of the content per unit area of ​​fluorescent substances contained in dental plaque and tartar attached to teeth by using the first image, which is an image obtained by attenuating at least a portion of the blue light region from reflected light and fluorescence from teeth, dental plaque and dental plaque in the oral cavity irradiated with irradiation light of a specified wavelength, wherein the specified wavelength is a wavelength that excites fluorescent substances contained in dental plaque and dental plaque.

[0223] For example, the detection unit 102 generates an HSV image from the second RGB image, and detects the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth based on the brightness value of the HSV image. Thus, the dental plaque detection device can accurately detect the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth based on the brightness value of the HSV image.

[0224] For example, the detection unit 102 determines a specific pixel area where at least one fourth pixel that satisfies the conditions that the chroma is within the first specified range, the hue is within the second specified range, and the lightness is within the third specified range among a plurality of fourth pixels in the HSV image is located, and detects the content per unit area of ​​the fluorescent substance contained in the dental plaque and tartar attached to the teeth according to the value of the lightness in the determined pixel area. Thus, the dental plaque detection device detects the content per unit area of ​​the fluorescent substance contained in the dental plaque and tartar attached to the teeth on the basis of determining the dental plaque area in the image of the teeth, thereby being able to improve the detection accuracy of the content.

[0225] For example, the detection unit 102 generates an HSL image from the second RGB image, and detects the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth based on the brightness value of the HSL image. Thus, the dental plaque detection device can accurately detect the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth based on the brightness value of the HSL image.

[0226] For example, the detection unit 102 determines a specific pixel region where at least one of the fifth pixels of the HSL image that satisfies at least one of the conditions that the chroma is within the fourth prescribed range, the hue is within the fifth prescribed range, and the brightness is within the sixth prescribed range is located, and detects the content per unit area of ​​the fluorescent substance contained in the dental plaque and tartar attached to the teeth according to the value of the brightness in the determined pixel region. Thus, the dental plaque detection device detects the content per unit area of ​​the fluorescent substance contained in the dental plaque and tartar attached to the teeth on the basis of determining the dental plaque region in the image of the teeth, thereby being able to improve the detection accuracy of the content.

[0227] For example, the fluorescent substance is porphyrin. For example, the detection unit 102 distributes the content per unit area of ​​the fluorescent substance contained in the dental plaque and tartar attached to the teeth into three or more shades, and generates a third image (e.g., a fourth RGB image), wherein the third image is an image obtained by superimposing the content per unit area of ​​the fluorescent substance contained in the dental plaque and tartar attached to the teeth showing the shades on the second image (e.g., the first RGB image or the second RGB image) based on the first RGB image. Thus, for example, based on the generated third image, the user can be notified of the content per unit area of ​​the fluorescent substance contained in the dental plaque and tartar attached to the teeth.

[0228] For example, the dental plaque detection device further includes: an identification unit 104 for identifying the type of the photographed tooth; and a storage unit 105 for storing the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the tooth detected from the photographed tooth in correspondence with the identified type of the tooth. Thus, the dental plaque detection device can manage the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the tooth for each tooth.

[0229] As mentioned above, the intraoral camera system according to the embodiment of the present disclosure has been described, but the present disclosure is not limited to this embodiment.

[0230] For example, in the above description, an example of using the intraoral camera 10 mainly for photographing teeth is described, but the intraoral camera 10 may also be an intraoral care device equipped with a camera. For example, the intraoral camera 10 may be an intraoral cleaner equipped with a camera.

[0231] In addition, each processing unit included in the intraoral camera system of the above-mentioned embodiment is typically implemented by an LSI which is an integrated circuit. They may be integrated into one chip individually or in part or in whole.

[0232] In addition, circuit integration is not limited to LSI, and can also be realized using dedicated circuits or general-purpose processors. FPGA (Field Programmable Gate Array) that can be programmed after LSI manufacturing or reconfigurable processors that can reconfigure the connections and settings of circuit cells inside LSI can be used.

[0233] In addition, in each of the above-mentioned embodiments, each component element may be formed by dedicated hardware, or may be implemented by executing a software program suitable for each component element. Each component element may be implemented by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

[0234] In addition, the present disclosure may be implemented as an image display method performed by an intraoral camera system, etc. In addition, the present disclosure may be implemented as an intraoral camera, a portable terminal, or a cloud server included in the intraoral camera system.

[0235] In addition, the division of the functional blocks in the block diagram is an example, and multiple functional blocks can be implemented as one functional block, or one functional block can be divided into multiple blocks, or a part of the function can be transferred to other functional blocks. In addition, a single hardware or software can process the functions of multiple functional blocks with similar functions in parallel or in time division.

[0236] In addition, the order of executing each step in the flowchart is used for illustration to specifically explain the present disclosure, and may be an order other than the above. In addition, some of the above steps may be executed simultaneously (in parallel) with other steps.

[0237] In the above, one or more modes of intraoral camera systems and the like are described based on the embodiments, but the present disclosure is not limited to the embodiments. As long as it does not deviate from the main purpose of the present disclosure, various modifications thought of by those skilled in the art applied to the present embodiment and the modes constructed by combining the constituent elements in different embodiments may also be included in the scope of one or more modes.

[0238] Industrial Applicability

[0239] The present disclosure can be applied to intraoral camera systems.

[0240] Description of Reference Numerals

[0241] 10Intraoral Camera

[0242] 10a Head

[0243] 10b Handle

[0244] 10c Neck

[0245] 12Photographic Optical System

[0246] 12a Entrance

[0247] 14 Camera Components

[0248] 16 lenses

[0249] 18 reflectors

[0250] 20 blue light cutoff filters

[0251] 24 aperture

[0252] 26A First LED

[0253] 26B Second LED

[0254] 26C Third LED

[0255] 26D Fourth LED

[0256] 28 caps

[0257] 30 Composition adjustment mechanism

[0258] 32 Focus adjustment mechanism

[0259] 34 frame

[0260] 36, 40 actuator

[0261] 38 Lens Holder

[0262] 50 Central Control Department

[0263] 54LED control unit

[0264] 56 lens driver

[0265] 58 wireless communication module

[0266] 60 Power supply control unit

[0267] 62 Controller

[0268] 64 Memory

[0269] 66 batteries

[0270] 68 coils

[0271] 69 Charger

[0272] 70 Portable Terminal

[0273] 72 Touch screen

[0274] 80 cloud servers

[0275] 90 Position Sensor

[0276] 101 Acquisition Department

[0277] 102 Detection Department

[0278] 103 Display unit

[0279] 104 Identification Department

[0280] 105 Storage

[0281] 301 Teeth

[0282] 302 gums

[0283] 303 Tartar

[0284] Area 304

[0285] 305 Mature Tartar

[0286] 306 Immature Tartar

Claims

1. A dental plaque detection device, wherein: have: an acquisition unit that acquires a first RGB image from reflected light and fluorescence from teeth, dental plaque, and calculus in the oral cavity irradiated with irradiation light of a predetermined wavelength that excites fluorescent substances contained in the dental plaque and calculus; as well as a detection unit that generates a second RGB image by performing image processing including first image processing on the first RGB image, and detects the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the tooth based on the value of the fluorescence intensity of the fluorescent reaction of the fluorescent substance in the second RGB image, The first image processing is as follows: extracting a natural tooth area without attached plaque and tartar from the first RGB image, and adjusting the gains of at least two color components among the red component, green component and blue component of the first RGB image so that the first red pixel average value of the multiple red pixel values ​​of the multiple first pixels constituting the natural tooth area, the first green pixel average value of the multiple green pixel values ​​of the multiple first pixels, and the first blue pixel average value of the multiple blue pixel values ​​of the multiple first pixels are equal.

2. The dental plaque detection device according to claim 1, wherein: In the extraction of the natural tooth area, A first area is detected and the natural tooth area is extracted based on the first area, wherein the first area is (i) an area in the full pixel area of ​​the first RGB image whose brightness value is above a predetermined first threshold, or (ii) an area in the full pixel area of ​​the first RGB image whose green pixel value is above a predetermined second threshold.

3. The dental plaque detection device according to claim 2, wherein: In the extraction of the natural tooth area, A region excluding regions of plaque and calculus is extracted from the first region as the natural tooth region.

4. The dental plaque detection device according to any one of claims 1 to 3, wherein: The first RGB image is an image in which at least a portion of a blue light region is attenuated by reflected light and fluorescence from the teeth and dental plaque in the oral cavity.

5. The dental plaque detection device according to any one of claims 1 to 4, wherein: The detection unit generates an HSV image based on the second RGB image, and detects the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth based on a value of brightness of the HSV image.

6. The dental plaque detection device according to claim 5, wherein: The detection unit, Determine a pixel region where one or more fourth pixels satisfying at least one of the chroma within a first prescribed range, the hue within a second prescribed range, and the lightness within a third prescribed range among a plurality of fourth pixels of the HSV image are located, The content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth is detected based on the value of the brightness in the determination pixel area.

7. The dental plaque detection device according to any one of claims 1 to 4, wherein: The detection unit generates an HSL image based on the second RGB image, and detects the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth based on a brightness value of the HSL image.

8. The dental plaque detection device according to claim 7, wherein: The detection unit, determining a pixel region where one or more fifth pixels satisfying at least one of chroma within a fourth prescribed range, hue within a fifth prescribed range, and brightness within a sixth prescribed range are located among a plurality of fifth pixels possessed by the HSL image, The content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth is detected based on the value of the brightness in the determination pixel area.

9. The dental plaque detection device according to any one of claims 1 to 8, wherein: The fluorescent substance is porphyrin.

10. The dental plaque detection device according to any one of claims 1 to 9, wherein: The detection unit, The content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth is allocated to three or more shades of light and dark, A third image is generated, the third image being an image in which the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the teeth showing light and dark levels is superimposed on the second image based on the first RGB image.

11. The dental plaque detection device according to any one of claims 1 to 10, wherein: The dental plaque detection device also has: An identification unit, which identifies the type of teeth photographed; as well as The storage unit stores the content per unit area of ​​the fluorescent material contained in the dental plaque and calculus attached to the teeth detected from the imaged teeth in association with the recognized type of the teeth.

12. A method for detecting dental plaque, wherein: Acquiring a first RGB image from reflected light and fluorescence from teeth, plaque and calculus in the oral cavity irradiated with irradiation light of a predetermined wavelength, wherein the irradiation light of the predetermined wavelength excites fluorescent substances contained in the plaque and calculus; as well as generating a second RGB image by performing image processing including first image processing on the first RGB image, detecting the content per unit area of ​​the fluorescent substance contained in the dental plaque and calculus attached to the tooth based on the value of the fluorescence intensity of the fluorescent reaction of the fluorescent substance in the second RGB image, The first image processing is as follows: extracting a natural tooth area without attached plaque and tartar from the first RGB image, and adjusting the gains of at least two color components among the red component, green component and blue component of the first RGB image so that the first red pixel average value of the multiple red pixel values ​​of the multiple first pixels constituting the natural tooth area, the first green pixel average value of the multiple green pixel values ​​of the multiple first pixels, and the first blue pixel average value of the multiple blue pixel values ​​of the multiple first pixels are equal.

13. A program, wherein: Used to cause a computer to execute the dental plaque detection method according to claim 12.

Images