Information processing device, information processing method, and program

Abstract

An information processing device according to the present invention comprises: an acquisition unit (for example, a communication unit (51)) that acquires a first image including a specific tooth, obtained by imaging the inside of an oral cavity; a detection unit (52) that detects a first tooth region of the specific tooth from the first image; and a selection unit (53) that, when the ratio of the first tooth region in the first image is equal to or greater than a prescribed value, selects the first image as a reference image of the specific tooth.

Description

Information processing device, information processing method, and program

[0001] The present disclosure relates to an information processing device, an information processing method, and a program.

[0002] Patent Document 1 discloses an apparatus for setting a tooth region in a captured image of teeth arranged side by side.

[0003] Japanese Patent Application Laid-Open No. 2017-18152

[0004] It is desirable to perform processing such as setting plaque regions on an image suitable for checking the state of the oral cavity, but Patent Document 1 does not disclose a technology for acquiring an image suitable for checking the state of the oral cavity.

[0005] Therefore, the present disclosure provides an information processing device, an information processing method, and a program that can acquire images that are more suitable for checking the condition of the oral cavity.

[0006] An information processing device according to one aspect of the present disclosure includes an acquisition unit that acquires a first image including a specific tooth obtained by photographing the inside of the oral cavity, a detection unit that detects a first tooth region of the specific tooth from the first image, and a selection unit that selects the first image as a reference image of the specific tooth if the proportion of the first tooth region in the first image is equal to or greater than a predetermined value.

[0007] An information processing method according to one aspect of the present disclosure acquires an image including a specific tooth obtained by photographing the oral cavity, detects the tooth region of the specific tooth from the image, and, if the proportion of the tooth region in the image is equal to or greater than a predetermined value, selects the image as a reference image of the specific tooth.

[0008] A program according to one aspect of the present disclosure is a program for causing a computer to execute the above-described information processing method.

[0009] According to one aspect of the present disclosure, it is possible to realize an information processing device or the like that can acquire images that are more suitable for checking the condition of the oral cavity.

[0010] FIG. 1 is a perspective view of an intraoral camera in an information processing system according to an embodiment. FIG. 2 is a schematic configuration diagram of the information processing system according to an embodiment. FIG. 3 is a block diagram showing the functional configuration of a mobile terminal according to an embodiment. FIG. 4 is a flowchart showing a first operation of the information processing system according to an embodiment. FIG. 5 is a diagram for explaining step S13 shown in FIG. 4. FIG. 6 is a flowchart showing a second operation of the information processing system according to an embodiment. FIG. 7A is a first diagram for explaining step S24 shown in FIG. 6. FIG. 7B is a second diagram for explaining step S24 shown in FIG. 6. FIG. 8 is a flowchart showing an operation for evaluating an oral care state according to an embodiment. FIG. 9 is a diagram for explaining an imaging guide in an information processing system according to a first modification of the embodiment. FIG. 10 is a flowchart showing an operation of the information processing system according to the first modification of the embodiment. FIG. 11 is a diagram showing a screen for selecting a tooth to be imaged according to a second modification of the embodiment.

[0011] An information processing device according to a first aspect of the present disclosure includes an acquisition unit that acquires a first image including a specific tooth obtained by photographing the inside of the oral cavity, a detection unit that detects a first tooth region of the specific tooth from the first image, and a selection unit that selects the first image as a reference image of the specific tooth if the proportion of the first tooth region in the first image is equal to or greater than a predetermined value.

[0012] This makes it possible to obtain a reference image that has a larger tooth region than an image in which the proportion of the first tooth region is less than the predetermined value, making it easier to check the condition of a specific tooth. Thus, it is possible to obtain an image that is more suitable for checking the condition of the oral cavity.

[0013] Also, for example, an information processing device according to a second aspect may be the information processing device according to the first aspect, wherein the acquisition unit further acquires a second image including the specific tooth obtained by photographing the oral cavity, the detection unit detects a second tooth region of the specific tooth from the second image, and the selection unit selects the second image having a second tooth region similar to the first tooth region of the first image as a comparison image to be compared with the reference image.

[0014] This allows a comparison image to be acquired that has a similar dental region to the reference image. Such a comparison image is an image that makes it easy to compare the state of the dental region with the reference image. Therefore, an image that is more suitable for comparing and checking the state of the oral cavity can be acquired.

[0015] Also, for example, an information processing device according to a third aspect may be the information processing device according to the second aspect, wherein the acquisition unit acquires one or more of the second images, and the selection unit selects as the comparison image a second image from among the one or more second images, in which the degree of overlap between the second tooth area in the second image and the first tooth area in the reference image is equal to or greater than a predetermined degree.

[0016] This allows reference images and comparison images with a large overlap of tooth regions to be selected, so that, for example, when calculating a plaque score that indicates the condition of the oral cavity using the area of ​​the tooth region, a more accurate plaque score can be calculated.

[0017] Furthermore, for example, an information processing device according to the fourth aspect may be an information processing device according to the second or third aspect, and may further include a control unit that, when an imaging device that photographs the oral cavity photographs the second image, superimposes an imaging guide based on the reference image on the image obtained from the imaging device and displayed on a display device.

[0018] This can assist the user in taking a second image that is similar to the reference image.

[0019] Also, for example, an information processing device according to a fifth aspect may be an information processing device according to the fourth aspect, and the imaging guide may include a silhouette, outline, or translucent display of the specific tooth included in the reference image.

[0020] This may assist the user in capturing a second image similar to the reference image by displaying a silhouette, outline, or semi-transparent representation of a particular tooth.

[0021] Furthermore, for example, an information processing device according to a sixth aspect may be an information processing device according to any one of the second to fifth aspects, wherein the detection unit further detects a first plaque region of the specific tooth based on the reference image and a second plaque region of the specific tooth based on the comparison image, and further includes a score calculation unit that calculates a first plaque score based on the first tooth region and the first plaque region in the reference image and calculates a second plaque score based on the second tooth region and the second plaque region in the comparison image, and a comparison unit that compares oral care conditions based on the first plaque score and the second plaque score.

[0022] This allows the plaque score to be calculated using an image with a large tooth region, which can result in a more accurate plaque score. Therefore, it is possible to obtain an image that is more suitable for comparing plaque scores.

[0023] Also, for example, an information processing device according to a seventh aspect may be the information processing device according to the sixth aspect, wherein the score calculation unit calculates the first plaque score based on the ratio of the number of pixels in the first plaque region to the number of pixels in the natural tooth region in the reference image, and calculates the second plaque score based on the ratio of the number of pixels in the second plaque region to the number of pixels in the natural tooth region in the comparison image.

[0024] This makes it possible to calculate a more accurate plaque score when calculating the plaque score based on the ratio of the number of pixels in the plaque region to the number of pixels in the natural tooth region.

[0025] Also, for example, an information processing device according to an eighth aspect is an information processing device according to any one of the first to seventh aspects, and the selection unit may select the first image as the reference image if the ratio of the number of pixels of the first tooth region to the total number of pixels of the first image is equal to or greater than the predetermined value.

[0026] This makes it possible to select an image that is more suitable for checking the state of the oral cavity by using the ratio of the number of pixels in the first tooth region to the total number of pixels.

[0027] Furthermore, for example, an information processing device according to the ninth aspect may be an information processing device according to any one of the second to seventh aspects, wherein the first image is an image of the inside of the oral cavity before the user performs oral care treatment, and the second image is an image of the inside of the oral cavity after the user has performed oral care treatment.

[0028] This allows the user to obtain a reference image and a comparison image that allow for more appropriate comparison before and after oral care treatment.

[0029] An information processing method according to a tenth aspect of the present disclosure acquires an image including a specific tooth obtained by photographing the inside of the oral cavity, detects the tooth region of the specific tooth from the image, and if the proportion of the tooth region in the image is equal to or greater than a predetermined value, selects the image as a reference image of the specific tooth.

[0030] This provides the same effects as the above-described information processing device.

[0031] A program according to an eleventh aspect of the present disclosure is a program for causing a computer to execute the above-described information processing method.

[0032] This provides the same effects as the above-described information processing device.

[0033] These general or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or as any combination of the system, method, integrated circuit, computer program, or recording medium. The program may be pre-stored in the recording medium, or may be supplied to the recording medium via a wide area communication network including the Internet.

[0034] Furthermore, each figure is a schematic diagram and is not necessarily an exact illustration. Therefore, for example, the scales of the figures do not necessarily match. Furthermore, in each figure, substantially the same components are given the same reference numerals, and redundant explanations are omitted or simplified.

[0035] Furthermore, in this specification, terms indicating relationships between elements such as parallelism, as well as numerical values ​​and numerical ranges, are not expressions that only express a strict meaning, but are expressions that also include a substantially equivalent range, for example, a difference of about several percent (or about 10%).

[0036] Furthermore, in this specification, ordinal numbers such as "first" and "second" do not refer to the number or order of components unless otherwise specified, but are used for the purpose of avoiding confusion and distinguishing between components of the same type.

[0037] (Embodiment) Hereinafter, an information processing system according to the present embodiment will be described with reference to FIGS.

[0038] [1. Configuration of Information Processing System] First, the configuration of an information processing system including an intraoral camera according to the present embodiment will be described with reference to Figures 1 to 3. Figure 1 is a perspective view of an intraoral camera 10 in the information processing system according to the present embodiment.

[0039] As shown in FIG. 1, the intraoral camera 10 has a toothbrush-shaped housing that can be handled with one hand, and the housing has a head portion 10a that is placed in the user's oral cavity when taking a photograph, a handle portion 10b that the user holds, and a neck portion 10c that connects the head portion 10a and the handle portion 10b.

[0040] The imaging unit 21 images the surfaces of the dentition and dental plaque in the oral cavity irradiated with light including the wavelength range of blue light. The surfaces of the dentition include at least one of the buccal (outer) side surface of the dentition, the lingual (inner) side surface of the dentition, and the occlusal surface of the dentition. The dentition may include, for example, one or more teeth. The imaging unit 21 is an example of an imaging device.

[0041] The photographing unit 21 is incorporated into the head unit 10a and the neck unit 10c. The photographing unit 21 has an image pickup element (not shown) and a lens (not shown) arranged on its optical axis LA.

[0042] The imaging element is a photographing device such as a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) element, and an image of the teeth is formed by a lens. The imaging element outputs a signal (image data) corresponding to the formed image to the outside. The image photographed by the imaging element is also referred to as an RGB image. The RGB image is an image obtained by irradiating the dentition with blue light, and may be, for example, an image of the side of the dentition or an image of the occlusal surface of the dentition. The image includes, for example, one or more images (e.g., time-series images) photographed along the direction of the dentition. The side of the dentition may be a lingual side or a buccal side. The RGB image includes, for example, one or more teeth.

[0043] The photographing unit 21 may further include an optical filter that blocks light of a color emitted from the illumination unit (illumination device) and transmits fluorescence emitted by plaque in response to the light. In this embodiment, the photographing unit 21 may include, as an optical filter, a blue light cut filter that cuts out blue wavelength light components contained in light incident on the image sensor. When light including a blue wavelength range is applied to teeth to detect plaque, if the light including the blue wavelength range is increased to enhance the excitation fluorescence of plaque, the blue pixel values ​​will become dominant over the red and green pixel values, and the entire RGB image will appear blue. To address this, a blue light cut filter cuts out a portion of the light including the blue wavelength range from the light before it enters the image sensor. Note that the photographing unit 21 does not necessarily have to include a blue light cut filter.

[0044] The intraoral camera 10 is also equipped with a plurality of first to fourth LEDs 23A to 23D as an illumination unit that irradiates light onto the teeth to be photographed during photography. The first to fourth LEDs 23A to 23D irradiate dental plaque with light of a color that causes the plaque to fluoresce (e.g., single-color light). The first to fourth LEDs 23A to 23D are, for example, blue LEDs that irradiate blue light including a wavelength having a peak at 405 nm (an example of a predetermined wavelength). Note that the first to fourth LEDs 23A to 23D are not limited to blue LEDs, and may be any light source that irradiates light including the wavelength range of blue light.

[0045] 2 is a schematic diagram of an information processing system according to this embodiment. The information processing system according to this embodiment is generally configured such that the imaging unit 21 captures fluorescence emitted by dental plaque in response to light from the illumination unit 23, and acquires an image more suitable for confirming the condition of the oral cavity from one or more captured RGB images (one or more images). Furthermore, the information processing system according to this embodiment is configured to select two images that allow the condition of the oral cavity to be compared.

[0046] As shown in FIG. 2 , the information processing system includes an intraoral camera 10 and a mobile terminal 50 .

[0047] The intraoral camera 10 includes a hardware unit 20, a signal processing unit 30, and a communication unit 40.

[0048] The hardware unit 20 is a physical element of the intraoral camera 10 and includes an imaging unit 21 , a sensor unit 22 , an illumination unit 23 , and an operation unit 24 .

[0049] The photographing unit 21 generates image data by photographing the teeth in the user's oral cavity. It can also be said that the photographing unit 21 generates image data by photographing the surface of the dentition and dental plaque in the oral cavity, which are irradiated with light of a predetermined wavelength that excites fluorescent substances contained in the plaque. The photographing unit 21 receives a control signal from the camera control unit 31, performs operations such as photographing in accordance with the received control signal, and outputs image data of a moving image or a still image obtained by photographing to the image processing unit 32. The photographing unit 21 has the above-mentioned image sensor, optical filter, and lens. The image data is generated based on light that has passed through the optical filter, for example. Furthermore, the image data is an image showing multiple teeth, but it is sufficient that the image shows at least one tooth.

[0050] The sensor unit 22 detects external light incident on the imaging area of ​​the RGB image. For example, the sensor unit 22 detects whether external light is incident into the oral cavity. The sensor unit 22 is disposed, for example, near the imaging unit 21. The sensor unit 22 may be disposed, for example, in the head unit 10a of the intraoral camera 10, similar to the imaging unit 21. That is, the sensor unit 22 is located in the user's oral cavity when the imaging unit 21 captures an image. Note that, instead of using the sensor unit 22, the image processing unit 32 (described later) may detect external light using the image captured by the imaging unit 21. That is, the signal processing unit 30 (for example, the image processing unit 32) may have the function of the sensor unit 22.

[0051] The illumination unit 23 irradiates light onto a region among multiple regions in the oral cavity that is to be photographed by the imaging unit 21. As is also known from quantitative light-induced fluorescence (QLF (Quantitative Light-induced Fluorescence)), porphyrin, a substance produced by bacteria in dental plaque, is known to fluoresce a reddish pink color (excited fluorescence) when irradiated with blue light. In this embodiment, the illumination unit 23 irradiates blue light onto the region that is to be photographed by the imaging unit 21.

[0052] The illumination unit 23 has the above-mentioned first to fourth LEDs 23A to 23D. The first to fourth LEDs 23A to 23D irradiate the image capture area with light from different directions, for example. This makes it possible to prevent shadows from appearing in the image capture area.

[0053] Each of the first to fourth LEDs 23A to 23D is configured to be at least controllable in terms of dimming. Each of the first to fourth LEDs 23A to 23D may be configured to be controllable in terms of dimming and color adjustment. The first to fourth LEDs 23A to 23D are arranged to surround the imaging unit 21.

[0054] The illumination unit 23 controls the illumination intensity (light emission intensity) according to the photographing area. The illumination intensity of each of the first to fourth LEDs 23A to 23D may be controlled uniformly, or may be controlled to be different from one another. The number of LEDs included in the illumination unit 23 is not particularly limited, and may be one, or five or more. Furthermore, the illumination unit 23 is not limited to having an LED as a light source, and may include other light sources.

[0055] The operation unit 24 receives operations from the user. The operation unit 24 is configured with, for example, push buttons, but may also be configured to receive operations by voice or the like.

[0056] The hardware unit 20 may further include a battery (e.g., a secondary battery) that supplies power to each component of the intraoral camera 10, a coil for wireless charging by an external charger connected to a commercial power source, and an actuator necessary for at least one of composition adjustment and focus adjustment.

[0057] The signal processing unit 30 has functional components implemented by a CPU (Central Processing Unit) or an MPU (Micro Processor Unit) that execute various processes described below, and a memory unit 35 such as a ROM (Read Only Memory) or RAM (Random Access Memory) that stores programs for causing the functional components to execute various processes. Specifically, the signal processing unit 30 has a camera control unit 31, an image processing unit 32, a control unit 33, a lighting control unit 34, and the memory unit 35.

[0058] The camera control unit 31 is mounted on, for example, the handle unit 10b of the intraoral camera 10 and controls the imaging unit 21. The camera control unit 31 controls at least one of the aperture and the shutter speed of the imaging unit 21 in response to a control signal from the image processing unit 32, for example.

[0059] The image processing unit 32 is mounted on, for example, the handle unit 10b of the intraoral camera 10, acquires the RGB image (image data) captured by the imaging unit 21, performs image processing on the acquired RGB image, and outputs the processed RGB image to the camera control unit 31 and the control unit 33. The image processing unit 32 may also output the processed RGB image to the memory unit 35, and store the processed RGB image in the memory unit 35.

[0060] The image processing unit 32 is configured by, for example, a circuit, and performs image processing such as noise removal, edge enhancement, etc. on the RGB image. Note that the noise removal, edge enhancement, etc. may be performed by the mobile terminal 50.

[0061] The RGB image output from the image processing unit 32 (the RGB image after image processing) may be transmitted to the portable terminal 50 via the communication unit 40, and an image based on the transmitted RGB image may be displayed on the display unit 56 of the portable terminal 50. This allows the user to be presented with an image based on the RGB image.

[0062] The control unit 33 is a control device that controls the signal processing unit 30. The control unit 33 controls each component of the signal processing unit 30 based on the detection result of the sensor unit 22, such as external light.

[0063] The illumination control unit 34 is mounted, for example, on the handle portion 10b of the intraoral camera 10 and controls the turning on and off of the first to fourth LEDs 23A to 23D. The illumination control unit 34 is configured, for example, by a circuit. For example, when a user performs an operation on the display unit 56 of the mobile terminal 50 to start the intraoral camera 10, a corresponding signal is transmitted from the mobile terminal 50 to the signal processing unit 30 via the communication unit 40. The illumination control unit 34 of the signal processing unit 30 turns on the first to fourth LEDs 23A to 23D based on the received signal.

[0064] In addition to the above programs, the memory unit 35 stores RGB images (image data) captured by the image capturing unit 21. The memory unit 35 is realized by, for example, a semiconductor memory such as a ROM or a RAM, but is not limited to this.

[0065] The communication unit 40 is a wireless communication module for wirelessly communicating with the mobile terminal 50. The communication unit 40 is mounted, for example, on the handle portion 10b of the intraoral camera 10, and performs wireless communication with the mobile terminal 50 based on a control signal from the signal processing unit 30. The communication unit 40 performs wireless communication with the mobile terminal 50 in accordance with an existing communication standard such as Wi-Fi (registered trademark) or Bluetooth (registered trademark). Via the communication unit 40, an RGB image is transmitted from the intraoral camera 10 to the mobile terminal 50, and an operation signal is transmitted from the mobile terminal 50 to the intraoral camera 10.

[0066] The mobile terminal 50 executes a process for obtaining an image more suitable for checking the condition of the oral cavity, for example, by using an RGB image of the surface of the dentition and dental plaque that have reacted to fluorescence when the teeth are irradiated with light including a wavelength range of blue light. The mobile terminal 50 functions as a user interface for the information processing system. The mobile terminal 50 is also an example of an information processing device.

[0067] FIG. 3 is a block diagram showing the functional configuration of the mobile terminal 50 according to this embodiment.

[0068] 3 , the mobile terminal 50 includes a communication unit 51, a detection unit 52, a selection unit 53, a score calculation unit 54, a control unit 55, and a display unit 56. The mobile terminal 50 includes a processor, a memory, and the like. The memory is a ROM, a RAM, or the like, and can store a program executed by the processor. The communication unit 51, the detection unit 52, the selection unit 53, the score calculation unit 54, the control unit 55, and the display unit 56 are realized by the processor or the like that executes a program stored in the memory. The mobile terminal 50 may be realized, for example, by a smartphone or tablet terminal capable of wireless communication.

[0069] The communication unit 51 is a wireless communication module for wireless communication with the intraoral camera 10. The communication unit 51 acquires RGB images from the intraoral camera 10. Specifically, the communication unit 51 acquires images including one or more teeth generated by the imaging unit 21. The RGB images are images obtained by the intraoral camera 10 by irradiating the teeth with light including a wavelength range of blue light and imaging the teeth that are undergoing a fluorescent reaction. The communication unit 51 may include a wired communication module that communicates with the intraoral camera 10 via a wire. The communication unit 51 is an example of an acquisition unit. Hereinafter, RGB images will also be referred to as images.

[0070] The detection unit 52 detects a tooth region and a plaque region of a specific tooth from the image acquired by the communication unit 51. Because teeth and plaque emit different fluorescence, the detection unit 52 detects the tooth and plaque based on the color of the fluorescent (excited fluorescence) portion in the image. Specifically, the detection unit 52 detects a first tooth region, a second tooth region, a first plaque region, and a second plaque region, which will be described later. The first tooth region and the second tooth region are natural tooth regions.

[0071] As explained above, plaque (plaque region) fluoresces a reddish pink color (excitation fluorescence) when irradiated with blue light. It is also known that when a tooth (tooth region) is irradiated with excitation light, excitation fluorescence is emitted from the dentin, which transmits through the enamel and emits green fluorescence. A filling (e.g., a metal inlay) left behind after caries treatment does not emit excitation fluorescence under blue LED light and is captured darkly (at low brightness) by the camera. In other words, the tooth region and plaque region in the image are distinguishable, and within the tooth region, the natural tooth region and the filling are distinguishable. Hereinafter, the tooth region (natural tooth region) refers to the region of the tooth excluding the filling. Any known technology may be used to detect the tooth region and plaque region.

[0072] The selection unit 53 selects an image that is more suitable for checking the condition of the oral cavity based on the image acquired by the communication unit 51. If the proportion of the tooth region in the image is equal to or greater than a first predetermined value, the selection unit 53 selects the first image as a reference image of the specific tooth. For example, the selection unit 53 may select the image as a reference image of the specific tooth if the proportion of the pixel count of the tooth region to the total pixel count of the image is equal to or greater than a first predetermined value. The first predetermined value is set in advance and stored in a memory unit (not shown) of the mobile terminal 50. The first predetermined value is an example of a predetermined value.

[0073] The selection unit 53 may further execute a process for selecting a comparison image to be compared with the reference image.

[0074] The score calculation unit 54 calculates a plaque score indicating the degree of plaque based on the tooth region and plaque region in the image. The plaque score is a numerical value indicating the condition of the oral cavity. In this embodiment, the score calculation unit 54 calculates the plaque score based on the ratio of the plaque area (e.g., the number of pixels in the plaque) or the fluorescence amount of the plaque to the tooth area (e.g., the number of pixels in the tooth region). The score calculation unit 54 calculates the plaque score, for example, based on the following formula 1:

[0075] Plaque score = plaque area / tooth area Equation 1

[0076] The tooth area here refers to the area of ​​the natural tooth region in the image.

[0077] The method for calculating the plaque score is not limited to this, and the score may be calculated by other methods that use the area of ​​the tooth region.

[0078] The control unit 55 executes various processes on the plaque score calculated by the score calculation unit 54. The control unit 55 may, for example, compare oral care conditions based on the plaque scores of two images. The control unit 55 may, for example, compare the plaque scores of images taken before and after an oral care treatment such as tooth brushing. The control unit 55 may, for example, evaluate the effectiveness of the oral care treatment based on the plaque score of the oral care treatment. In this way, the control unit 55 may function as a comparison unit that compares oral care conditions.

[0079] The display unit 56 is a display device included in the mobile terminal 50, and displays the plaque score calculated by the score calculation unit 54, the comparison result of the oral care state by the control unit 55, etc. The display unit 56 is realized by, for example, a liquid crystal display panel, but is not limited to this. The display unit 56 is an example of a display device.

[0080] [2. Operation of Information Processing System] Next, the operation of the information processing system configured as described above will be described with reference to Figs. 4 to 8. An example of checking for missed brushing areas by comparing images before and after tooth brushing will be described below. Fig. 4 is a flowchart showing a first operation (information processing method) of the information processing system according to this embodiment. Fig. 4 describes an operation of selecting an appropriate image as an image before teeth brushing. Note that in this embodiment, the information processing system only needs to execute at least the operation shown in Fig. 4 (for example, the operation of selecting a before image) among the operations shown in Figs. 4, 6, and 8.

[0081] 4, the communication unit 51 acquires a before image of the user's oral cavity before brushing from the intraoral camera 10 (S11). The before image is an example of a first image, and in this case, is an image including a specific tooth that is the target tooth to be checked for any remaining brushing.

[0082] Next, the control unit 55 determines whether or not there are multiple before images (S12). The control unit 55 determines whether or not there are multiple before images acquired by the communication unit 51 in step S11.

[0083] Next, if the control unit 55 determines that there are multiple before images (Yes in S12), the selection unit 53 determines whether there is any image among the multiple before images in which the tooth area ratio is equal to or greater than a first predetermined value (S13).

[0084] In step S13, the detection unit 52 first detects the tooth area of ​​a specific tooth in each of the multiple before images, and calculates the tooth area ratio by dividing the area (e.g., number of pixels) of the tooth area in the before image by the area (e.g., number of pixels) of the entire before image.

[0085] 5 is a diagram for explaining step S13 shown in FIG. 4. Images A and B are schematic diagrams of binarized before images of the same tooth (specific tooth). The before image here is an image of the occlusal surface of a first molar. In the schematic diagram, the natural tooth region is shown without hatching, and areas other than the natural tooth region are shown with oblique hatching. Image B is an image in which the photographing position has been shifted to the right compared to image A, resulting in a smaller area of ​​the natural tooth region occupying the entire screen.

[0086] 5, the detection unit 52 performs, for example, binarization processing on the tooth region and other regions in the before image, and calculates the proportion of the natural tooth region in the image. In image A, the proportion of the natural tooth region (tooth area proportion) is 62%, and in image B, the proportion of the natural tooth region is 27%.

[0087] For example, when calculating the plaque score using Equation 1, the plaque score for Image B is more susceptible to the influence of plaque remaining at the boundary between the natural tooth and the prosthesis than for Image A. Because plaque tends to remain at the boundary, if Image B is an image taken after oral care treatment, the plaque score may increase even though the image has been taken after oral care treatment. In other words, if the proportion of natural tooth area is small, the plaque score may not be calculated accurately. Therefore, in the present disclosure, as shown in step S13, a process is performed to extract images whose tooth area proportion is equal to or greater than a first predetermined value.

[0088] The first predetermined value is preferably a high value from the viewpoint of accurately calculating the plaque score, and may be 40% or more, more preferably 50% or more, and even more preferably 60% or more. Furthermore, the first predetermined value may be set to a different value for each type of tooth (e.g., "incisor," "canine," "molar"), or may be set to the same value.

[0089] The detecting unit 52 calculates the tooth area ratio for each of the plurality of before images. The selecting unit 53 then determines whether or not there is an image among the plurality of before images whose tooth area ratio is equal to or greater than a first predetermined value.

[0090] 4 , next, when the selection unit 53 determines that there is an image in which the tooth area ratio is equal to or greater than the first predetermined value (Yes in S13), the selection unit 53 further selects a before image whose tooth area ratio satisfies a predetermined condition (S14). The predetermined condition may be that the tooth area ratio is close to the average value, median value, or mode of the tooth area ratios in one or more before images whose tooth area ratio is equal to or greater than the first predetermined value, or that the tooth area ratio is the maximum value. For example, the selection unit 53 may select a before image that is closest to the average value of the tooth area ratios in the one or more before images from among the one or more before images whose tooth area ratio is equal to or greater than the first predetermined value.

[0091] Next, the selection unit 53 determines the before image whose tooth area ratio satisfies a predetermined condition, i.e., the selected before image, as the reference image (S15). That is, the selection unit 53 selects the before image that satisfies the predetermined condition from one or more before images as the reference image. The selection unit 53 stores the reference image in the storage unit.

[0092] The selection unit 53 determines one before image as the reference image, but may also determine, for example, two or more before images that satisfy a predetermined condition as the reference image. In this way, the selection unit 53 functions as a first selection unit for selecting a reference image from one or more before images.

[0093] Furthermore, if the control unit 55 determines that there are not multiple before images (No in S12), the selection unit 53 determines whether the tooth area ratio of the before image is equal to or greater than a first predetermined value (S16), and if it determines that the tooth area ratio is equal to or greater than the first predetermined value (Yes in S16), the selection unit 53 proceeds to step S15. For example, if the ratio of the number of pixels in the tooth region to the total number of pixels in the before image is equal to or greater than a first predetermined value, the selection unit 53 may select the before image as the reference image. The pixel number ratio is synonymous with the area ratio.

[0094] Furthermore, if the selection unit 53 determines that there is no image in which the tooth area ratio is equal to or greater than the first predetermined value (No in S13), or if it determines that the tooth area ratio is not equal to or greater than the first predetermined value (No in S16), the control unit 55 issues a notification to re-photograph the oral cavity because there is no before image that can be used as a reference image (S17, S18). This makes it possible to prevent the calculation of a plaque score for a tooth that is mostly covered by a metal inlay, or an image in which a natural tooth is not visible or is small even if it is visible.

[0095] Next, the operation of selecting an appropriate image as an image after brushing teeth will be described with reference to Figures 6 to 7B. Figure 6 is a flowchart showing a second operation (information processing method) of the information processing system according to this embodiment.

[0096] 6 , the communication unit 51 acquires an after image of the user's oral cavity after brushing from the intraoral camera 10 (S21). The after image is an example of a second image, and in this case, is an image including a specific tooth that is the target tooth for which brushing is to be checked.

[0097] Next, the control unit 55 determines whether or not there are multiple After images (S22). The control unit 55 determines whether or not there are multiple After images acquired by the communication unit 51 in step S21.

[0098] Next, if the control unit 55 determines that there are multiple After images (Yes in S22), the selection unit 53 determines whether any of the multiple After images has a tooth area ratio equal to or greater than a first predetermined value (S23). The first predetermined value used in step S23 is, for example, the same value as the first predetermined value used in step S13, but is not limited to this. The first predetermined value used in step S23 may be changed, for example, depending on the tooth area ratio of the Before image determined as the reference image in step S15.

[0099] The determination method in step S23 is the same as the determination method in step S13, and therefore a description thereof will be omitted.

[0100] Next, if the selection unit 53 determines that there is an image with a tooth area ratio equal to or greater than a first predetermined value (Yes in S23), it further determines whether there is an image with a tooth area ratio equal to or greater than the first predetermined value that has an overlapping degree with the reference image equal to or greater than a second predetermined value (S24). From the one or more After images for which the determination in step S23 is Yes is made, the selection unit 53 extracts After images that have a similar angle of view to the reference image, i.e., that are similar to the reference image. For example, the selection unit 53 may select, as the comparison image, an After image that has a second tooth region similar to the first tooth region in the Before image. Furthermore, for example, the selection unit 53 may select, as the comparison image, an After image from one or more second images that shows the same tooth as the tooth in the reference image but captured from a position closer (e.g., the closest) to the position where the reference image was captured. The overlapping degree is an example of the overlapping degree, and the second predetermined value is an example of the predetermined degree.

[0101] In addition, the determination of whether or not the tooth is the same as the tooth included in the reference image can be made by any known method, and can be made, for example, by the posture (position and orientation) of the intraoral camera 10 when the reference image and the comparison image were captured, the shape of the tooth (for example, pattern matching based on the shape of the tooth included in the reference image), etc.

[0102] The selection unit 53 extracts at least three feature points on the contours of the natural teeth contained in the reference image and the after image, and performs image correction to adjust the relative positions of the feature points so that their coordinates match, using the feature points in the reference image and the after image. Image correction will be described with reference to Figures 7A and 7B.

[0103] 7A and 7B are diagrams for explaining the image correction performed in step S24 shown in Fig. 6. Fig. 7A is a diagram for explaining the process of adjusting the relative position when the after image is misaligned with respect to the reference image in the X-axis direction and the Y-axis direction, and Fig. 7B is a diagram for explaining the process of adjusting the relative position when the after image is rotated with respect to the reference image. Note that (c) of Fig. 7A and (c) of Fig. 7B show enlarged images for convenience.

[0104] 7A shows the reference image, and FIG. 7A shows the after image for which the determination in step S23 is Yes, which has deviations in the X-axis and Y-axis directions. The deviations in the X-axis and Y-axis directions can occur, for example, when the position where the reference image was captured and the position where the after image was captured are misaligned in the X-axis and Y-axis directions.

[0105] 7A(c), the selection unit 53 uses the feature points to move at least one of the reference image and the after image parallel to the X-axis and Y-axis directions so that the contours of the tooth region (natural tooth region) in the reference image and the tooth region (natural tooth region) in the after image overlap. Parallel movement is an example of image correction.

[0106] 7B (a) shows the reference image, and FIG. 7B (b) shows the after image for which the determination in step S23 is Yes and in which rotation has occurred. Rotation can occur, for example, when the orientation of the intraoral camera 10 when capturing the after image is rotated around the optical axis of the intraoral camera 10 relative to the orientation of the intraoral camera 10 when capturing the reference image.

[0107] In this case, as shown in (c) of Fig. 7B, the selection unit 53 rotates at least one of the reference image and the after image using the feature points so that the contours of the tooth region (natural tooth region) in the reference image and the tooth region (natural tooth region) in the after image overlap. Rotation is an example of image correction.

[0108] Then, the selection unit 53 calculates the area of ​​the overlapping portion of the tooth regions in a state where the contours of the tooth regions overlap as shown in (c) of FIG. 7A or (c) of FIG. 7B.

[0109] 7A and 7B , the selection unit 53 aligns the reference image and the after image relative to each other based on feature points (e.g., feature points on the contour line) in the outline of the tooth region in the reference image and feature points in the outline of the tooth region in the after image, and identifies overlapping portions between the aligned reference image and the after image. In (c) of Fig. 7A and (c) of Fig. 7B , the overlapping portions between the tooth region in the reference image and the tooth region in the after image are indicated by dot hatching.

[0110] Note that image correction is not limited to the above, and may include, for example, enlarging or reducing the image.

[0111] The degree of overlap may be a value obtained by dividing the area (or number of pixels) of the overlapping portion by the area (or number of pixels) of the tooth region in the reference image, or may be the area of ​​the overlapping portion itself. The area of ​​the overlapping portion may also be the number of pixels in the overlapping portion.

[0112] Note that the tooth region in the images shown in (a) of Figure 7A and (a) of Figure 7B is an example of the first tooth region, and the tooth region in the images shown in (b) of Figure 7A and (b) of Figure 7B is an example of the second tooth region.

[0113] 6 again, the selection unit 53 next determines, as the comparison image, an After image whose overlapping degree is equal to or greater than the second predetermined value (S25). That is, the selection unit 53 selects, from one or more After images, an After image whose overlapping degree is equal to or greater than the second predetermined value as the comparison image.

[0114] Such a reference image and a comparison image are similar images with many commonalities in the tooth regions shown, and can be images that facilitate comparison of the tooth conditions. It can also be said that the selection unit 53 selects, as a comparison image, an After image that has a high similarity to the reference image of a specific tooth from, for example, one or more After images. The similarity is, for example, a value corresponding to the degree of overlap of the tooth regions. It is assumed that a high degree of overlap also leads to a high degree of similarity. The overlap degree is an example of a value based on the area of ​​the overlapping portion between the tooth region of the reference image and the tooth region of the After image. The value based on the area is not limited to the degree of overlap, and may be, for example, the area of ​​the tooth region.

[0115] The selector 53 stores the comparison image in the storage unit in association with the reference image. Note that the selector 53 determines one After image as the comparison image, but may also determine, for example, two or more After images whose degree of overlap is equal to or greater than a second predetermined value as the comparison images. In this way, the selector 53 functions as a second selector for selecting a comparison image from one or more After images.

[0116] Furthermore, if the control unit 55 determines that there are not multiple after images (No in S22), the selection unit 53 determines whether the tooth area ratio of the after image is equal to or greater than a first predetermined value (S26), and if it determines that the tooth area ratio is equal to or greater than the first predetermined value (Yes in S26), proceeds to step S24.

[0117] Furthermore, if the selection unit 53 determines that the degree of overlap is less than the second predetermined value (No in S24), if it determines that there are no images in which the tooth area ratio is equal to or greater than the first predetermined value (No in S23), or if it determines that the tooth area ratio is not equal to or greater than the first predetermined value (No in S26), the control unit 55 notifies the user to re-photograph the oral cavity because there is no after image that can be used as a comparison image (S27, S28).

[0118] In step S24, the selector 53 may determine whether the difference between the proportion of the tooth region in the After image and the proportion of the tooth region in the reference image is within a third predetermined value, and select the After image for which it is determined that the difference is within the third predetermined value as the comparison image. The third predetermined value is set in advance and stored in the storage unit.

[0119] Next, the operation of evaluating the state of the oral cavity using the reference image and the comparative image determined above will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the operation of evaluating the oral care state (information processing method) according to this embodiment.

[0120] As shown in Fig. 8, the detection unit 52 detects plaque regions in the reference image and the comparison image (S41). The detection unit 52 detects, as plaque regions, portions in the reference image and the comparison image where porphyrin, a substance produced by bacteria in plaque, fluoresces reddish pink (excited fluorescence). The plaque region of a specific tooth based on the reference image is an example of a first plaque region, and the plaque region of a specific tooth based on the comparison image is an example of a second plaque region.

[0121] Next, the score calculation unit 54 calculates plaque scores for the reference image and the comparison image (S42). The score calculation unit 54 calculates a plaque score (first plaque score) indicating the condition of the teeth based on the first tooth region and the first plaque region for the reference image, and calculates a plaque score (second plaque score) based on the second tooth region and the second plaque region for the comparison image. For example, the score calculation unit 54 calculates the first plaque score based on the ratio of the number of pixels of the first plaque region to the number of pixels of the natural tooth region in the reference image, and calculates the second plaque score based on the ratio of the number of pixels of the second plaque region to the number of pixels of the natural tooth region in the comparison image. The first plaque score and the second plaque score are plaque scores for the same tooth.

[0122] Next, the control unit 55 evaluates the user's oral care state based on the plaque scores of the reference image and the comparison image (S43). If the reference image and the comparison image are images before and after oral care treatment (e.g., before and after brushing), the control unit 55 may evaluate the effectiveness of the oral care treatment based on the first plaque score and the second plaque score. For example, the control unit 55 may determine which teeth have the most areas left unbrushed by calculating the plaque score for each tooth.

[0123] It can also be said that the control unit 55 compares the oral care state before and after the oral care treatment based on the first plaque score and the second plaque score. In this way, the control unit 55 may function as a comparison unit that compares the oral care state.

[0124] Next, the control unit 55 displays the evaluation result of the oral care state on the display unit 56 (S44). The control unit 55 may display the first plaque score and the second plaque score on the display unit 56, or may display a numerical value or a level indicating the effect of the oral care treatment based on the first plaque score and the second plaque score on the display unit 56.

[0125] (Variation 1 of the Embodiment) An information processing system according to this variation will be described below with reference to Figures 9 and 10. Note that the following description will focus on differences from the embodiment, and descriptions of content that is the same as or similar to the embodiment will be omitted or simplified. The configuration of the information processing system according to this variation may be the same as the information processing system according to the embodiment, and description thereof will be omitted. Furthermore, the following description will use the reference numerals of the information processing system according to the embodiment.

[0126] 9 is a diagram for explaining the imaging guide in the information processing system according to this modification. This modification differs from the information processing system according to the embodiment in that an imaging guide image is displayed on the display unit 56 when capturing an After image.

[0127] 9A shows a reference image, and FIG. 9B shows a photography guide image for photography guide based on the reference image. The photography guide image is an image that includes, as a photography guide, a silhouette, outline, or semi-transparent display of a specific tooth included in the reference image. In FIG. 9B, the tooth included in the reference image and its surrounding outline are shown by dashed lines.

[0128] 9C shows an image in which a shooting guide image is superimposed on a captured image. In FIG. 9C, the captured image displayed by the light received by the shooting unit 21 is shown by a solid line, and the shooting guide image superimposed on the captured image is shown by a dashed line. The captured image here is an image that has not yet been captured, but will be displayed on the display unit 56 when a shooting operation is performed. Note that the captured image, i.e., the After image, does not include a shooting guide.

[0129] The user captures an image of the oral cavity after the oral care treatment (after image) using the intraoral camera 10. At that time, the user adjusts the photographing position and the like while checking the photographed image displayed on the display unit 56 by the light received by the photographing unit 21. In this modification, a photographing guide image is superimposed on the photographed image for adjusting the photographing position and the like.

[0130] This allows the user to refer to the imaging guide and, for example, check the positions of the imaging guide in the imaging guide image and the teeth in the captured image while adjusting the posture (position and direction) of the intraoral camera 10. For example, imaging is performed when the imaging guide in the imaging guide image and the teeth in the captured image match or are closer than a predetermined distance, so that an After image captured at a position close to that of the reference image can be obtained.

[0131] The display of the photography guide image on the display unit 56 is executed by, for example, the control unit 55. For example, when the user takes an After image using the intraoral camera 10, the control unit 55 superimposes the photography guide on the image (taken image) acquired from the intraoral camera 10 and displayed on the display unit 56.

[0132] Next, the operation of the information processing system according to this modified example will be described with reference to Fig. 10. Fig. 10 is a flowchart showing the operation of the information processing system according to this modified example (information processing system).

[0133] As shown in FIG. 10, the information processing system executes step S31 in addition to the flowchart shown in FIG.

[0134] The control unit 55 superimposes an imaging guide on the display of the display unit 56 before the intraoral camera 10 captures an After image (S31). For example, when the control unit 55 receives an operation from the user indicating that an After image is to be captured, the control unit 55 may display the imaging guide on the display unit 56. The display of the imaging guide can be set to on or off, and the control unit 55 may display the imaging guide on the display unit 56 only when the setting for displaying the imaging guide is on, for example. The control unit 55 may also display the imaging guide on the display unit 56 when the notification of step S27 or S28 is given and re-imaging is to be performed.

[0135] (Variation 2 of the Embodiment) The information processing system according to this variation will be described below with reference to FIG. 11 . Note that the following description will focus on differences from the embodiment, and descriptions of content that is the same as or similar to the embodiment will be omitted or simplified. The configuration of the information processing system according to this variation may be the same as the information processing system according to the embodiment, and description thereof will be omitted. Furthermore, the following description will use the reference numerals of the information processing system according to the embodiment.

[0136] Fig. 11 is a diagram showing a screen for selecting a tooth to be photographed according to this modification. Fig. 11 illustrates an example in which RGB images (e.g., a reference image and an after image) photographed by a user are stored in association with information indicating which tooth the image represents.

[0137] 11 , the display unit 56 of the mobile terminal 50 displays a screen for the user to select an RGB image to be captured. The display unit 56 displays, for example, the rows of teeth of the upper and lower jaws. The display of the rows of teeth may be an image or an illustration.

[0138] The user selects the next tooth to be photographed. Figure 11 shows an example in which the first molar on the left side of the lower jaw is selected.

[0139] When the control unit 55 acquires an RGB image from the intraoral camera 10 after accepting the user's selection, the control unit 55 associates the acquired RGB image with information indicating the first molar on the left side of the mandible and stores the associated image. For example, the control unit 55 associates the reference image with information indicating the first molar on the left side of the mandible and stores the associated image.

[0140] The tooth selection may be performed before or after capturing the RGB image, as long as the RGB image and the type or position of the tooth are stored in association with each other.

[0141] In addition, imaging direction information indicating whether the RGB image was taken from the cheek side, the tongue side, or the occlusal side of the tooth may be further stored in association with the RGB image. The imaging direction information may be input by, for example, the user.

[0142] Furthermore, when an After image is captured, if a reference image is selected by the user, the display unit 56 may display information indicating which tooth the reference image is an image of. For example, in a display of the row of teeth in the oral cavity as shown in Fig. 11 , the display unit 56 may indicate which tooth the reference image is an image of, or may display text information.

[0143] Then, the degree of overlap between the reference image and the after image thus captured may be determined.

[0144] The control unit 55 may store the tooth type and the amount of plaque in association with each other in the storage unit.

[0145] Furthermore, when the control unit 55 reads out the reference image and the after image from the storage unit and displays them on the display unit 56, the control unit 55 may superimpose the tooth type or tooth type on the image.

[0146] Other Embodiments Although the information processing system according to the embodiments of the present disclosure has been described above, the present disclosure is not limited to the embodiments.

[0147] For example, in the above-described embodiment, an example has been described in which the intraoral camera 10 is used primarily for photographing teeth, but the intraoral camera 10 may be an oral care device equipped with a camera. For example, the intraoral camera 10 may be an oral irrigator equipped with a camera.

[0148] Furthermore, in the above-described embodiment, the mobile terminal 50 is exemplified as an information processing device, but the information processing device may be a stationary information terminal or a server device.

[0149] Furthermore, in the above-described embodiment, the overlapping degree is given as an example of the overlapping degree, but the overlapping degree may be indicated in stages such as "high", "medium", and "low".

[0150] Furthermore, in the above embodiments, an example has been described in which the information processing device (in the embodiments, the mobile terminal 50) is equipped with a display unit, but this is not limited to this, and a display device capable of communicating with the information processing device may be provided as a device separate from the information processing device.

[0151] Furthermore, each processing unit included in the information processing system according to the above-described embodiments is typically realized as an LSI, which is an integrated circuit. These may be individually implemented as single chips, or some or all of them may be integrated into a single chip.

[0152] Furthermore, the integrated circuit is not limited to an LSI, but may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells within the LSI may also be used.

[0153] In addition, in each of the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0154] The division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to another functional block.Furthermore, the functions of multiple functional blocks having similar functions may be processed in parallel or in time-sharing by a single piece of hardware or software.

[0155] Furthermore, the information processing device (e.g., the mobile terminal 50) according to the above-described embodiments may be realized as a single device or may be realized by multiple devices. When the information processing device is realized by multiple devices, the components of the information processing device may be distributed among the multiple devices in any manner. For example, at least some of the functions of the information processing device may be realized by the intraoral camera 10 (e.g., the signal processing unit 30). When the information processing device is realized by multiple devices, the communication method between the multiple devices is not particularly limited and may be wireless communication or wired communication. Furthermore, wireless communication and wired communication may be combined between the devices.

[0156] The present disclosure may also be realized as an information processing method executed by an information processing system, or as an intraoral camera, a mobile terminal, or a cloud server included in the information processing system.

[0157] The order in which the steps are executed in the sequence diagram is merely an example for specifically explaining the present disclosure, and other orders may be used. Also, some of the steps may be executed simultaneously (in parallel) with other steps.

[0158] Furthermore, one aspect of the present disclosure may be a computer program that causes a computer to execute each of the characteristic steps included in the information processing method shown in any one of FIG. 4, FIG. 6, and FIG.

[0159] Furthermore, for example, the program may be a program to be executed by a computer. Another aspect of the present disclosure may be a computer-readable non-transitory recording medium on which such a program is recorded. For example, such a program may be recorded on a recording medium and distributed or circulated. For example, the distributed program may be installed in a device having another processor, and the program may be executed by the processor, thereby causing the device to perform each of the above processes.

[0160] The information processing system etc. according to one or more aspects has been described above based on the embodiments etc., but the present disclosure is not limited to these embodiments etc. As long as it does not deviate from the spirit of the present disclosure, various modifications that a person skilled in the art can conceive of to this embodiment and forms constructed by combining components of different embodiments may also be included within the scope of one or more aspects.

[0161] The present disclosure can be applied to an information processing system for checking the condition of the oral cavity.

[0162] REFERENCE SIGNS LIST 10 Intraoral camera 10a Head unit 10b Handle unit 20 Hardware unit 21 Photography unit (photography device) 22 Sensor unit 23 Illumination unit 23A First LED 23B Second LED 23C Third LED 23D Fourth LED 24 Operation unit 30 Signal processing unit 31 Camera control unit 32 Image processing unit 33 Control unit 34 Illumination control unit 35 Memory unit 40 Communication unit 50 Portable terminal (information processing device) 51 Communication unit (acquisition unit) 52 Detection unit 53 Selection unit (first selection unit, second selection unit) 54 Score calculation unit 55 Control unit (comparison unit) 56 Display unit (display device)

Claims

1. An acquisition unit that acquires a first image including a specific tooth obtained by photographing the inside of the oral cavity, A detection unit for detecting the first tooth region of a specific tooth from the first image, The system includes a selection unit that selects the first image as a reference image for a specific tooth if the proportion of the first tooth region in the first image is equal to or greater than a predetermined value, The acquisition unit further acquires a second image including the specific tooth obtained by photographing the inside of the oral cavity, The system further includes a control unit that determines whether or not there are multiple second images. If the detection unit determines that there are multiple second images, it detects the second tooth region of the specific tooth for each of the multiple second images. The selection unit selects from among the plurality of second images a second image having a second tooth region similar to the first tooth region of the first image, as a comparison image to be compared with the reference image. Information processing device.

2. An acquisition unit that acquires a first image including a specific tooth obtained by photographing the inside of the oral cavity, A detection unit for detecting the first tooth region of a specific tooth from the first image, The system includes a selection unit that selects the first image as a reference image for a specific tooth if the proportion of the first tooth region in the first image is equal to or greater than a predetermined value, The acquisition unit further acquires a second image including the specific tooth obtained by photographing the inside of the oral cavity, The detection unit detects the second tooth region of the specific tooth from the second image, The selection unit selects a second image having a second tooth region similar to the first tooth region of the first image as a comparison image to be compared with the reference image. Information processing device.

3. The acquisition unit acquires one or more of the second images, The selection unit selects, from among one or more of the second images, the second image in which the degree of overlap between the second tooth region in the second image and the first tooth region in the reference image is greater than or equal to a predetermined degree, as the comparison image. The information processing apparatus according to claim 1 or 2.

4. The control unit further causes the imaging device for photographing the inside of the oral cavity to superimpose the imaging guide based on the reference image onto the image acquired from the imaging device and displayed on the display device when the imaging device captures the second image. The information processing apparatus according to claim 1.

5. The aforementioned imaging guide includes the silhouette, contour, or translucent representation of the specific tooth included in the reference image. The information processing apparatus according to claim 4.

6. The detection unit further detects a first plaque area of ​​the specific tooth based on the reference image and a second plaque area of ​​the specific tooth based on the comparison image. moreover, A score calculation unit calculates a first plaque score based on the first tooth region and the first plaque region in the reference image, and calculates a second plaque score based on the second tooth region and the second plaque region in the comparison image. The system includes a comparison unit that compares the oral care status based on the first plaque score and the second plaque score. The information processing apparatus according to claim 1 or 2.

7. The score calculation unit, Based on the ratio of the number of pixels in the first plaque region to the number of pixels in the natural tooth region in the reference image, the first plaque score is calculated. The second plaque score is calculated based on the ratio of the number of pixels in the second plaque region to the number of pixels in the natural tooth region in the comparison image. The information processing apparatus according to claim 6.

8. The selection unit selects the first image as the reference image if the ratio of the number of pixels in the first tooth region to the total number of pixels in the first image is equal to or greater than the predetermined value. The information processing apparatus according to claim 1 or 2.

9. The first image above is an image of the inside of the mouth taken by the user before performing oral care. The second image is an image of the inside of the mouth after the user has performed oral care treatment. The information processing apparatus according to claim 1 or 2.

10. By taking a photograph of the inside of the mouth, a first image including a specific tooth is obtained, The first tooth region of the specific tooth is detected from the first image, If the proportion of the first tooth region in the first image is greater than or equal to a predetermined value, the image is selected as the reference image for the specific tooth. Furthermore, a second image including the specific tooth obtained by photographing the oral cavity is acquired. Determine whether there are multiple second images. If it is determined that there are multiple second images, the second tooth region of the specific tooth is detected for each of the multiple second images. From among the multiple second images, the second image having a second tooth region similar to the first tooth region of the first image is selected as the comparison image to be compared with the reference image. Information processing methods.

11. A first image including a specific tooth obtained by photographing the inside of the oral cavity is acquired, The first tooth region of the specific tooth is detected from the first image, If the proportion of the first tooth region in the first image is greater than or equal to a predetermined value, the first image is selected as the reference image for the specific tooth. Furthermore, a second image including the specific tooth obtained by photographing the oral cavity is acquired. The second tooth region of the specific tooth is detected from the second image, A second image having a second tooth region similar to the first tooth region of the first image is selected as a comparison image to be compared with the reference image. Information processing methods.

12. A program for causing a computer to execute the information processing method described in claim 10 or 11.