Periodontal disease detection system, periodontal disease detection method, and program

Abstract

This periodontal disease detection system comprises: an acquisition unit (51) that acquires a first image obtained by imaging specific teeth and a periodontal region including gingiva in the oral cavity of a user; an image processing unit (53) that generates a second image including a periodontal region of the specific teeth from the first image; and a region detection unit (151) that detects which region is represented by the second image among a plurality of regions in the oral cavity defined by dividing a dental arch; a rule selection unit (152) that selects a periodontal disease detection tool corresponding to the detected region from among a plurality of periodontal disease detection tools respectively corresponding to the plurality of regions and each generated so as to output, in response to an input of image data including the region, information relating to a periodontal disease in the region; and a periodontal disease detection unit (55) that detects a periodontal disease of the user on the basis of the selected periodontal disease detection tool and image data based on the second image.

Description

Periodontal disease detection system, periodontal disease detection method and program

[0001] The present disclosure relates to a periodontal disease detection system, a periodontal disease detection method, and a program.

[0002] In recent years, with the spread of mobile devices with cameras, a system has been proposed in which a user photographs the vicinity of their oral cavity with the camera of the mobile device before undergoing a dental examination, and the state of the oral cavity is easily and initially determined from the obtained image data. For example, Patent Literature 1 discloses a system in which, when an oral cavity region is photographed with the camera of the mobile device, image data of an oral cavity target region to be estimated is extracted from the image data of the oral cavity region, and the state of the oral cavity target region is estimated based on the image data of the extracted oral cavity target region.

[0003] Japanese Patent Application Laid-Open No. 2019-155027

[0004] When detecting periodontal diseases such as periodontal disease using images, it is desirable to improve the accuracy of detecting periodontal diseases.

[0005] Therefore, the present disclosure provides a periodontal disease detection system, a periodontal disease detection method, and a program that can improve the accuracy of periodontal disease detection using images.

[0006] A periodontal disease detection system according to one aspect of the present disclosure is a periodontal disease detection system that detects periodontal disease in a user based on an image of a specified area in the oral cavity, and includes: an acquisition unit that acquires a first image of the periodontal area of ​​a specific tooth in the user's oral cavity, including the specific tooth and gums; an image processing unit that generates a second image from the first image, including the periodontal area of ​​the specific tooth; a region detection unit that detects which of a plurality of areas in the oral cavity defined by dividing the tooth row the second image is an image of; a tool selection unit that selects a periodontal disease detection tool corresponding to the area detected by the region detection unit from a plurality of periodontal disease detection tools corresponding to each of the plurality of areas, each of which is generated to input image data including the area and output information regarding periodontal disease in that area; and a periodontal disease detection unit that detects the periodontal disease of the user based on the selected periodontal disease detection tool and image data based on the second image.

[0007] A periodontal disease detection method according to one aspect of the present disclosure is a periodontal disease detection method executed by a periodontal disease detection system that detects periodontal disease in a user based on an image of a specified area in the oral cavity, the method comprising: obtaining a first image of the periodontal area of ​​a specific tooth in the user's oral cavity, including the specific tooth and gums; generating a second image from the first image that includes the periodontal area of ​​the specific tooth; detecting which of a plurality of areas in the oral cavity defined by dividing the tooth row the second image is an image of; selecting a periodontal disease detection tool corresponding to the detected area from a plurality of periodontal disease detection tools that correspond to each of the plurality of areas, each of which receives image data including the area as input and outputs information about periodontal disease in that area; and detecting the periodontal disease of the user based on the selected periodontal disease detection tool and image data based on the second image.

[0008] A periodontal disease detection method according to one aspect of the present disclosure is a periodontal disease detection method executed by one or more processors, wherein the one or more processors output a first image of a specific tooth in a user's oral cavity, including the periodontal region of the specific tooth, and the specific tooth being included in the first image, obtain a detection result of the user's periodontal disease based on the first image, and perform a predetermined processing on the obtained detection result, wherein the obtained detection result includes a periodontal disease detection tool selected from a plurality of periodontal disease detection tools corresponding to each of a plurality of regions in the oral cavity, each of which is generated to input image data including the region and output information regarding periodontal disease in the region, depending on which of the plurality of regions defined by dividing the tooth row a second image including the periodontal region of the specific tooth generated from the first image is an image of, and the detection result of the user's periodontal disease detected based on image data based on the second image.

[0009] A program according to one aspect of the present disclosure is a program for causing a computer to execute the periodontal disease detection method described above.

[0010] According to one aspect of the present disclosure, it is possible to realize a periodontal disease detection system or the like that can improve the accuracy of detecting periodontal disease using images.

[0011] FIG. 1 is a perspective view of an intraoral camera in a periodontal disease detection system according to Embodiment 1. FIG. 2 is a schematic configuration diagram of the periodontal disease detection system according to Embodiment 1. FIG. 3 is a block diagram showing the functional configuration of a mobile terminal according to Embodiment 1. FIG. 4 is a flowchart showing the operation of the periodontal disease detection system according to Embodiment 1. FIG. 5 is a first diagram illustrating a periodontal region in an oral cavity according to Embodiment 1. FIG. 6 is a second diagram illustrating a periodontal region in an oral cavity according to Embodiment 1. FIG. 7A is a diagram illustrating image data extracted by an image data extraction unit according to Embodiment 1. FIG. 7B is a diagram showing a specific example of a method for forming a rectangular region according to Embodiment 1. FIG. 8A is a diagram illustrating an example of a first group rule base according to Modification 1 of Embodiment 1. FIG. 8B is a diagram illustrating an example of a second group rule base according to Modification 1 of Embodiment 1. FIG. 8C is a diagram illustrating an example of a third group rule base according to Modification 1 of Embodiment 1. FIG. 9 is a diagram illustrating training data according to Modification 2 of Embodiment 1. FIG. 10 is a block diagram showing the functional configuration of a mobile terminal according to Embodiment 2. FIG. 11 is a diagram illustrating multiple regions in an oral cavity according to Embodiment 2. FIG. 12 is a diagram showing an example of an image obtained by photographing a plurality of regions according to Embodiment 2. FIG. 13 is a diagram showing an example of an image used for training a learning model according to Embodiment 2. FIG. 14 is a flowchart showing the operation of a periodontal disease detection system according to Embodiment 2. FIG. 15 is a flowchart showing the operation of a periodontal disease detection system according to a modified embodiment of Embodiment 2. FIG. 16 is a diagram showing the angle of the tip of the interdental papilla gingiva according to a modified embodiment of Embodiment 2. FIG. 17A is a first diagram explaining a method for calculating the angle of the tip of the interdental papilla gingiva according to a modified embodiment of Embodiment 2. FIG. 17B is a second diagram explaining a method for calculating the angle of the tip of the interdental papilla gingiva according to a modified embodiment of Embodiment 2. FIG. 17C is a third diagram explaining a method for calculating the angle of the tip of the interdental papilla gingiva according to a modified embodiment of Embodiment 2. FIG. 18A is a first diagram showing the radius of the tip of the interdental papilla gingiva according to a modified embodiment of Embodiment 2. FIG. 18B is a second diagram showing the radius of the tip of the interdental papilla gingiva according to a modified embodiment of Embodiment 2.

[0012] (Background to the present disclosure) Detection of periodontal disease based on image data captured inside the oral cavity is performed based on the color and shape of the gums in the detection target area. However, the shape of the teeth and gums varies depending on at least one of the tooth type (molars, canines, anterior teeth) and the imaging direction (buccal, lingual), and the degree of gum deformation associated with the progression of periodontal disease also varies. Therefore, a learning model trained by inputting image data of all tooth types and imaging directions may erroneously detect the type and progression of periodontal disease based on the tooth-specific gum shape. Note that erroneous detection includes erroneously detecting at least one of the type and progression of periodontal disease, detecting periodontal disease when it is not present (i.e., not detecting it), detecting periodontal disease when it is not present, etc.

[0013] Therefore, the inventors of the present application conducted extensive research into periodontal disease detection systems that can improve the accuracy of detecting periodontal disease using images, particularly periodontal disease detection systems that can reduce false positives in detecting periodontal disease based on image data taken inside the oral cavity, and have devised the periodontal disease detection system shown below.

[0014] A periodontal disease detection system according to a first aspect of the present disclosure is a periodontal disease detection system that detects periodontal disease in a user based on an image of a specified region in the oral cavity, and includes: an acquisition unit that acquires a first image of the periodontal region of a specific tooth in the user's oral cavity, including the specific tooth and gums; an image processing unit that generates a second image from the first image, including the periodontal region of the specific tooth; a region detection unit that detects which of a plurality of regions in the oral cavity defined by dividing the tooth row the second image is an image of; a tool selection unit that selects a periodontal disease detection tool corresponding to the region detected by the region detection unit from a plurality of periodontal disease detection tools corresponding to each of the plurality of regions, each of which is generated to input image data including the region and output information regarding periodontal disease in that region; and a periodontal disease detection unit that detects the periodontal disease of the user based on the selected periodontal disease detection tool and image data based on the second image.

[0015] As a result, a periodontal disease detection tool corresponding to the detected region is selected as the periodontal disease detection tool to be used for detecting periodontal disease in the second image. Since this periodontal disease detection tool is generated using image data of the region, it is a periodontal disease detection tool that reflects the shape of the teeth and gums in the region, the degree of gum deformation associated with the progression of periodontal disease, etc. Use of such a periodontal disease detection tool can reduce false positives in detecting periodontal disease based on image data captured within the oral cavity, compared to when a single periodontal disease detection tool generated using the shape of the teeth in each region is used. Therefore, the accuracy of periodontal disease detection using images can be improved.

[0016] Also, for example, a periodontal disease detection system according to a second aspect may be a periodontal disease detection system according to the first aspect, wherein the plurality of periodontal disease detection tools include a plurality of learning models each trained to input image data including the region and output information regarding periodontal disease in the region, and the periodontal disease detection unit detects the periodontal disease of the user by inputting image data based on the second image into a learning model selected from the plurality of learning models.

[0017] As a result, the selected learning model is a learning model trained using image data of the region, and is therefore a learning model that has effectively learned the shape of the teeth and gums in the region, the degree of gum deformation associated with the progression of periodontal disease, etc. By using such a learning model, it is possible to reduce false positives in detecting periodontal disease based on image data of intraoral images, compared to when a single learning model that has learned the shape of the teeth in each region is used.

[0018] Also, for example, a periodontal disease detection system according to a third aspect may be the periodontal disease detection system according to the first aspect, wherein the plurality of periodontal disease detection tools each include a plurality of judgment rules for inputting an image of the gums between adjacent teeth for a tooth in image data including the region and outputting information regarding periodontal disease in the region, and the periodontal disease detection unit may detect the periodontal disease of the user by inputting image data based on the second image into a judgment rule selected from the plurality of judgment rules.

[0019] This makes it possible to reduce false positives in detecting periodontal disease based on image data captured inside the oral cavity, compared to when a single determination rule is used.

[0020] Also, for example, a periodontal disease detection system according to a fourth aspect may be a periodontal disease detection system according to the first aspect, wherein the plurality of periodontal disease detection tools include a plurality of learning models, each trained to input image data including the region and output information regarding periodontal disease in the region, and a plurality of judgment rules, each trained to input a gingival image between adjacent teeth for a tooth in the image data including the region and output information regarding periodontal disease in the region, the rule selection unit selects one or more learning models from the plurality of learning models and selects one or more judgment rules from the plurality of judgment rules, and the periodontal disease detection unit detects the periodontal disease of the user based on a first detection result of the one or more learning models and a second detection result of the one or more judgment rules.

[0021] This makes it possible to reduce false positives in detecting periodontal disease based on image data captured inside the oral cavity, compared to when using only either a learning model or a judgment rule.

[0022] Also, for example, a periodontal disease detection system according to a fifth aspect may be a periodontal disease detection system according to the fourth aspect, in which the periodontal disease detection unit compares the first detection result with the second detection result, and detects the periodontal disease of the user using the detection result that determines that the symptoms of periodontal disease are worse.

[0023] This allows the user to be notified of the detection result that indicates that the symptoms of periodontal disease are worse, and can effectively encourage the user to visit a doctor, for example.

[0024] Also, for example, a periodontal disease detection system according to a sixth aspect may be a periodontal disease detection system according to any one of the first to fifth aspects, wherein each of the plurality of regions may be a region including a group of teeth having similar shapes.

[0025] This allows detecting periodontal disease based on a periodontal disease detection tool generated using an image of a periodontal region with a similar tooth shape, thereby further reducing false detections.

[0026] Also, for example, the periodontal disease detection system according to the seventh aspect may be the periodontal disease detection system according to the sixth aspect, wherein the multiple regions include a region on the buccal side of the front teeth, a region on the lingual side of the front teeth, a region on the buccal side of the back teeth, and a region on the lingual side of the back teeth.

[0027] As a result, periodontal disease detection tools corresponding to four areas, namely the buccal area of ​​the front teeth, the lingual area of ​​the front teeth, the buccal area of ​​the molars, and the lingual area of ​​the molars, are generated, and by using the periodontal disease detection tool corresponding to the area in question out of the four areas, it is possible to further reduce false detections.

[0028] Furthermore, for example, the periodontal disease detection system according to the eighth aspect may be a periodontal disease detection system according to any one of the first to seventh aspects, and may further include an image data extraction unit that extracts the periodontal region from the tip of the interdental papilla gingiva of the specific tooth to the free gingival sulcus from the second image as the image data based on the second image.

[0029] This allows for more accurate detection of periodontal disease, as the periodontal region image data includes the periodontal region from the tip of the interdental papilla gingiva of the tooth to the free gingival sulcus, which is prone to changes when periodontal disease develops.

[0030] Also, for example, a periodontal disease detection system according to a ninth aspect may be the periodontal disease detection system according to the eighth aspect, wherein the image data extraction unit extracts from the second image a rectangular frame area that circumscribes the left and right sides of the contour of the specific tooth and includes from the tip of the specific tooth to the free gingival sulcus as the image data based on the second image.

[0031] This makes it possible to easily and reliably extract periodontal region image data that enables more accurate detection of periodontal disease.

[0032] Furthermore, for example, the periodontal disease detection system according to the tenth aspect may be a periodontal disease detection system according to any one of the third to fifth aspects, and may further include an image data extraction unit that extracts from the second image an area including at least a portion of the specific tooth and at least one of the interdental papilla gingiva on the left and right sides of the specific tooth as the image data based on the second image.

[0033] This allows the user to be notified of the detection result that the symptoms of periodontal disease are severe, and effectively encourages the user to, for example, undergo treatment.

[0034] Furthermore, for example, a periodontal disease detection system according to an eleventh aspect is the periodontal disease detection system according to any one of the first to tenth aspects, and each of the plurality of periodontal disease detection tools may output an estimated result of at least one of periodontal pocket depth, BOP (Bleeding On Probing), and GI (Gingival Index) value as the information related to the periodontal disease.

[0035] This makes it possible to obtain information related to periodontal disease, such as an estimated result of at least one of periodontal pocket depth, BOP (Bleeding On Probing), and GI (Gingival Index) value. Such information is useful for understanding periodontal disease. Periodontal pocket depth is one index for determining periodontal disease; for example, 3 mm or less is considered normal, 4 mm and 5 mm are considered moderate, and 6 mm or more is considered severe periodontal disease (note that these numbers are just examples), and is useful information for determining periodontal disease. In addition, the BOP value is an index for checking the presence or absence of bleeding, and is useful information for determining, for example, "swelling," "illness," etc.

[0036] Furthermore, for example, the periodontal disease detection system of the 12th aspect may be a periodontal disease detection system of any of the first to 11th aspects, and may further include an output unit that outputs information indicating the periodontal disease detected by the periodontal disease detection unit to the user's information terminal.

[0037] This allows the user to be notified of the state of periodontal disease.

[0038] Furthermore, for example, a periodontal disease detection system according to a thirteenth aspect is a periodontal disease detection system according to any one of the first to twelfth aspects, and further includes an output unit that outputs information indicating the periodontal disease detected by the periodontal disease detection unit to a first information terminal of a dentist other than the user, and the output unit may further output to a second information terminal of the user information regarding whether or not the user needs to receive a medical examination, which information was acquired via the acquisition unit and input to the first information terminal.

[0039] This allows dentists to understand the periodontal disease status of users who are remotely located (i.e., users who are not actually being examined), thereby assisting in the treatment of periodontal disease.

[0040] In addition, a periodontal disease detection method according to a fourteenth aspect of the present disclosure is a periodontal disease detection method executed by a periodontal disease detection system that detects periodontal disease in a user based on an image of a specified area in the oral cavity, the method comprising: obtaining a first image of the periodontal area of ​​a specific tooth in the user's oral cavity, including the specific tooth and gums; generating a second image from the first image, including the periodontal area of ​​the specific tooth; detecting which of a plurality of areas in the oral cavity defined by dividing the tooth row the second image is an image of; selecting a periodontal disease detection tool corresponding to the detected area from a plurality of periodontal disease detection tools corresponding to each of the plurality of areas, each of which receives image data including the area as input and outputs information regarding periodontal disease in that area; and detecting the periodontal disease of the user based on the selected periodontal disease detection tool and image data based on the second image. Furthermore, a periodontal disease detection method according to a fifteenth aspect of the present disclosure is a periodontal disease detection method executed by one or more processors, wherein the one or more processors output a first image capturing a specific tooth and the periodontal region of the specific tooth including the gums in the user's oral cavity, obtain a detection result of the user's periodontal disease based on the first image, and perform a predetermined processing on the obtained detection result, wherein the obtained detection result includes a periodontal disease detection tool selected from a plurality of periodontal disease detection tools corresponding to each of a plurality of regions in the oral cavity, each of which is generated to input image data including the region and output information regarding periodontal disease in the region, depending on which of the plurality of regions defined by dividing the tooth row a second image including the periodontal region of the specific tooth generated from the first image is an image of, and the detection result of the user's periodontal disease detected based on image data based on the second image.

[0041] This provides the same effects as the periodontal disease detection system described above.

[0042] A program according to a sixteenth aspect of the present disclosure is a program for causing a computer to execute the periodontal disease detection method of the fourteenth or fifteenth aspect.

[0043] This provides the same effects as the periodontal disease detection system described above.

[0044] These general or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or as any combination of the system, method, integrated circuit, computer program, or recording medium. The program may be pre-stored in the recording medium, or may be supplied to the recording medium via a wide area communication network including the Internet.

[0045] Furthermore, each figure is a schematic diagram and is not necessarily an exact illustration. Therefore, for example, the scales of the figures do not necessarily match. Furthermore, in each figure, substantially the same components are given the same reference numerals, and redundant explanations are omitted or simplified.

[0046] Furthermore, in this specification, terms indicating relationships between elements such as parallelism, as well as numerical values ​​and numerical ranges, are not expressions that only express a strict meaning, but are expressions that also include a substantially equivalent range, for example, a difference of about several percent (or about 10%).

[0047] Furthermore, in this specification, ordinal numbers such as "first" and "second" do not refer to the number or order of components unless otherwise specified, but are used for the purpose of avoiding confusion and distinguishing between components of the same type.

[0048] (Embodiment 1) Hereinafter, a periodontal disease detection system according to this embodiment will be described with reference to Figs. 1 to 7B.

[0049] [1-1. Configuration of Periodontal Disease Detection System] First, the configuration of the periodontal disease detection system according to this embodiment will be described with reference to Figures 1 to 3. Figure 1 is a perspective view of an intraoral camera 10 in the periodontal disease detection system according to this embodiment.

[0050] As shown in FIG. 1, the intraoral camera 10 has a toothbrush-shaped housing that can be handled with one hand, and the housing has a head portion 10a that is placed in the user's oral cavity when taking a photograph, a handle portion 10b that the user holds, and a neck portion 10c that connects the head portion 10a and the handle portion 10b.

[0051] The imaging unit 21 images the surface of the dentition and the periodontal region in the oral cavity irradiated with light including the wavelength range of blue light. The surface of the dentition includes at least one of the buccal (outer) side surface of the dentition and the lingual (inner) side surface of the dentition. The dentition may include, for example, one or more teeth. The blue light is an example of the second light.

[0052] The photographing unit 21 is incorporated into the head unit 10a and the neck unit 10c. The photographing unit 21 has an image pickup element (not shown) and a lens (not shown) arranged on its optical axis LA.

[0053] The imaging element is a photographing device such as a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device), and an image of the teeth is formed by a lens. The imaging element outputs a signal (image data) corresponding to the formed image to the outside. The photographed image by the imaging element is an example of a first RGB image or a first image.

[0054] Furthermore, the captured image is obtained by irradiating the dentition with blue light, and therefore has a blue tint. The captured image is, for example, an image of the side of the dentition and the periodontal region. The side of the dentition may be the lingual side or the buccal side. The side of the dentition may be the maxillary side or the mandibular side. The captured image is, for example, an image of a captured region including one or more teeth (an example of a specific tooth) and the periodontal region of the one or more teeth. The periodontal region is a region including the gingiva of the tooth. The gingiva is the periodontal tissue surrounding the root of a tooth and is also called the gum. The specific tooth may be any tooth, for example, a tooth in a region in which a user or a medical professional such as a dentist wants to detect periodontal disease, or any tooth.

[0055] The photographing unit 21 may further include an optical filter that blocks light of a color emitted from the illumination unit (illumination device) and transmits fluorescence emitted by plaque in response to the light. In this embodiment, the photographing unit 21 may include, as an optical filter, a blue light cut filter that cuts out blue wavelength light components contained in light incident on the image sensor. When light including a blue wavelength range is applied to teeth to detect plaque, if the light including the blue wavelength range is increased to enhance the excitation fluorescence of plaque, blue pixel values ​​will become dominant compared to red and green pixel values, and the entire photographed image will appear blue. To address this, a blue light cut filter cuts out a portion of light including the blue wavelength range from the light before it enters the image sensor. Note that the photographing unit 21 does not necessarily have to include a blue light cut filter.

[0056] The intraoral camera 10 is also equipped with a plurality of first to fourth LEDs 23A to 23D as an illumination unit that irradiates light onto the teeth to be photographed during photography. The first to fourth LEDs 23A to 23D irradiate dental plaque with light of a color that causes the plaque to fluoresce (e.g., single-color light). The first to fourth LEDs 23A to 23D are, for example, blue LEDs that irradiate blue light including a wavelength having a peak at 405 nm (an example of a predetermined wavelength). Note that the first to fourth LEDs 23A to 23D are not limited to blue LEDs, and may be any light source that irradiates light including the wavelength range of blue light.

[0057] The first to fourth LEDs 23A to 23D may be configured to emit light for capturing reference color data that serves as a reference for the color of the user's teeth. The light for capturing the reference color data includes light of a color different from blue light, such as white light. White light is an example of the first light.

[0058] 2 is a schematic diagram of a periodontal disease detection system according to this embodiment. The periodontal disease detection system according to this embodiment is, generally, an information processing system capable of detecting periodontal disease with higher accuracy, and specifically, is configured such that the imaging unit 21 captures fluorescence emitted by dental plaque in response to light from the illumination unit 23, and from one or more captured images, an image more suitable for detecting periodontal disease is obtained, and periodontal disease is detected based on the acquired image.

[0059] As shown in FIG. 2 , the periodontal disease detection system includes an intraoral camera 10 and a mobile terminal 50 .

[0060] The intraoral camera 10 includes a hardware unit 20, a signal processing unit 30, and a communication unit 40.

[0061] The hardware unit 20 is a physical element of the intraoral camera 10 and includes an imaging unit 21 , a sensor unit 22 , an illumination unit 23 , and an operation unit 24 .

[0062] The photographing unit 21 generates image data by photographing the side of the dentition and the periodontal region in the oral cavity, which are irradiated with light of a predetermined wavelength that excites fluorescent substances contained in plaque. The photographing unit 21 receives a control signal from the camera control unit 31, performs operations such as photographing in accordance with the received control signal, and outputs image data of moving or still images obtained by photographing to the image processing unit 32. The photographing unit 21 has the above-mentioned image sensor, optical filter, and lens. The image data is generated, for example, based on light that has passed through the optical filter. Furthermore, the image data is an image that shows multiple teeth, but it is sufficient that the image shows at least one tooth. Note that plaque may be attached to the side of the dentition.

[0063] The photographing unit 21 may also perform photographing to acquire reference color data. The photographing unit 21 may generate image data for acquiring reference color data by photographing the side surface of the dentition and the periodontal region in the oral cavity illuminated with reference light such as white light. The reference light may be light from the illumination unit 23 or light from a light source external to the intraoral camera 10 (external light).

[0064] The sensor unit 22 detects external light incident on the photographing area of ​​the photographed image. For example, the sensor unit 22 detects whether external light is incident into the oral cavity. The sensor unit 22 is disposed, for example, near the photographing unit 21. The sensor unit 22 may be disposed, for example, in the head unit 10a of the intraoral camera 10, similar to the photographing unit 21. In other words, the sensor unit 22 is located inside the user's oral cavity when the photographing unit 21 photographs. Furthermore, when photographing an image for acquiring reference color data, the sensor unit 22 may detect whether white light within a predetermined chromaticity range or a predetermined color temperature is irradiated into the oral cavity.

[0065] The illumination unit 23 irradiates light onto a region among multiple regions in the oral cavity that is to be photographed by the photographing unit 21. As is also known from quantitative visible light induced fluorescence (QLF) methods, bacteria in dental plaque are known to fluoresce reddish pink (excited fluorescence) when irradiated with blue light, and in this embodiment, the illumination unit 23 irradiates blue light onto the region that is to be photographed by the photographing unit 21.

[0066] The illumination unit 23 has the above-mentioned first to fourth LEDs 23A to 23D. The first to fourth LEDs 23A to 23D irradiate the image capture area with light from different directions, for example. This makes it possible to prevent shadows from appearing in the image capture area.

[0067] Each of the first to fourth LEDs 23A to 23D is configured to be at least controllable in terms of dimming. Each of the first to fourth LEDs 23A to 23D may be configured to be controllable in terms of dimming and color adjustment. The first to fourth LEDs 23A to 23D are arranged to surround the imaging unit 21.

[0068] The illumination unit 23 controls the illumination intensity (light emission intensity) according to the photographing area. The illumination intensity of each of the first to fourth LEDs 23A to 23D may be controlled uniformly, or may be controlled to be different from one another. The number of LEDs included in the illumination unit 23 is not particularly limited, and may be one, or five or more. Furthermore, the illumination unit 23 is not limited to having an LED as a light source, and may include other light sources.

[0069] The illumination unit 23 may be configured to emit reference light for acquiring reference color data of the user's teeth. The reference light is emitted at a timing different from that of the blue light.

[0070] The operation unit 24 receives operations from the user. The operation unit 24 is configured with, for example, push buttons, but may also be configured to receive operations by voice, etc. The operation unit 24 receives an operation from the user, for example, whether to take an image for detecting periodontal disease or to take an image for obtaining reference color data.

[0071] The hardware unit 20 may further include a battery (e.g., a secondary battery) that supplies power to each component of the intraoral camera 10, a coil for wireless charging by an external charger connected to a commercial power source, and an actuator necessary for at least one of composition adjustment and focus adjustment.

[0072] The signal processing unit 30 has functional components implemented by a CPU (Central Processing Unit) or an MPU (Micro Processor Unit) that execute various processes described below, and a memory unit 35 such as a ROM (Read Only Memory) or RAM (Random Access Memory) that stores programs for causing the functional components to execute various processes. Specifically, the signal processing unit 30 has a camera control unit 31, an image processing unit 32, a control unit 33, a lighting control unit 34, and the memory unit 35.

[0073] The camera control unit 31 is mounted on, for example, the handle unit 10b of the intraoral camera 10 and controls the imaging unit 21. The camera control unit 31 controls at least one of the aperture and the shutter speed of the imaging unit 21 in response to a control signal from the image processing unit 32, for example.

[0074] The image processing unit 32 is mounted on, for example, the handle unit 10b of the intraoral camera 10, acquires the captured image captured by the imaging unit 21, performs image processing on the acquired captured image, and outputs the captured image after the image processing to the camera control unit 31 and the control unit 33. The image processing unit 32 may also output the captured image after the image processing to the memory unit 35, and store the captured image after the image processing in the memory unit 35.

[0075] The image processing unit 32 is configured by, for example, a circuit, and performs image processing such as noise removal, edge enhancement, etc. on the captured image. Note that the noise removal, edge enhancement, etc. may be performed by the mobile terminal 50.

[0076] The captured image output from the image processing unit 32 (the captured image after image processing) may be transmitted to the mobile terminal 50 via the communication unit 40, and an image based on the transmitted captured image may be displayed on the display unit 56 of the mobile terminal 50. In this way, an image based on the captured image can be presented to the user.

[0077] The control unit 33 is a control device that controls the signal processing unit 30. The control unit 33 controls each component of the signal processing unit 30 based on the detection result of the sensor unit 22, such as external light.

[0078] The illumination control unit 34 is mounted, for example, on the handle portion 10b of the intraoral camera 10 and controls the turning on and off of the first to fourth LEDs 23A to 23D. The illumination control unit 34 is configured, for example, by a circuit. For example, when a user operates the display unit 56 of the portable terminal 50 to start the intraoral camera 10, a corresponding signal is transmitted from the portable terminal 50 to the signal processing unit 30 via the communication unit 40. The illumination control unit 34 of the signal processing unit 30 turns on the first to fourth LEDs 23A to 23D based on the received signal. For example, the illumination control unit 34 may cause the illumination unit 23 to emit blue light when the operation unit 24 receives an operation indicating that imaging for periodontal disease detection will be performed, or may cause the illumination unit 23 to emit white light when the operation unit 24 receives an operation indicating that imaging for reference color data acquisition will be performed.

[0079] In addition to the above programs, the memory unit 35 stores images captured by the image capturing unit 21. The memory unit 35 is realized by, for example, a semiconductor memory such as a ROM or a RAM, but is not limited to this.

[0080] The communication unit 40 is a wireless communication module for wirelessly communicating with the mobile terminal 50. The communication unit 40 is mounted, for example, on the handle portion 10b of the intraoral camera 10, and performs wireless communication with the mobile terminal 50 based on a control signal from the signal processing unit 30. The communication unit 40 performs wireless communication with the mobile terminal 50 in accordance with an existing communication standard such as Wi-Fi (registered trademark) or Bluetooth (registered trademark). Via the communication unit 40, captured images are transmitted from the intraoral camera 10 to the mobile terminal 50, and operation signals are transmitted from the mobile terminal 50 to the intraoral camera 10.

[0081] The mobile terminal 50 acquires an image more suitable for detecting periodontal disease and executes processing for detecting periodontal disease by using, for example, photographing the surface of the dentition and the periodontal region that have reacted to fluorescence when the teeth are irradiated with light including the wavelength range of blue light. The mobile terminal 50 functions as a user interface for the periodontal disease detection system. The mobile terminal 50 is an example of a second information terminal.

[0082] FIG. 3 is a block diagram showing the functional configuration of the mobile terminal 50 according to this embodiment.

[0083] 3 , the mobile terminal 50 includes an acquisition unit 51, a tooth type identification unit 52, an image processing unit 53, an image data extraction unit 54, a periodontal disease detection unit 55, a display unit 56, an output unit 57, and a storage unit 58. The mobile terminal 50 includes a processor, a memory, and the like. The memory is a ROM, a RAM, or the like, and can store programs executed by the processor. The acquisition unit 51, the tooth type identification unit 52, the image processing unit 53, the image data extraction unit 54, the periodontal disease detection unit 55, the display unit 56, and the output unit 57 are realized by a processor or the like that executes programs stored in the memory. The mobile terminal 50 may be realized, for example, by a smartphone or tablet terminal capable of wireless communication.

[0084] The acquisition unit 51 is a wireless communication module for wireless communication with the intraoral camera 10. The acquisition unit 51 acquires captured images from the intraoral camera 10. Specifically, the acquisition unit 51 acquires captured images including one or more teeth and periodontal regions generated by the imaging unit 21. The captured images are images obtained by the intraoral camera 10 photographing teeth that are undergoing a fluorescent reaction by irradiating the teeth with light including a wavelength range of blue light. In this way, the acquisition unit 51 functions as a first acquisition unit that acquires captured images.

[0085] The acquisition unit 51 also acquires reference color data, which indicates the reference color of the natural teeth included in the imaging area and is tailored to the user. For example, the acquisition unit 51 may acquire, as the reference color data, an image (color image) obtained by irradiating the oral cavity with white light from the intraoral camera 10, or may acquire reference color information of the natural teeth. The color information may be, for example, the chromaticity of the natural teeth in the image obtained by irradiating the oral cavity with white light. The color information of the natural teeth may be color information of any one location on the natural teeth, color information of a specific natural tooth, or a statistical value of color information of multiple locations on the natural teeth (e.g., multiple natural teeth). For example, taking chromaticity as an example, the statistical value of the color information is the average value of the chromaticity, but may also be the maximum value, minimum value, mode, median, or the like. In this way, the acquisition unit 51 functions as a second acquisition unit that acquires reference color data.

[0086] The acquisition unit 51 may also acquire information (e.g., medical examination necessity information, which will be described later) from a first information terminal carried by a person other than the user (e.g., a medical professional such as a dentist). The first information terminal may be, for example, a portable terminal such as a mobile terminal, or a stationary terminal such as a PC. The first information terminal is an information terminal different from the portable terminal 50.

[0087] The acquisition unit 51 may include a wired communication module that performs wired communication with the intraoral camera 10.

[0088] The tooth type identification unit 52 identifies the type of tooth (e.g., a specific tooth) included in the captured image (i.e., the first image) from the captured image. Identifying the type of tooth may mean identifying whether the tooth is an incisor, canine, or molar, or whether the tooth is a central incisor, lateral incisor, canine, first premolar, second premolar, first molar, second molar, or third molar (wisdom tooth). The tooth type identification unit 52 may also identify the region of the oral cavity (upper jaw, lower jaw, left or right) in which the tooth is located. Note that the method by which the tooth type identification unit 52 identifies the type of tooth is not particularly limited, and may be, for example, a method using a machine learning model, a method using pattern matching, or any other known method. The machine learning model is a learning model that is trained to output the type of tooth shown in an image containing teeth when the image is input.

[0089] A machine learning model (learning model) is an example of a periodontal disease detection tool. That is, the periodontal disease detection tool may include multiple learning models, each trained to receive image data including a region as input and output information related to periodontal disease in the region. The periodontal disease detection tool may be represented by a machine learning model such as a neural network for estimating periodontal disease detection results from input information (here, an image).

[0090] The tooth type identifying unit 52 may use a corrected image (to be described later) instead of the photographed image to identify the type of tooth included in the corrected image.

[0091] The image processing unit 53 generates a corrected image by adjusting the gain of at least two of the red, green, and blue color components that make up the natural tooth region in the photographed image based on the reference color data. In this embodiment, since the photographed image is a bluish image, the image processing unit 53 performs a process of correcting the color of the natural teeth in the photographed image to the color of the actual natural teeth of the user based on the reference color data of the user's natural teeth. In this way, the reference color data is set based on, for example, the color of the user's natural teeth.

[0092] The image processing unit 53 does not simply perform a process of rendering the teeth achromatic (so-called white balance processing), but rather determines a gain adjustment amount for correcting the color of the natural teeth in the captured image to the reference color data using reference color data stored in the memory unit 58, and uniformly corrects the entire captured image (i.e., the entire image including the teeth and periodontal region) using the determined gain adjustment amount. "Uniformly" here means that the teeth and periodontal region are corrected using the same gain adjustment amount. The corrected image is an example of the second RGB image or the second image.

[0093] In the corrected image generated in this way, the color of the natural teeth matches the color of the actual user's natural teeth. Furthermore, since the teeth and periodontal region are corrected with the same gain, the color of the periodontal region also becomes closer to the color of the actual user's periodontal region compared to when white balance processing is performed.

[0094] The image processing unit 53 may generate a corrected image from the photographed image by adjusting the gain of at least two color components, for example, a first red pixel average value of multiple red pixel values ​​of multiple pixels (first pixels) constituting the natural tooth region in the photographed image, a first green pixel average value of multiple green pixel values ​​of multiple pixels (first pixels), and a first blue pixel average value of multiple blue pixel values ​​of multiple pixels (first pixels), so that the difference from the reference color data falls within a predetermined range. Note that the image processing unit 53 is not limited to using the first red pixel average value, the first blue pixel average value, and the first green pixel average value, and may generate a corrected image from the photographed image using statistical values ​​of the first red pixels, the first blue pixel average value, and the first green pixel average value. Examples of statistical values ​​include, but are not limited to, maximum values, minimum values, modes, medians, etc.

[0095] It is known that when natural teeth are irradiated with excitation light, excitation fluorescence is emitted from the dentin, which passes through the enamel (see FIG. 6 described below) and emits green fluorescence. It is also known that fillings for caries treatment marks (e.g., metal inlays) do not emit excitation fluorescence under blue LED light and are captured as dark (low brightness) images by a camera. For these reasons, the image processing unit 53 can detect the natural teeth excluding the caries treatment marks from the captured image. The caries treatment mark area is an example of a specific pixel area.

[0096] The image processing unit 53 may further detect a plaque region of the tooth from the corrected image. Because teeth and plaque emit different fluorescence, the image processing unit 53 can detect the teeth and plaque from the color of the fluorescent (excited fluorescence) portion in the corrected image. When irradiated with blue light, plaque (plaque region) fluoresces a reddish pink (excited fluorescence). Note that any known method other than the above may be used as a method for detecting the tooth region and plaque region. The plaque region is an example of a specific pixel region.

[0097] The image data extracting unit 54 extracts periodontal region image data from the corrected image, including the gingival region near the boundary between the tooth and the gingiva. It can also be said that the image data extracting unit 54 extracts periodontal region image data from the corrected image, including the free gingiva, which is the gingiva that surrounds the cervical region, including the interdental papilla and marginal gingiva, in a band shape, and a portion of the attached gingiva that is continuous with the free gingiva and extends from the bottom of the gingival sulcus to the gingival-alveolar junction. For example, the periodontal region image data may include the interdental papilla and gingiva on both sides of the tooth.

[0098] 7A and 7B, the periodontal region image data is an image extracted from the corrected image that includes a periodontal region where periodontal disease develops and minimizes tooth regions that are unnecessary for detecting periodontal disease. The image data extracting unit 54 may extract the periodontal region image data by removing part of the tooth region from the corrected image. The periodontal region image data is an example of image data based on the corrected image.

[0099] The periodontal disease detection unit 55 detects periodontal disease of the user based on image data derived from the corrected image. Here, the periodontal disease may be the current progress of periodontal disease or a sign of periodontal disease. The sign includes a symptom that is not currently recognized as periodontal disease but indicates a high possibility of developing periodontal disease. An example of a symptom of periodontal disease is, but is not limited to, the occurrence of gingivitis.

[0100] In this embodiment, the periodontal disease detection unit 55 detects periodontal disease in the user by inputting image data based on the corrected image (periodontal region image data in this embodiment) into a learning model that has been trained to input image data including the gingival region and output information related to periodontal disease in the image data. The information related to periodontal disease includes the presence or absence of periodontal disease, the progress of periodontal disease, signs of periodontal disease, etc. The information related to periodontal disease may also include an estimated result of at least one of periodontal pocket depth, BOP, and GI value.

[0101] The learning model is trained in advance by supervised learning using a learning dataset in which image data including the gingival region (periodontal region image data in this embodiment) is used as input data and information regarding periodontal disease is used as ground truth data. The learning model may be, for example, a model capable of determining each of a plurality of diseases, including cases in which multiple diseases are present, that has been previously obtained by training using teacher data including a plurality of sample data having features for identifying the presence or absence of diseases related to the periodontal region of a plurality of teeth.

[0102] The image data used for learning is a plurality of images showing different stages of periodontal disease progression, such as image data of a normal periodontal region, image data of a periodontal region suspected of having periodontal disease, and image data of periodontal regions with mild, moderate, or severe periodontitis. Furthermore, images for learning may be prepared for each tooth type, region where the tooth is located, and imaging direction (e.g., image captured from the lingual side or image captured from the buccal side). The image data may be an image including the periodontal region extracted by the image data extracting unit 54 (i.e., an edited image), or the corrected image itself.

[0103] The processing unit that generates the learning model may be provided in the periodontal disease detection system, or may be provided in a device outside the periodontal disease detection system.

[0104] The display unit 56 is a display device included in the mobile terminal 50, and displays information indicating the periodontal disease detected by the periodontal disease detection unit 55. The display unit 56 is realized by, for example, a liquid crystal display panel, but is not limited to this.

[0105] The output unit 57 is a wireless communication module for wirelessly communicating with a first information terminal of a person other than the user (e.g., a medical professional such as a dentist). The output unit 57 outputs information indicating periodontal disease detected by the periodontal disease detection unit 55 to the first information terminal of the person other than the user. The output unit 57 also outputs the consultation necessity information acquired by the acquisition unit 51, which is information regarding the need for the user to consult a doctor and entered by the person into the first information terminal, to a second information terminal of the user. The second information terminal may be, for example, an information terminal different from the mobile terminal 50. The dentist may be, for example, the user's doctor.

[0106] The output unit 57 may associate information indicating periodontal disease detected by the periodontal disease detection unit 55 with the type of tooth identified by the tooth type identification unit 52 and output the associated information to at least one of the first information terminal and the second information terminal. In other words, the output unit 57 may display information indicating periodontal disease and the type of tooth in association with each other on at least one of the first information terminal and the second information terminal. This makes it possible to notify at least one of the user and the healthcare professional what type of periodontal disease the periodontal region of which tooth has (or does not have) periodontal disease.

[0107] The output unit 57 may include a wired communication module that performs wired communication with the first information terminal and the second information terminal.

[0108] The memory unit 58 is a storage device that stores various information for detecting periodontal disease in a user. The memory unit 58 may store, for example, a learning model used by the periodontal disease detection unit 55, or may store reference color data. The memory unit 58 may be realized by, for example, a semiconductor memory or the like, but is not limited to this.

[0109] The portable terminal 50 only needs to have a configuration for generating an image more suitable for detecting periodontal disease. For example, the portable terminal 50 does not need to have a configuration for detecting periodontal disease, such as the periodontal disease detection unit 55. In this case, the dentist makes a diagnosis while checking an image more suitable for detecting periodontal disease (e.g., a corrected image) generated by the portable terminal 50. In other words, the portable terminal 50 may function as a support device for generating an image that allows the dentist to make a more accurate diagnosis.

[0110] 4 to 7B, the operation of the periodontal disease detection system configured as described above will be described. Fig. 4 is a flowchart showing the operation of the periodontal disease detection system (periodontal disease detection method) according to this embodiment.

[0111] 4, first, the acquisition unit 51 acquires reference color data of the user's natural teeth and stores it in the storage unit 58 (S11). The acquisition unit 51 may acquire the reference color data before acquiring the photographed image. Note that the reference color data only needs to be acquired before executing step S14, and is not limited to being acquired before acquiring the photographed image.

[0112] Furthermore, since the reference color data differs for each user, the acquisition unit 51 may store the reference color data in association with information indicating the user in the storage unit 58. Furthermore, the process of step S11 may be performed, for example, once for each user. For example, if reference color data is already stored for the user, step S11 may be omitted.

[0113] Next, the acquisition unit 51 acquires a captured image of the photographed region in the user's oral cavity (S12). The captured image may be an image taken by the user himself / herself, or an image of the user's oral cavity taken by a medical professional. The acquisition unit 51 may store the captured image in the storage unit 58.

[0114] Next, the tooth type identification unit 52 identifies tooth regions in the captured image (S13). For example, the tooth type identification unit 52 may identify in which region of the oral cavity the tooth in the captured image is located by using tooth regions that are output from a trained machine learning model obtained by inputting the captured image into the machine learning model.

[0115] Next, the image processing unit 53 generates a corrected image by correcting the color of the teeth in the photographed image based on the reference color data (S14). The image processing unit 53 determines the amount of gain adjustment so that the color data of the teeth in the photographed image matches the reference color data or falls within a predetermined range, and uniformly corrects the entire photographed image (i.e., the entire image including the teeth and periodontal region) using the determined amount of gain adjustment.

[0116] Here, the image processing unit 53 may generate the corrected image by performing the following processing. For example, the image processing unit 53 may identify a specific pixel area including pixels whose values ​​based on the pixel values ​​of the first corrected image obtained by correcting the color of the tooth in the photographed image are within a predetermined range, and may generate a third corrected image as the corrected image by adjusting the gain of at least two of the red, green, and blue color components that constitute the area of ​​the natural tooth in the photographed image excluding the specific pixel area based on the reference color data. Image data based on the third corrected image is an example of image data based on a corrected image. The specific pixel area is an area of ​​the natural tooth where the natural tooth is not exposed, such as an area where plaque is attached.

[0117] Furthermore, for example, the image processing unit 53 may generate an HSV image by converting the color space of a first corrected image in which the color of the teeth in the photographed image has been corrected into an HSV space, and identify as the specific pixel area a pixel area in which one or more pixels among the multiple pixels in the HSV image that satisfy at least one of the following conditions are located: saturation within a first predetermined range, hue within a second predetermined range, and brightness within a third predetermined range.

[0118] Furthermore, for example, the image processing unit 53 may identify as a specific pixel area a pixel area in which one or more pixels are located that satisfy at least one of the following conditions: a plurality of red pixel values ​​within a first range, a plurality of green pixel values ​​within a second range, and a plurality of blue pixel values ​​within a third range, which are possessed by a plurality of images that constitute the natural tooth area in the first corrected image.

[0119] In this way, the image processing unit 53 may identify the specific pixel region from the HSV image obtained by converting the first corrected image, or may identify the specific pixel region from the first corrected image itself.

[0120] Next, the image data extraction unit 54 extracts a gingival region based on the corrected image (S15). Note that the image data extraction unit 54 may generate an HSV image by converting the color space of the first image (RGB image) into an HSV space, and detect an area whose hue (H) falls within a predetermined range as the gingival region.

[0121] Here, the image of the extracted gingival region (periodontal region image) will be further described with reference to FIGS. 5 to 7B. First, the periodontal region will be described with reference to FIGS. 5 and 6. FIG. 5 is a first diagram for explaining the periodontal region in the oral cavity according to this embodiment. FIG. 5 is an image of the oral cavity including the dental region and the periodontal region. FIG. 6 is a second diagram for explaining the periodontal region in the oral cavity according to this embodiment. FIG. 6 shows a cross-sectional view of the periodontal region. Note that although FIG. 6 is a cross-sectional view, hatching has been omitted for convenience.

[0122] As shown in Figures 5(a) and (b), the gingiva includes free gingiva, attached gingiva, and alveolar mucosa. The free gingiva is the part of the gingiva closest to the tooth head and moves with the movement of the cheek, etc. The attached gingiva is the part of the gingiva attached to the bone and does not move with the movement of the cheek, etc. The alveolar mucosa is the part of the gingiva located on the opposite side of the attached gingiva from the free gingiva and moves with the movement of the cheek, etc. The interdental papilla gingiva is the thin gingiva between the teeth; it is weak and prone to inflammation and swelling, and is a place where dirt, plaque, etc. easily accumulate. Periodontal disease is one of the causes of loss of interdental papilla gingiva.

[0123] As shown in Figure 6, the free gingiva is the part of the gingiva that forms the gingival sulcus. The gingival sulcus is a groove between the tooth and the gingiva, where plaque that leads to periodontal disease is likely to accumulate. The gingival margin is the top of the free gingiva and is located on the opposite side of the attached gingiva. The free gingival sulcus is located at the boundary between the free gingiva and the attached gingiva, and the gingival-alveolar junction is located at the boundary between the attached gingiva and the alveolar mucosa. The attached gingiva is the part of the gingiva from the free gingival sulcus to the gingival-alveolar junction.

[0124] Fig. 7A is a diagram for explaining image data extracted by the image data extraction unit 54 according to this embodiment. Fig. 7A (a) is a diagram for explaining a region extracted from a corrected image by the image data extraction unit 54, and Fig. 7A (b) is a diagram showing the extracted image data. The image shown in Fig. 7A (a) is an example of a corrected image, and the image shown in Fig. 7A (b) shows an image obtained by extracting a rectangular region surrounded by a dashed line in Fig. 7A (a) (periodontal region image data extracted by the image data extraction unit 54).

[0125] As shown in (a) of Figure 7A, the image data extraction unit 54 draws dashed-dotted lines (dotted-dotted lines extending in the vertical direction on the paper) that circumscribe the left and right sides of the tooth contour and are perpendicular to the tooth alignment direction. The left and right sides are the parts of the tooth that protrude most toward the adjacent teeth. The image data extraction unit 54 also draws dashed-dotted lines (the upper dashed-dotted lines extending in the horizontal direction on the paper) parallel to the tooth alignment direction so as to include the apex (tip) of the interdental papilla gingiva, and draws dashed-dotted lines (the lower dashed-dotted lines extending in the horizontal direction on the paper) parallel to the tooth alignment direction so as to include the peripheral tooth edge.

[0126] Then, as shown in (b) of Fig. 7A , the image data extraction unit 54 generates periodontal region image data by extracting the rectangular region surrounded by the dashed line shown in (a) of Fig. 7A from the corrected image. In this way, the image data extraction unit 54 extracts, from the corrected image, a rectangular region of the periodontal region of the tooth, including the tip of the interdental papilla gingiva (e.g., the periodontal margin) to the free gingival sulcus, as periodontal region image data. In this case, the periodontal region image data is an image that includes only a portion of the tooth. The tooth region is an area that is unnecessary for detecting periodontal disease (e.g., an area that may reduce detection accuracy), and therefore, as shown in (b) of Fig. 7A , it is preferable that the periodontal region image data include as little tooth region as possible.

[0127] When the periodontal region image data includes the area from the periodontal edge to the free gingival sulcus, the depth of the periodontal pocket can be estimated based on the image data, and when the interdental papilla gingiva becomes periodontally diseased, the interdental papilla gingiva recedes, and gaps appear to form between the teeth, making it possible to determine the progression of the periodontal disease.

[0128] The image data extraction unit 54 is not limited to extracting the region shown in (a) of Figure 7A, and may extract, for example, a rectangular region that is elongated in the vertical direction and surrounded by a dashed line extending in the vertical direction (e.g., a region that includes the periodontal region as well as most of the tooth) from the corrected image as periodontal region image data. From the perspective of detecting periodontal disease with higher accuracy, it is preferable to extract the region surrounded by the dashed line shown in (a) of Figure 7A. The extracted image data is not limited to a rectangular image. The image data extraction unit 54 may extract, from the captured image, a region in the captured image that corresponds to the dashed line shown in (a) of Figure 7A as periodontal region image data.

[0129] Here, a method for forming a rectangular area will be described with reference to Fig. 7B. Fig. 7B is a diagram showing a specific example of a method for forming a rectangular area according to this embodiment. A frame showing a rectangular area will also be referred to as a rectangular frame. Fig. 7B shows an example of a rectangular frame for a maxillary central incisor.

[0130] As shown in (a) of Fig. 7B, the image data extracting unit 54 forms a rectangular frame F1 surrounding a specific tooth in accordance with the dentition. The image data extracting unit 54 identifies the tooth based on, for example, the color of the tooth and gums, and forms a rectangular frame F1 surrounding one tooth based on the shape of the tooth. The four sides of the rectangular frame F1 are formed so as to be in contact with the outer shape of one tooth.

[0131] 7B(b), the image data extraction unit 54 then moves the bottom edge of the rectangular frame F1 upward within a range that includes both the left and right interdental papillae and gingiva (e.g., the tips of the interdental papillae and gingiva) to form a rectangular frame F2. At this time, it is preferable to move the rectangular frame F1 upward within a range that includes both the left and right interdental papillae and gingiva in order to remove as much tooth information as possible.

[0132] As shown in (c) of Fig. 7B, the image data extraction unit 54 then forms a rectangular frame F3 by moving the upper end of the rectangular frame F2 upward from the gingival margin by a predetermined distance D. The predetermined distance D is, for example, 2 mm, but is not limited to this, and may be 1 mm or 3 mm or more. As a result, the dashed-dotted frame shown in (b) of Fig. 7A is formed.

[0133] In feature engineering, it is more important to find meaningful "features" that are closely related to the target variable (disease progression) than to find complex mathematical or statistical transformations to improve prediction accuracy. However, finding such features requires knowledge, experience, and intuition related to dentistry, knowledge of data (the meaning of data items and relationships between tables), and knowledge of statistics and machine learning (statistical stability and predictive power), making feature engineering one of the most important and most difficult steps in the process of developing a machine learning model.

[0134] Through careful study, the inventors of the present application discovered that, as described above, by reducing the area in which teeth are visible (reducing the influence of teeth on the detection of periodontal disease), training data can be generated that makes it easier to obtain gingival characteristics.

[0135] 4 again, the periodontal disease detection unit 55 performs detection of periodontal disease based on the image of the gingival region (S16). The periodontal disease detection unit 55 inputs the periodontal region image data extracted by the image data extraction unit 54 (e.g., the image shown in (b) of FIG. 7A) into the learning model, and obtains an estimation result of periodontal disease, which is the output of the learning model. The periodontal disease detection unit 55 obtains, for example, the estimation result as a detection result of periodontal disease.

[0136] Next, the output unit 57 outputs the detection result of the periodontal disease detection unit 55 (S17). The output unit 57 transmits the detection result via communication to an information terminal of at least one of the user and the medical professional. This allows the periodontal disease detection result to be notified to at least one of the user and the medical professional.

[0137] As described above, in the periodontal disease detection system according to the present embodiment, color casts of specific teeth and gums caused by the light irradiating the oral cavity when capturing an image are corrected based on pre-stored reference color data for specific teeth that is specific to the user, without using color calibration color patches or the like. This allows the colors of the teeth and gums in the image to be closer to the colors of the teeth and gums of the user. By detecting periodontal disease using such an image (corrected image), it is possible to prevent a decrease in the accuracy of periodontal disease detection due to the image.

[0138] (Variation 1 of Embodiment 1) A periodontal disease detection system according to this variation will be described below with reference to Figures 8A to 8C. The following description will focus on differences from Embodiment 1, and descriptions of content that is the same as or similar to Embodiment 1 will be omitted or simplified. In this variation, an example will be described in which the periodontal disease detection unit detects periodontal disease using a rule base.

[0139] For example, in this modified example, in determining periodontal disease risk, a method is adopted in which a determination rule is used to determine risk for content for which a person can construct a determination rule (for example, a determination rule based on the shape of the tip of the interdental papilla gingiva). The determination rule is an example of a periodontal disease detection tool. The determination rule is generated to output information related to periodontal disease from image data.

[0140] The configuration of the periodontal disease detection system of this modified example may be the same as that of the periodontal disease detection system 1 according to Embodiment 1, and will be described below using the reference numerals of the periodontal disease detection system 1. In step S16 shown in Fig. 4, the periodontal disease detection unit 55 according to this modified example automatically detects periodontal disease in the user using a preset determination rule.

[0141] In all of the R, G, and B values ​​in the RGB images of the gums, there is a large variation in the color distribution of the free and attached gums across the user for both the normal and diseased gums, and there is overlap in the color distribution of the normal gums and the gums suspected of having periodontal disease. Therefore, if a judgment is made without taking into account the R, G, and B values ​​of the user's normal gums, there are cases where gums with a light gum color are suspected of having periodontal disease, and cases where gums with a dark gum color are normal.

[0142] Therefore, in this modified example, multiple determination rules are set based on the ranges of the R, G, and B values ​​of the normal gingival group. A specific setting method will be described below. FIGS. 8A to 8C are diagrams showing examples of each group rule base (determination rule) in this modified example. FIG. 8A shows the determination rule when the normal gingival R value is A1 to A2 for the free gingival and B1 to B2 for the attached gingival. FIG. 8B shows the determination rule when the normal gingival R value is A3 to A4 for the free gingival and B3 to B4 for the attached gingival. FIG. 8C shows the determination rule when the normal gingival R value is A5 to A6 for the free gingival and B5 to B6 for the attached gingival.

[0143] 8A to 8C, for example, the n-th group rule base stores periodontal disease determination criteria for the case where the normal range for free gingiva is A2n-1≦R value≦A2n and the normal range for attached gingiva is B2n-1≦R value≦B2n. Here, the rule base groups are classified only by the R value, but the determination rules may also be classified based on the G value, the B value, or a combination of two of the R value, the G value, and the B value, or a combination of three of the R value, the G value, and the B value.

[0144] For example, the rule base (each determination rule) for each group may set a determination rule for the progression level of periodontal disease based on the amount of change in the R value, G value, and B value from normal gingiva obtained from the user. The rule base (each determination rule) for each group may divide gingiva into free gingiva and attached gingiva, and set the progression level of periodontal disease based on the amount of change in each of the R value, G value, and B value from normal gingiva values. In other words, the determination criteria for periodontal disease may be set for each user. The progression level is an example of the degree of progression.

[0145] The progression level may be, for example, a stage of progression of periodontal disease, for example, progression level 1 may be gingivitis, progression level 2 may be mild periodontal disease, progression level 3 may be moderate periodontal disease, and progression level 4 may be severe periodontal disease.

[0146] Generally, as the progression level of periodontal disease progresses from normal gingiva to attached gingiva, all brightness indices (R, G, B) decrease, and the amount of change tends to be greater for free gingiva than for attached gingiva. Based on this knowledge, the periodontal disease detection unit 55 may output the progression level of periodontal disease by prioritizing the evaluation based on free gingiva. Alternatively, the periodontal disease detection unit 55 may output the progression level of periodontal disease for both free gingiva and attached gingiva.

[0147] 8A to 8C are set in advance and stored in the storage unit 58. The number of determination rules stored in the storage unit 58 is not particularly limited as long as it is two or more. The determination rules may also be rules for detecting the presence or absence of periodontal disease.

[0148] Note that each determination rule may be set for each color space, or may be set individually for each region in the oral cavity. Alternatively, if color information (e.g., absolute color values ​​and color variations) of the user's healthy gums has been acquired in advance, the determination rule may be set according to the color information. In other words, the determination rule may be set for each user. For example, color information (absolute color values, e.g., R value, G value, B value) at multiple points on the user's gums may be acquired, the variation in the R value may be calculated, and the normal range of the R value may be determined based on the statistical value (e.g., average value) of the R value and the variation in the R value. The G value and B value may also be determined in the same manner as the R value.

[0149] The periodontal disease detection unit 55 detects periodontal disease using the above-described judgment rule in step S16 shown in Fig. 4. The periodontal disease detection unit 55 is configured to compare the input image data with the judgment rule and detect periodontal disease according to the comparison result. Processing the input image data using the judgment rule is also referred to as inputting the image data to the judgment rule.

[0150] For example, the periodontal disease detection unit 55 detects periodontal disease in the user based on the R, G, and B values ​​of the color of the gums acquired from image data based on the second RGB image and the judgment rule. The periodontal disease detection unit 55 extracts a judgment rule corresponding to the region to which the target tooth belongs or the user from among the multiple judgment rules, and judges periodontal disease using the extracted judgment rule.

[0151] The periodontal disease detection unit 55 extracts color information of the gums from image data based on the second RGB image, and determines whether the gums are normal or not based on the difference (amount of change) between the color value indicated by the extracted color information and a predetermined normal range, and if not normal, determines the progression level of the periodontal disease for both the free gums and the attached gums. The color value may be the R value, G value, or B value, or may be hue information, saturation information, lightness information, or hue information, saturation information, or brightness information.

[0152] For example, the periodontal disease detection unit 55 may use a judgment rule created from the range of R, G, and B values ​​in the RGB color space of normal gum color and the range of R, G, and B values ​​of gum color for each of multiple stages of periodontal disease, and input the R, G, and B values ​​of the gum color obtained from image data based on the second RGB image to detect the user's periodontal disease in that area.

[0153] Furthermore, for example, the periodontal disease detection unit 55 may use a judgment rule created from at least one of the hue information, saturation information, and brightness information of normal gum color in the HSV color space and at least one of the ranges of hue information, saturation information, and brightness information of gum color for each of multiple stages of periodontal disease to input at least one of the hue information, saturation information, and brightness information of the gum color obtained from image data based on the second RGB image and detect the user's periodontal disease in the area.

[0154] Furthermore, for example, the periodontal disease detection unit 55 may use a judgment rule created from at least one of the hue information, saturation information, and luminance information of normal gum color in the HSL color space and at least one range of hue information, saturation information, and luminance information of gum color for each of multiple stages of periodontal disease to input at least one of the hue information, saturation information, and luminance information of the gum color obtained from image data based on the second RGB image and detect the user's periodontal disease in the area.

[0155] The difference in gum color is the difference in color between the darkest and lightest parts of the gums, and the determination rule may be a rule that associates this difference with normal and advanced levels. Since the color of gums (e.g., normal gum color) varies from user to user, using the color difference between the dark and light parts of the gums makes it possible to detect periodontal disease without relying on the color of each user's own gums.

[0156] Although the example in which color is used as an evaluation parameter when using the determination rule has been described, the shape of the gums may be used instead of or in addition to the color. An example in which the shape of the gums is used will be described in a modification of the second embodiment.

[0157] (Variation 2 of Embodiment 1) A periodontal disease detection system according to this variation will be described below with reference to Fig. 9. The following description will focus on differences from Embodiment 1, and descriptions of content that is the same as or similar to Embodiment 1 will be omitted or simplified. In this variation, training data for training a machine learning model will be described.

[0158] The configuration of the periodontal disease detection system of this modified example may be the same as that of the periodontal disease detection system 1 according to the first embodiment, and the following description will use the reference numerals of the periodontal disease detection system 1.

[0159] Fig. 9 is a diagram for explaining the learning data according to this modification. In Fig. 9, numbers are assigned to each type of tooth in the upper and lower jaws. For example, "1" indicates a central incisor, and "2" indicates a lateral incisor.

[0160] As shown in Fig. 9, multiple points are set for one tooth to obtain correct answer data for the training data. In the example of Fig. 9, three points (1 (left), 2 (center), and 3 (right)) are set on the outer sides of the maxillary central incisors, and three points (4 to 6) are set on the buccal sides of the maxillary central incisors. In the example of Fig. 9, three points (11 to 13) are set on the outer sides of the mandibular central incisors, and three points (14 to 16) are set on the buccal sides of the mandibular central incisors. The measurement points are set in the gingival region at the boundary between the tooth and the gingiva.

[0161] Measurements include at least one of periodontal pocket size and BOP value, and the measurement items in this modification include both periodontal pocket size and BOP value. That is, in this modification, periodontal pocket size and BOP value measurements are performed at six locations for each tooth (e.g., locations 1 to 6 for the maxillary central incisor). Also, in this modification, periodontal pocket size and BOP value measurements are performed at three locations for each tooth (e.g., locations 1 to 3 or 4 to 6 for the maxillary central incisor).

[0162] The pocket values ​​of periodontal pockets may be set into nine classes, one class for each of 1 to 9 mm, or two classes, 3 mm or less and 4 mm or more, or three classes, 2 mm or less, 3 to 5 mm, and 6 mm or more, or five classes, 2 mm or less, 3 mm, 4 mm, 5 mm, and 6 mm or more.

[0163] The BOP score is assigned a value of 0 or 1 depending on whether bleeding is present or absent.

[0164] Here, the following additional data is added to the image as learning data for the machine learning model:

[0165] The information includes (1) information indicating whether the image was taken of the upper or lower jaw, (2) information indicating whether the image was taken of the right or left side, with the front teeth (central incisors) at the center, (3) information indicating tooth numbers (1 to 8), (4) information indicating whether the image is of the lingual side or the buccal side, (5) information indicating periodontal pocket values, and (6) information indicating BOP values. The information indicating (5) periodontal pocket values ​​and (6) BOP values ​​include the measurement results for each measurement point. For example, the following information may be added: "upper jaw, left, number 5, lingual side, pocket values ​​3, 2, 5, BOP values ​​0, 0, 1."

[0166] (5) Periodontal pocket value and (6) BOP value are used as correct labels during learning. Note that tooth numbers are set to exclude primary teeth, but are not limited to this.

[0167] By providing a set containing a large number of measured values ​​(correct labels) of periodontal pocket depth in each periodontal region as training data and performing machine learning using a predetermined algorithm, the judgment accuracy of the generated machine learning model can be improved. Furthermore, based on the estimated periodontal pocket depth and the BOP value, the machine learning model can be made to determine at least one of the degree of progression of periodontal disease and the need for periodontal surgical treatment.

[0168] Second Embodiment A periodontal disease detection system according to this embodiment will be described below with reference to FIGS.

[0169] [2-1. Configuration of Periodontal Disease Detection System] The periodontal disease detection system according to this embodiment is an information processing system for detecting periodontal disease in a user based on images of a predetermined region in the oral cavity, and differs from the periodontal disease detection system according to Embodiment 1 in that it includes a mobile terminal 50a instead of the mobile terminal 50. The following description will focus on differences from Embodiment 1, such as the configuration of the mobile terminal 50a, and descriptions of identical or similar configurations will be omitted or simplified.

[0170] FIG. 10 is a block diagram showing the functional configuration of a mobile terminal 50a according to this embodiment.

[0171] As shown in FIG. 10, the mobile terminal 50 a does not include the tooth type identification unit 52 of the mobile terminal 50 according to the first embodiment, but includes an area detection unit 151 and a rule selection unit 152 .

[0172] The image processing unit 53 generates a corrected image in which the color of the captured image has been corrected. The image processing unit 53 may generate the corrected image from the captured image by using reference color data as in the first embodiment, or may generate the corrected image from the captured image by executing white balance processing.

[0173] The region detection unit 151 detects which of multiple regions in the oral cavity defined by dividing the tooth row the tooth in the corrected image generated by the image processing unit 53 belongs to. The multiple regions are regions obtained by dividing the oral cavity into groups of teeth with similar characteristics. Similar characteristics include similar tooth shapes. Note that if the shape of the teeth changes, the shape of the gums may also change. Therefore, similar tooth shapes here mean similar gum shapes.

[0174] 11A and 11B are diagrams illustrating a plurality of intraoral regions according to the present embodiment. Fig. 11A is a diagram in which regions having similar shapes of teeth and gums are divided by dashed lines. Fig. 11B is a diagram showing an example of a plurality of regions set in the present embodiment.

[0175] 11A shows six regions in total, three for each of the upper and lower jaws. Specifically, the upper jaw includes three regions: the right upper jaw including the right molars, the front upper jaw including the central incisors, lateral incisors, and canines, and the left upper jaw including the left molars. The lower jaw includes three regions: the right lower jaw including the right molars, the front lower jaw including the central incisors, lateral incisors, and canines, and the left lower jaw including the left molars.

[0176] 11(b), the multiple regions include a first region R1 to a fourth region R4. Specifically, the first region R1 indicates the tongue-side region of the molars of the upper and lower jaws, the second region R2 indicates the buccal-side region of the molars of the upper and lower jaws, the third region R3 indicates the tongue-side region of the anterior teeth of the upper and lower jaws, and the fourth region R4 indicates the buccal-side region of the anterior teeth of the upper and lower jaws.

[0177] The number of regions may be two or more, and may be two or three, or may be five or more.

[0178] FIG. 12 is a diagram showing an example of an image obtained by photographing a plurality of regions according to this embodiment. FIG. 12(a) shows an image of a mandibular anterior tooth photographed from the lingual side. The image shown in FIG. 12(a) is an example of an image photographed in the third region R3 shown in FIG. 11(b). FIG. 12(b) shows an image of a mandibular anterior tooth photographed from the buccal side. The image shown in FIG. 12(b) is an example of an image photographed in the fourth region R4 shown in FIG. 11(b). FIG. 12(c) shows an image of a mandibular molar photographed from the lingual side. The image shown in FIG. 12(c) is an example of an image photographed in the first region R1 shown in FIG. 11(b). FIG. 12(d) shows an image of a mandibular molar photographed from the buccal side. The image shown in FIG. 12(d) is an example of an image photographed in the second region R2 shown in FIG. 11(b).

[0179] In this embodiment, the region detection unit 151 detects which of the first region R1 to fourth region R4 the corrected image is an image of. In other words, the region detection unit 151 detects from which of the first region R1 to fourth region R4 the teeth and periodontal region shown in the corrected image were photographed.

[0180] The method by which the region detection unit 151 detects the region is not particularly limited, and may be, for example, a method using a machine learning model, a method using pattern matching, or any other known method. The machine learning model is a learning model that is trained to output, when an image including teeth and periodontal regions is input, which of the first region R1 to fourth region R4 the teeth and periodontal region shown in the image belongs to.

[0181] 12(a) to 12(d), the shapes of teeth and gums vary depending on at least one of the tooth type (molar, canine, anterior tooth) and imaging direction (cheek side, lingual side). Furthermore, the degree of gum deformation associated with the type and progression of periodontal disease is also thought to vary depending on at least one of the tooth type and imaging direction. Therefore, if all teeth and gums are trained using a single learning model, there is a risk of erroneous detection of the progression of periodontal disease due to the unique shapes of each tooth and gum.

[0182] Therefore, in this embodiment, the memory unit 58 pre-stores, for each of the multiple regions, multiple learning models trained using images corresponding to the region. The multiple learning models correspond to each of the multiple regions, and each learning model is trained to input image data of the teeth and gums in the region and output information about periodontal disease in the region. In other words, for each of the multiple regions, a learning model corresponding to the region on a one-to-one basis is generated. The information about periodontal disease includes the presence or absence of periodontal disease, the progression of periodontal disease, signs of periodontal disease, etc. The information about periodontal disease may also include estimated results of at least one of periodontal pocket depth, BOP, and GI value.

[0183] Each of the plurality of learning models is associated with information indicating which of the plurality of regions the learning model corresponds to. In this embodiment, each of the plurality of learning models is associated with information indicating one of the first region R1 to the fourth region R4.

[0184] In this embodiment, there are learning models corresponding to the corrected image based on the captured image of the first region R1, the corrected image based on the captured image of the second region R2, the corrected image based on the captured image of the third region R3, and the corrected image based on the captured image of the fourth region R4. That is, in this embodiment, four learning models corresponding one-to-one to each of the four regions are created in advance and stored in the storage unit 58.

[0185] The rule selection unit 152 selects a learning model corresponding to the area detected by the area detection unit 151 from among the multiple learning models stored in the storage unit 58. The rule selection unit 152 is an example of a tool selection unit.

[0186] The periodontal disease detection unit 55 detects periodontal disease in the user using the learning model selected by the rule selection unit 152. The periodontal disease detection unit 55 detects periodontal disease in the user by inputting image data based on the corrected image (e.g., periodontal region image data) into the selected learning model.

[0187] Here, images used for training a plurality of learning models will be described with reference to FIG. 13 . FIG. 13 is a diagram showing an example of an image used for training a learning model according to this embodiment. In each image shown in FIG. 13 , the area enclosed by a dashed-dotted rectangular frame may be extracted by the image data extraction unit 54 or the like, and the extracted image data may be used as image data for training. Note that the dashed-dotted rectangular frame may be formed by the method shown in FIG. 7A (a).

[0188] The image shown in (a) of Fig. 13 is an image of a mandibular anterior tooth photographed from the lingual side, and is used for training a learning model to which an image of the third region R3 is input. The image shown in (b) of Fig. 13 is an image of a mandibular anterior tooth photographed from the buccal side, and is used for training a learning model to which an image of the fourth region R4 is input. The image shown in (c) of Fig. 13 is an image of a mandibular molar photographed from the lingual side, and is used for training a learning model to which an image of the first region R1 is input. The image shown in (d) of Fig. 13 is an image of a mandibular molar photographed from the buccal side, and is used for training a learning model to which an image of the second region R2 is input.

[0189] In this way, a learning image (periodontal region image) is generated for each group of teeth belonging to a region, and the periodontal region image corresponding to that region is input into a learning model corresponding to that region, thereby performing learning. Furthermore, the images shown in (a), (b), and (d) of Fig. 13 are not used in the learning model to which an image of the first region R1 is input; the images shown in (a) to (c) of Fig. 13 are not used in the learning model to which an image of the second region R2 is input; the images shown in (b) to (d) of Fig. 13 are not used in the learning model to which an image of the third region R3 is input; and the images shown in (a), (c), and (d) of Fig. 13 are not used in the learning model to which an image of the fourth region R4 is input.

[0190] Each learning model is trained using a plurality of images showing different stages of progression of periodontal disease. The training method for each learning model is the same as that in the first embodiment, and therefore a description thereof will be omitted.

[0191] The processing unit that generates the learning model may be provided in the periodontal disease detection system, or may be provided in a device outside the periodontal disease detection system.

[0192] [2-2. Operation of Periodontal Disease Detection System] Next, the operation of the periodontal disease detection system configured as described above will be described with reference to Fig. 14. Fig. 14 is a flowchart showing the operation of the periodontal disease detection system (periodontal disease detection method) according to this embodiment.

[0193] 14, first, the acquisition unit 51 acquires a captured image of a region in the oral cavity of a user (S101). Step S101 corresponds to step S12 shown in FIG.

[0194] Next, the image processing unit 53 and the image data extraction unit 54 generate an image of the periodontal region of the specific tooth from the captured image (S102). Step S102 corresponds to steps S14 and S15 shown in FIG.

[0195] Next, the region detection unit 151 detects an intraoral region from the image of the periodontal region or the corrected image based on the type of tooth shown in the image and the imaging direction (S103). The region detection unit 151 detects which of the first region R1 to fourth region R4 the image corresponds to by using a machine learning model, pattern matching, or the like.

[0196] Next, the rule selection unit 152 selects a learning model for detecting periodontal disease from among the plurality of learning models stored in the storage unit 58 based on the intraoral region detected by the region detection unit 151 (S104). The rule selection unit 152 selects, from the plurality of learning models, a learning model associated with the region detected by the region detection unit 151 as the learning model for detecting periodontal disease in the image.

[0197] In this way, there are learning models corresponding to each of the multiple regions in the oral cavity, and the learning model corresponding to the region detected by the region detection unit 151 is selected by the rule selection unit 152.

[0198] Next, the periodontal disease detection unit 55 performs periodontal disease detection based on the image of the gingival region and the selected learning model (S105). The periodontal disease detection unit 55 inputs the image of the gingival region to the selected learning model, and obtains the detection result of periodontal disease as the output of the learning model. Step S105 corresponds to step S16 shown in FIG. 4.

[0199] Next, the output unit 57 outputs the detection result of the periodontal disease detection unit 55 (S106). Step S106 corresponds to step S17 shown in Fig. 4. Note that information indicating which learning model was used to detect periodontal disease may be included in the detection result and output.

[0200] In this way, in this embodiment, periodontal disease in the periodontal region of a tooth can be detected based on a learning model trained using periodontal region images with similar tooth shapes, thereby improving the accuracy of periodontal disease detection compared to, for example, detecting periodontal disease using a single learning model. Furthermore, evaluation is performed for each region using a learning model corresponding to that region, thereby improving the accuracy of periodontal disease detection.

[0201] Furthermore, when detecting periodontal disease using a single learning model, the types of classification (deformation, discoloration) performed by the learning model are the same as when detecting periodontal disease using learning models for each region as in this embodiment, and the number of classes (e.g., progression levels of periodontal disease) is not very large, so a large network is not required. Therefore, the learning model according to this embodiment can reduce the amount of processing.

[0202] Furthermore, since the features to be determined (i.e., the number of classes) are narrowed down, the learning efficiency of the learning model is high. That is, according to this embodiment, the learning efficiency of the learning model is high, so that the detection accuracy of the learning model can be effectively improved.

[0203] (Variation of Embodiment 2) A periodontal disease detection system according to this variation will be described below with reference to FIGS. 15 to 18B. The following description will focus on differences from Embodiment 2, and descriptions of content that is the same as or similar to Embodiment 2 will be omitted or simplified. This variation differs from the periodontal disease detection system according to the embodiment in that the multiple periodontal disease detection tools selected by the rule selection unit (an example of a tool selection unit) include at least one or more determination rules. The multiple periodontal disease detection tools selected by the rule selection unit may include one or more machine learning models, and may include, for example, one or more determination rules and one or more machine learning models. The machine learning model is generated to output information related to periodontal disease from image data. The determination rule and the machine learning model are examples of tools.

[0204] The configuration of the periodontal disease detection system of this modified example may be the same as that of the periodontal disease detection system 2 according to the second embodiment, and the following description will use the reference numerals of the periodontal disease detection system 2.

[0205] As shown in Figure 15, when an area within the oral cavity is detected by the area detection unit 151 (S103), the rule selection unit 152 selects two periodontal disease detection tools that perform periodontal disease detection from among the multiple periodontal disease detection tools stored in the memory unit 58 based on the area within the oral cavity detected by the area detection unit 151 (S204).

[0206] Here, the plurality of periodontal disease detection tools each include at least one or more determination rules for inputting an image of the gingiva between adjacent teeth for a tooth in image data including the region and outputting information about periodontal disease in the region. The one or more determination rules are created, for example, from gingival shape data at a plurality of stages of periodontal disease for each tooth belonging to the plurality of regions.

[0207] The multiple periodontal disease detection tools may include multiple determination rules, or may include one or more machine learning models and one or more determination rules. The determination rule may be a rule using gingival color as shown in the modified example of the first embodiment, or may be a rule using gingival shape. The gingival color and gingival shape (e.g., the shape of the gingival papilla) are evaluation parameters in the rule base.

[0208] For example, when a machine learning model and a determination rule are selected, a periodontal disease that is difficult to detect by one of the machine learning model and the determination rule may be detected by the other of the machine learning model and the determination rule. Since the detection of periodontal disease can be mutually complemented, it is possible to prevent periodontal disease from being overlooked.

[0209] The rule selection unit 152 may select two periodontal disease detection tools using a table that corresponds areas of the oral cavity with the periodontal disease detection tools to be used, or may select two periodontal disease detection tools based on the history of periodontal disease detection tools used to detect periodontal disease in the area, or may select two periodontal disease detection models using some other method.

[0210] The rule selection unit 152 may select three or more periodontal disease detection tools in step S204. When three or more periodontal disease detection tools are selected, the number of selected machine learning models and determination rules is not particularly limited. The number of periodontal disease detection tools selected by the rule selection unit 152 may be set by the user, for example.

[0211] Next, the periodontal disease detection unit 55 performs periodontal disease detection based on the image and the two selected periodontal disease detection models (S205). When a machine learning model and a determination rule are selected as the two periodontal disease detection models, the periodontal disease detection unit 55 performs periodontal disease detection using the machine learning model and performs periodontal disease detection using the determination rule. Furthermore, when two determination rules are selected as the two periodontal disease detection models, the periodontal disease detection unit 55 performs periodontal disease detection using each of the two determination rules.

[0212] Here, the process of detecting periodontal disease using the shape of the gingiva will be described with reference to Figures 16 to 18B. The angle of the tip of the interdental papilla gingiva, which is one example of the shape of the gingiva, will be described with reference to Figures 16 to 17C, and the radius of the tip of the interdental papilla gingiva, which is another example of the shape of the gingiva, will be described with reference to Figures 18A and 18B.

[0213] 16 is a diagram showing the angle θ of the tip of the interdental papilla gingiva in this modified example. When periodontal disease develops, the gums between the teeth swell due to inflammation, causing the shape of the interdental papilla gingiva to change. Therefore, by comparing the angle θ based on the tip of the interdental papilla gingiva as the shape of the interdental papilla gingiva with the angle based on the tip of the interdental papilla gingiva in healthy gums, it is possible to determine whether gingivitis or periodontal disease has developed from the angle of the tip.

[0214] Figure 16 shows the angle θ of each region when it is normal. The angle θ of each region when it is normal may differ. Therefore, one or more determination rules are used in each region, in which the angle θ is associated with normal and advanced levels. For convenience, Figure 16 shows only the angle θ of one of the interdental papilla-gingiva on the left and right sides of a tooth.

[0215] When using a determination rule including the angle θ, it is necessary to detect the angle θ, and therefore an image including a portion of the tooth and at least one of the left and right interdental papillae and gingiva of the tooth is required. For example, from the viewpoint of preventing periodontal disease from being overlooked, it is preferable to prepare an image including a portion of the tooth and the left and right interdental papillae and gingiva of the tooth. The example in Fig. 16 shows an image including the left and right interdental papillae and gingiva.

[0216] For example, the image data extractor 54 may extract, from the second image, an area including at least a portion of a tooth (e.g., a specific tooth) and at least one of the left and right interdental papillae and gingiva of the tooth as image data based on the second image. This extraction process may be performed in step S102 or in step S205. In this way, when using a determination rule including a rule related to shape, it is preferable to prepare an image different from the image input to the machine learning model.

[0217] 17A to 17C are diagrams illustrating a method for calculating the angle of the tip of the interdental papilla gingiva according to this modified example.

[0218] As shown in Fig. 17A, the periodontal disease detection unit 55 uses image processing to identify the tip (e.g., apex) of the interdental papilla gingiva between teeth (e.g., a specific tooth) from the image. For example, the periodontal disease detection unit 55 may determine the position of the uppermost gingiva between the teeth as the tip of the interdental papilla gingiva, or may determine a position having a predetermined shape as the tip of the interdental papilla gingiva. Fig. 17A shows an example in which the "●" portion at the tip of the arrow has been identified as the tip of the interdental papilla gingiva.

[0219] 17B and 17C, the periodontal disease detection unit 55 then draws a line segment downward from the tip of the interdental papilla gingiva, rotates the line segment clockwise and counterclockwise around the tip as an axis, and fixes the line segment at the position where it first contacts the tooth. As a result, an angle θ is formed between the two line segments and the tip.

[0220] When the shape of the gums includes the angle θ, a determination rule is used in which the angle θ is associated with normal and advanced levels, or normal and abnormal.

[0221] 18A and 18B are diagrams showing the radius r of the tip of the interdental papilla gingiva in this modified example. When periodontal disease develops, the gums between the teeth swell due to inflammation, causing the tips of the triangles formed by the interdental papilla gingiva to become rounded. Therefore, by comparing the radius of the rounded tip of the interdental papilla gingiva of healthy gums with the radius of the rounded tip of the interdental papilla gingiva of periodontal disease, it is possible to determine whether gingivitis or periodontal disease has developed from the radius of the rounded tip.

[0222] 18A shows the radius r when periodontal disease is present, and FIG. 18B shows the radius r when periodontal disease is not present. The radius r may be, for example, the radius of curvature of the tip of the interdental papilla gingiva. When periodontal disease is present, the radius r tends to be large.

[0223] When the shape of the gums includes the radius r, a determination rule is used in which the radius r is associated with normal and advanced levels, or normal and abnormal.

[0224] The periodontal disease detection unit 55 then detects the user's periodontal disease based on the periodontal disease detection results of the two periodontal disease detection tools. For example, the periodontal disease detection unit 55 may compare the periodontal disease detection results of the two periodontal disease detection tools and determine the detection result of the one with the worse symptoms of periodontal disease as the detection result of the user's periodontal disease.

[0225] Furthermore, when a determination rule and a machine learning model are selected, the periodontal disease detection unit 55 detects periodontal disease of the user by taking into consideration both a first detection result, which is a detection result of periodontal disease according to the determination rule, and a second detection result, which is a detection result of periodontal disease according to the machine learning model. For example, when periodontal disease is not detected in the first detection result and periodontal disease is detected in the second detection result, the periodontal disease detection unit 55 may prioritize the first detection result and determine that periodontal disease has not been detected as the detection result of periodontal disease of the user.

[0226] A detection result in which the symptoms of periodontal disease are judged to be worse can be rephrased as a detection result in which the periodontal disease is more advanced.

[0227] Referring again to FIG. 15 , next, the output unit 57 outputs the detection result of the periodontal disease detection unit 55 (S106). When the periodontal disease detection unit 55 determines that the detection result showing worse periodontal disease symptoms is the detection result of the user's periodontal disease, the output unit 57 notifies the user of the worse detection result. Furthermore, when a machine learning model is used, the periodontal disease symptoms may be detected as worse than they actually are due to the influence of factors such as the pattern of the gums. Therefore, when the periodontal disease detection unit 55 prioritizes the first detection result, the output unit 57 may also notify the user of the second detection result as additional information.

[0228] (Other Embodiments) The periodontal disease detection systems according to the first and second embodiments and each of the modified examples (embodiments, etc.) of the present disclosure have been described above, but the present disclosure is not limited to these embodiments, etc.

[0229] For example, in the above-described embodiment, an example has been described in which the intraoral camera 10 is used primarily for photographing teeth, but the intraoral camera 10 may be an oral care device equipped with a camera. For example, the intraoral camera 10 may be an oral irrigator equipped with a camera.

[0230] Furthermore, in the above-described embodiment, the mobile terminal 50, 50a is exemplified as the second information terminal, but the second information terminal may be a stationary information terminal.

[0231] In addition, in the above embodiments, examples have been described in which the mobile terminals 50, 50a are equipped with a display unit, but this is not limited to this, and a display device capable of communicating with the mobile terminals 50, 50a may be provided as a device separate from the mobile terminals 50, 50a.

[0232] Furthermore, the machine learning model according to the above-described embodiments has one or more computational parameters that can be adjusted by machine learning. The machine learning model may be configured, for example, by a neural network, a regression model, a decision tree model, a support vector machine, or other functional formulas (computational models). The machine learning method is appropriately selected depending on the machine learning model employed, and examples thereof include, but are not limited to, backpropagation.

[0233] Furthermore, the reference color data in the above-mentioned embodiment 1 and the modified example of embodiment 1 is not limited to using measured data as long as it is based on the color of the user's teeth, and may be, for example, the color indicated by a shade guide.

[0234] Furthermore, the periodontal disease detection unit 55 according to the modification of the second embodiment may, for example, determine which of the detection results of the machine learning model and the detection results of the judgment rule to adopt for each intraoral region, or may determine which of the detection results of the machine learning model and the detection results of the judgment rule to adopt for all intraoral regions. The intraoral regions may include, but are not limited to, two or more of the first region R1 to the fourth region R4 described above, and may be set arbitrarily.

[0235] Furthermore, the rule selection unit 152 according to the modified example of the second embodiment may select only two or more machine learning models, i.e., may not select a judgment rule, or may select only two or more judgment rules, i.e., may not select a machine learning model.

[0236] Furthermore, the mobile terminal 50 in the above-described first embodiment and the modified example of the first embodiment may include at least an output unit 57, an acquisition unit 51, and a processing unit. The processing unit performs predetermined processing, such as storing the detection result in a memory unit 58, displaying the detection result on a display unit 56, and transmitting the detection result to another device. The periodontal disease detection method (information processing method) executed by such a mobile terminal 50 is a periodontal disease detection method executed by one or more processors, and includes irradiating light into the user's oral cavity, outputting a first RGB image obtained by capturing an image of a photographed area including a specific tooth and the periodontal region of the specific tooth including the gums, obtaining a detection result of periodontal disease of the user based on the first RGB image, and performing predetermined processing on the obtained detection result, wherein the detection result may include the detection result of periodontal disease of the user detected based on image data based on a second RGB image adjusted based on the reference color data corresponding to the user, the gains of at least two of the red, green, and blue color components constituting the natural tooth area in the first RGB image being reference color data indicating a reference color of the natural tooth included in the photographed area. In this case, the functions of the tooth type identification unit 52, the image processing unit 53, the image data extraction unit 54, and the periodontal disease detection unit 55 are executed by a device external to the mobile terminal, such as a server device.

[0237] Furthermore, the mobile terminal 50a in the above-described second embodiment and the modified example of the second embodiment may include at least an output unit 57, an acquisition unit 51, and a processing unit. The processing unit performs predetermined processing, such as storing the detection results in a memory unit 58, displaying the detection results on a display unit 56, and transmitting the detection results to another device. A periodontal disease detection method (information processing method) executed by such a mobile terminal 50a is a periodontal disease detection method executed by one or more processors, and includes outputting a first image of a specific tooth in the user's oral cavity and the periodontal region of the specific tooth including the gums, obtaining a detection result of the user's periodontal disease based on the first image, and performing predetermined processing on the obtained detection result, wherein the detection result may include the detection result of the user's periodontal disease detected based on image data based on the second image, and a periodontal disease detection tool selected from a plurality of periodontal disease detection tools corresponding to each of a plurality of regions in the oral cavity, each of which is trained to input image data including the region and output information about periodontal disease in the region, depending on which of the plurality of regions defined by dividing the tooth row a second image including the periodontal region of the specific tooth generated from the first image is an image of. In this case, the functions of the image processing unit 53, area detection unit 151, rule selection unit 152, image data extraction unit 54, and periodontal disease detection unit 55 are executed by a device external to the mobile terminal, such as a server device.

[0238] Furthermore, each processing unit included in the periodontal disease detection system according to the first and second embodiments is typically realized as an LSI, which is an integrated circuit. These may be individually implemented as single chips, or some or all of them may be integrated into a single chip.

[0239] Furthermore, the integrated circuit is not limited to an LSI, but may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells within the LSI may also be used.

[0240] Furthermore, in the first and second embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0241] The division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to another functional block.Furthermore, the functions of multiple functional blocks having similar functions may be processed in parallel or in time-sharing by a single piece of hardware or software.

[0242] Furthermore, the periodontal disease detection systems (e.g., mobile terminals 50, 50a) according to the first and second embodiments may be realized as a single device or may be realized by multiple devices. When the periodontal disease detection system is realized by multiple devices, the components of the periodontal disease detection system may be distributed among the multiple devices in any manner. For example, at least some of the functions of the periodontal disease detection system may be realized by the intraoral camera 10 (e.g., the signal processing unit 30). When the periodontal disease detection system is realized by multiple devices, the communication method between the multiple devices is not particularly limited and may be wireless communication or wired communication. Furthermore, wireless communication and wired communication may be combined between the devices.

[0243] The present disclosure may also be realized as a periodontal disease detection method executed by a periodontal disease detection system, or as an intraoral camera, a mobile terminal, or a cloud server included in the periodontal disease detection system.

[0244] The order in which the steps are executed in the sequence diagram is merely an example for specifically explaining the present disclosure, and any order other than the above may be used. Also, some of the steps may be executed simultaneously (in parallel) with other steps.

[0245] Furthermore, one aspect of the present disclosure may be a computer program that causes a computer to execute each of the characteristic steps included in the periodontal disease detection method shown in any of FIG. 4, FIG. 14, and FIG.

[0246] Furthermore, for example, the program may be a program to be executed by a computer. Another aspect of the present disclosure may be a computer-readable non-transitory recording medium on which such a program is recorded. For example, such a program may be recorded on a recording medium and distributed or circulated. For example, the distributed program may be installed in a device having another processor, and the program may be executed by the processor, thereby causing the device to perform each of the above processes.

[0247] The periodontal disease detection system and the like according to one or more aspects have been described above based on Embodiments 1 and 2, but the present disclosure is not limited to Embodiments 1 and 2. As long as they do not deviate from the spirit of the present disclosure, various modifications that a person skilled in the art may make to Embodiments 1 and 2 and their modifications, or forms constructed by combining components of different embodiments, may also be included within the scope of one or more aspects.

[0248] The present disclosure is useful for a periodontal disease detection system for detecting periodontal disease, etc.

[0249] REFERENCE SIGNS LIST 10 Intraoral camera 10a Head unit 10b Handle unit 20 Hardware unit 21 Photography unit 22 Sensor unit 23 Illumination unit 23A First LED 23B Second LED 23C Third LED 23D Fourth LED 24 Operation unit 30 Signal processing unit 31 Camera control unit 32, 53 Image processing unit 33 Control unit 34 Illumination control unit 35 Memory unit 40 Communication unit 50, 50a Portable terminal (second information terminal) 51 Acquisition unit (first acquisition unit, second acquisition unit) 52 Tooth type identification unit 54 Image data extraction unit 55 Periodontal disease detection unit 56 Display unit 57 Output unit 58 Storage unit 151 Area detection unit 152 Rule selection unit (tool selection unit) F1, F2, F3 Rectangular frame D Predetermined distance r Radius R1 First region R2 Second region R3 Third region R4 Fourth region θ Angle

Claims

1. A periodontal disease detection system that detects periodontal disease in a user based on an image of a specified area in the oral cavity, comprising: an acquisition unit that acquires a first image of a specific tooth in the user's oral cavity, the specific tooth including the periodontal area of ​​the specific tooth, the specific tooth including the gums; an image processing unit that generates a second image from the first image, the second image including the periodontal area of ​​the specific tooth; a region detection unit that detects which of a plurality of areas in the oral cavity defined by dividing the tooth row the second image is an image of; a tool selection unit that selects a periodontal disease detection tool corresponding to the area detected by the region detection unit from a plurality of periodontal disease detection tools corresponding to each of the plurality of areas, each of which is generated so as to input image data including the area and output information regarding periodontal disease in that area; and a periodontal disease detection unit that detects the periodontal disease of the user based on the selected periodontal disease detection tool and image data based on the second image.

2. The periodontal disease detection system described in claim 1, wherein the plurality of periodontal disease detection tools include a plurality of learning models, each trained to input image data including the region and output information regarding periodontal disease in the region, and the periodontal disease detection unit detects the periodontal disease of the user by inputting image data based on the second image into a learning model selected from the plurality of learning models.

3. The periodontal disease detection system described in claim 1, wherein the plurality of periodontal disease detection tools each include a plurality of judgment rules for inputting an image of the gums between adjacent teeth for a tooth in image data including the region and outputting information regarding periodontal disease in the region, and the periodontal disease detection unit detects the periodontal disease of the user by inputting image data based on the second image into a judgment rule selected from the plurality of judgment rules.

4. The periodontal disease detection system described in claim 1, wherein the plurality of periodontal disease detection tools include a plurality of learning models, each trained to receive image data including the region as input and output information related to periodontal disease in the region, and a plurality of judgment rules, each trained to receive an image of the gums between adjacent teeth for a tooth in the image data including the region as input and output information related to periodontal disease in the region; the tool selection unit selects one or more learning models from the plurality of learning models and selects one or more judgment rules from the plurality of judgment rules; and the periodontal disease detection unit detects the periodontal disease of the user based on a first detection result of the one or more learning models and a second detection result of the one or more judgment rules.

5. The periodontal disease detection system of claim 4, wherein the periodontal disease detection unit compares the first detection result with the second detection result, and detects the user's periodontal disease using the detection result that determines that the symptoms of periodontal disease are worse.

6. A periodontal disease detection system according to any one of claims 1 to 5, wherein each of the plurality of regions is a region that includes a group of teeth that are similar in shape.

7. The periodontal disease detection system according to claim 6, wherein the plurality of regions include a front teeth buccal region, a front teeth lingual region, a molar buccal region, and a molar lingual region.

8. A periodontal disease detection system according to any one of claims 1 to 5, further comprising an image data extraction unit that extracts the periodontal region from the tip of the interdental papilla gingiva of the specific tooth to the free gingival sulcus from the second image as the image data based on the second image.

9. A periodontal disease detection system as described in claim 8, wherein the image data extraction unit extracts from the second image a rectangular frame area that circumscribes the left and right sides of the contour of the specific tooth and includes from the tip of the specific tooth to the free gingival sulcus as the image data based on the second image.

10. A periodontal disease detection system as described in claim 3 or 4, further comprising an image data extraction unit that extracts from the second image an area including at least a portion of the specific tooth and at least one of the left and right interdental papilla gingiva of the specific tooth as the image data based on the second image.

11. The periodontal disease detection system according to any one of claims 1 to 5, wherein each of the plurality of periodontal disease detection tools outputs an estimated result of at least one of periodontal pocket depth, BOP (Bleeding On Probing), and GI (Gingival Index) value as information relating to the periodontal disease.

12. A periodontal disease detection system according to any one of claims 1 to 5, further comprising an output unit that outputs information indicating the periodontal disease detected by the periodontal disease detection unit to the user's information terminal.

13. A periodontal disease detection system as claimed in any one of claims 1 to 5, further comprising an output unit that outputs information indicating the periodontal disease detected by the periodontal disease detection unit to a first information terminal of a dentist other than the user, and the output unit further outputs examination necessity information regarding the need for the user to undergo examination, which information was acquired via the acquisition unit and input to the first information terminal, to a second information terminal of the user.

14. A periodontal disease detection method executed by a periodontal disease detection system that detects periodontal disease in a user based on an image of a specified area in the oral cavity, comprising: obtaining a first image of a specific tooth in the user's oral cavity, including the periodontal area of ​​the specific tooth, and the specific tooth including the gums; generating a second image from the first image, including the periodontal area of ​​the specific tooth; detecting which of a plurality of areas in the oral cavity defined by dividing the tooth row the second image is an image of; selecting a periodontal disease detection tool corresponding to the detected area from a plurality of periodontal disease detection tools corresponding to each of the plurality of areas, each of which is generated so as to input image data including the area and output information regarding periodontal disease in that area; and detecting the periodontal disease of the user based on the selected periodontal disease detection tool and image data based on the second image.

15. A periodontal disease detection method executed by one or more processors, wherein the one or more processors: output a first image capturing a specific tooth in a user's oral cavity and the periodontal region of the specific tooth including the gums; obtain a detection result of the user's periodontal disease based on the first image; and perform predetermined processing on the obtained detection result, wherein the obtained detection result includes a detection result of the user's periodontal disease detected based on image data based on the second image, and a periodontal disease detection tool selected from a plurality of periodontal disease detection tools corresponding to each of a plurality of regions in the oral cavity, each of which is generated to receive image data including the region as input and output information regarding periodontal disease in the region, depending on which of a plurality of regions defined by dividing the tooth row a second image including the periodontal region of the specific tooth generated from the first image represents.

16. A program for causing a computer to execute the periodontal disease detection method according to claim 14 or 15.

Images