Oral cavity condition evaluation support system

Abstract

To provide an oral cavity condition evaluation support system that obtains a more appropriate evaluation result for the condition of an oral cavity.SOLUTION: In an oral cavity condition evaluation support system, an evaluation device 5 comprises: an operation receiving unit 22 corresponding to an oral cavity image acquisition unit that acquires an oral cavity image of a subject; a storage area 24 corresponding to an oral cavity finding information storage unit that stores oral cavity finding information of the subject; and a control unit 30 corresponding to an evaluation unit that, on the basis of the oral cavity image acquired by the oral cavity image acquisition unit and the oral cavity finding information, generates information on an evaluation result related to the oral cavity condition of the subject, by using an oral cavity condition evaluation model obtained by performing machine learning on the basis of the oral cavity condition of the subject and an evaluation result thereof.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a system for assisting in the evaluation of the condition of the oral cavity. [Background technology]

[0002] In order to provide oral care, etc., an assessment of the oral condition is first carried out (for example, Patent Document 1). In Patent Document 1, an assessment is first carried out on a person requiring oral care to create assessment information, and then an oral care assessment, an oral care plan, and a denture creation plan are created based on this assessment information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2001-167215 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present invention is to provide a novel technique that can obtain more appropriate evaluation results for the condition of the oral cavity. [Means for solving the problem]

[0005] The above-mentioned oral cavity evaluations are performed manually, which tends to result in variations in the evaluation results, and it is not always possible to obtain appropriate evaluation results.

[0006] The inventor discovered that more appropriate evaluation results can be obtained by using an oral condition evaluation model (neural network) obtained by machine learning based on the oral condition and its evaluation results, and generating information about the evaluation results based on images of the subject's oral cavity and sensor information, which is information about the subject's oral cavity obtained by a sensor.

[0007] The gist of the present invention is as follows. [1] an oral cavity image acquisition unit for acquiring an oral cavity image of a subject; an oral cavity finding information storage unit that stores oral cavity finding information of the subject; and an evaluation unit that generates information on the evaluation results regarding the subject's oral condition using an oral condition evaluation model obtained by machine learning based on the oral condition and its evaluation results, based on the oral images acquired by the oral image acquisition unit and the oral finding information. [2] The device further includes an odor information acquisition unit that acquires odor information of the oral cavity of the subject obtained by the sensor, the oral cavity finding information storage unit stores the odor information as the oral cavity finding information, The oral condition evaluation support system described in [1], wherein the oral condition evaluation model is obtained by machine learning based on oral images and their evaluation results, and machine learning based on oral odor and its evaluation results. [3] a denture image acquisition unit for acquiring a denture image obtained by imaging the target denture, the oral cavity finding information storage unit stores information about a state of a denture based on the denture image as the oral cavity finding information; The oral condition evaluation support system described in [1], wherein the oral condition evaluation model is obtained by further machine learning based on the denture condition and its evaluation results. [4] a medical interview information acquisition unit that acquires medical interview information related to a medical interview result of the subject; the oral cavity finding information storage unit stores the interview information as the oral cavity finding information, The oral condition evaluation support system described in [1], wherein the oral condition evaluation model is obtained by further machine learning based on the interview results and the evaluation results. [5] The apparatus further includes a denture image acquisition unit that acquires an image obtained by imaging the denture of the subject, and a medical interview information acquisition unit that acquires medical interview information regarding the result of the medical interview of the subject, the oral cavity finding information storage unit stores, as the oral cavity finding information, information on the state of the dentures based on the denture image and the medical interview information; The oral condition evaluation support system described in [1], wherein the oral condition evaluation model is obtained by machine learning based on the denture condition and its evaluation results, and further machine learning based on the interview results and its evaluation results. [6] An oral condition evaluation support system described in any one of [1] to [5], further comprising an auxiliary information storage unit that stores physical information about the subject's body and / or insurance information about the subject's insurance. [7] The oral condition evaluation support system described in [6] further comprises an information output unit that outputs information about the evaluation results generated by the evaluation unit in association with the physical information and / or the insurance information. [8] The oral condition evaluation support system described in [7] further comprises an evaluation result registration unit that associates information about the evaluation results generated by the evaluation unit with the physical information and / or the insurance information and stores it in the memory unit. The present invention also encompasses the following methods. In other words, the present invention encompasses an information processing method executed by a computer, which includes acquiring an image of the oral cavity of a subject, storing oral finding information of the subject, and generating information about the evaluation results of the oral condition of the subject using an oral condition evaluation model obtained by machine learning based on the oral condition and the evaluation results, based on the oral image and the oral finding information. The present invention also includes the following programs. That is, the present invention provides a computer including: an oral cavity image acquisition unit that acquires an oral cavity image of a target; The system includes an oral observation information storage unit that stores the oral observation information of the subject, and a program for causing the system to function as an evaluation unit that generates information on the evaluation results regarding the oral condition of the subject using an oral condition evaluation model obtained by machine learning based on the oral condition and its evaluation results, based on the oral images acquired by the oral image acquisition unit and the oral observation information. [Effects of the Invention]

[0008] According to the present invention, a novel technique can be provided that can obtain more appropriate evaluation results for the state of the oral cavity. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram showing an outline of the device configuration of an oral cavity condition evaluation support system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram showing an overview of an evaluation device included in the oral cavity condition evaluation support system of this embodiment. [Figure 3] FIG. 2 is a block diagram of an evaluation device included in the oral cavity condition evaluation support system of the present embodiment. [Figure 4] FIG. 10 is a diagram showing a procedure for generating a determination algorithm for an intraoral image of the intraoral state evaluation model according to the present embodiment. [Figure 5] 1 is a table showing the relationship between the evaluation items and the evaluation values ​​(scores) of the Oral Health Assessment Tool (OHAT). [Figure 6] FIG. 10 is a diagram showing a procedure for generating a determination algorithm related to odor information of the oral cavity condition evaluation model according to the present embodiment. [Figure 7] FIG. 10 is a diagram showing a procedure for generating a determination algorithm related to medical interview information of the oral cavity condition evaluation model according to the present embodiment. [Figure 8] FIG. 10 is a diagram showing a processing flow relating to evaluation value generation in this embodiment. [Figure 9] FIG. 10 is a diagram showing a processing flow relating to evaluation value registration in this embodiment. [Figure 10] 10 is a table showing the relationship between evaluation items and evaluation values ​​according to another embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] An embodiment of the present invention will be described in detail below. This embodiment relates to an oral cavity condition evaluation support system, which includes an oral cavity image acquisition unit, a sensor information acquisition unit, and an evaluation unit. The oral cavity image acquisition unit acquires an image of the oral cavity of a subject. The sensor information acquisition unit acquires sensor information about the oral cavity of the subject obtained by the sensor. The evaluation unit generates information about the evaluation results regarding the oral condition of the subject using an oral condition evaluation model obtained by machine learning based on the oral condition and its evaluation results, based on the oral image acquired by the oral image acquisition unit and the sensor information acquired by the sensor information acquisition unit. In this specification, an oral cavity image refers to an intraoral image, or an intraoral image and an image of the oral cavity periphery. In the following description of this embodiment, an example will be given in which an intraoral image and an image of the oral cavity periphery are used as the oral cavity image. In this embodiment, the sensor information acquisition unit acquires odor information about the odor of the subject's oral cavity as sensor information (in this embodiment, the sensor information acquisition unit corresponds to the odor information acquisition unit). The sensor information acquisition unit may also acquire information obtained by other sensors as sensor information. For example, the sensor information may include information about swallowing sounds obtained by a sound sensor, clenching strength obtained by a pressure sensor, and humidity in the oral cavity obtained by a humidity sensor. In the following description of this embodiment, an example will be given in which an evaluation value (score) is generated as information about the evaluation result. In addition, in this embodiment, the oral cavity finding information includes the subject's medical history information, the subject's oral cavity odor, swallowing sounds, bite pressure, oral humidity, and evaluations related to dentures.

[0011] FIG. 1 is a diagram showing an outline of the device configuration of an oral cavity condition evaluation support system 100 according to this embodiment. The oral cavity condition evaluation support system 100 includes a terminal device 1, an odor information acquisition device 3, an evaluation device 5, a prescription device 7, and databases DB1 and DB2, all of which are capable of communicating with each other via a network NW. The evaluation device 5 and the prescription device 7 may be, for example, a personal computer. The terminal device 1 is a device capable of capturing an image of the oral cavity of a subject, and is, for example, a portable information terminal such as a smartphone or a tablet PC. The odor information acquisition device 3 can use various information detection sensors capable of acquiring odor information. Examples of such sensors include breath sensors, and commercially available breath sensors capable of communicating via a network (NW) may be used. When using a breath sensor, a sensor device capable of detecting specific gas components, such as ammonia and acetone, contained in breath can be used. The amount of specific gas components, such as ammonia and acetone, that cause bad breath is correlated with human diseases and declining health. Examples of such correlates include periodontal disease, tooth decay, dental plaque, tartar, tongue coating, decreased saliva, oral cancer, nose and throat diseases, respiratory diseases, digestive diseases, and stress. Other correlates include a sweet odor associated with infectious diseases, acetone associated with diabetes, ammonia and protein gangrene odor associated with certain tumors and liver disease, isoprene associated with hypoglycemia and sleep disorders, methylmercaptan associated with oral bacteria and liver disease, carbon monoxide associated with stress, ethanol associated with alcohol use, trimethylamine associated with kidney disease, and nitric oxide associated with asthma. Therefore, by detecting the gas components that cause these odors in the breath, it is possible to infer the disease. In other words, the gas component data measured by the breath sensor and the disease information inferred from it are used as biological detection information. Furthermore, the database DB1 and the database DB2 can be of a cloud type, an on-premise type, or the like, as appropriate.

[0012] 2 is a schematic diagram of the evaluation device 5 of this embodiment. The prescription device 7 can also be configured to have the same configuration as the evaluation device 5. The evaluation device 5 has a processor 11 which is an arithmetic processing device, a memory 12 (such as RAM (Random Access Memory)) which is a main storage device, and an SSD (Solid State Drive) 13 which is an auxiliary storage device. The oral condition evaluation support system 100 also has a network IF (interface) 14 which controls communication with external units, a monitor 15, an input device 16 (such as a keyboard or mouse), and a media reading device 17. Programs for realizing aspects of the embodiment are stored in advance in the SSD 13. These programs can be installed by reading external installation media (such as a CD-ROM or DVD) with the media reading device 17 and storing the programs in the SSD 13, or by downloading the programs via the network IF 14 and storing them in the SSD 13. Furthermore, although the oral cavity condition evaluation support system 100 of this embodiment is configured as a standalone device such as a PC (Personal Computer) that includes each component within the device, one or more of the components may be located outside the device and connected via a network NW. For example, the programs that can be stored in the SSD 13 may also be stored on the cloud.

[0013] FIG. 3 is a block diagram of the evaluation device 5 of this embodiment. 3, the evaluation device 5 is composed of a display unit 20 that displays various information, an operation reception unit 22 that receives operations from a user such as a doctor or nurse, a memory area 24 that stores various information, a communication unit 26 connected to the network NW, and a control unit 30 that controls the operations of these components. The memory area 24 includes an oral observation information storage unit 25 that stores oral observation information of the subject.

[0014] Specifically, the display unit 20 is configured by, for example, a monitor, and has the function of displaying various information in response to a control signal from the control unit 30. In addition, the operation reception unit 22 is composed of an input device 16, such as a keyboard, a mouse, or a touch panel that also serves as the display unit 20, and has the function of receiving input operations by the user and the function of outputting information indicating the received input content to the control unit 30.

[0015] The storage area 24 is composed of the memory 12, the SSD 13, etc., and has the functions of writing and storing various information and reading out various information. The storage area 24 stores a program for implementing an oral condition evaluation model, a program for outputting physical information, a program for registering evaluation values, generated evaluation values ​​for each subject, and physical information for each subject. The oral findings information storage unit 25 stores odor information obtained by the odor information acquisition device 3, interview information related to the results of an interview conducted by a medical professional or the like to evaluate the oral condition of a subject, and information about the condition of the dentures based on denture images. The information about the condition of the dentures may be image data of the dentures, or may be information about the condition of the dentures obtained from the image data of the dentures. The odor information, interview information, and information about the condition of the dentures may be used for machine learning, as described below. Furthermore, the oral cavity finding information storage unit 25 may include information obtained by other sensors in addition to or instead of the smell information. Specifically, examples of such information include information about swallowing sounds obtained by the above-mentioned audio sensor, information about clenching strength obtained by a pressure sensor, and information about humidity in the oral cavity obtained by a humidity sensor.

[0016] The communication unit 26 is configured by the network IF 14 . The control unit 30 is composed of hardware such as a processor 11 and a memory 12, and software such as a control program.

[0017] This control unit 30 has processing functions related to the exchange of various signals with the display unit 20, operation reception unit 22, memory area 24, communication unit 26, etc., and the function of controlling the operation of each unit connected via a specified bus.

[0018] In this embodiment, the control unit 30 and the operation accepting unit 22 correspond to an oral cavity image acquiring unit, the control unit 30 and the operation accepting unit 22 correspond to a sensor information acquiring unit, the control unit 30 corresponds to an evaluation unit, the memory area 24 corresponds to an oral cavity finding information storing unit, the memory area 24 corresponds to an additional information storing unit, the control unit 30 and the display unit 20 correspond to an information output unit, the control unit 30 and the operation accepting unit 22 correspond to a denture image acquiring unit, the control unit 30 and the operation accepting unit 22 correspond to a medical interview information acquiring unit, and the control unit 30 corresponds to an evaluation result registering unit. The additional information storing unit can also be configured to include databases DB1 and DB2 connected via the communication unit 26 in addition to the memory area 24.

[0019] As described above, in this embodiment, an oral cavity condition evaluation model obtained by machine learning based on the oral cavity condition and its evaluation results is used to generate an evaluation value.

[0020] The determination algorithm of the oral cavity condition evaluation model may be generated, for example, according to the procedures shown in FIGS. The evaluation item index can be generated using the oral cavity image and the denture image, for example, as follows. For example, as shown in Fig. 4, first, a medical institution collects images of a patient's oral cavity and the area around the oral cavity, along with associated correct labels indicating a deterioration in oral condition (step S001). Correct labels are assigned based on the results of, for example, dental examinations, periodontal disease tests, saliva tests, bacterial tests, PCR tests, and virus isolation and culture tests that the patient has undergone, as well as the findings of a doctor or dentist. Although PCR tests and virus isolation and culture tests take several weeks to produce results, they are highly accurate and therefore suitable as correct labels.

[0021] Furthermore, the correct answer label used in this learning may include, for example, image data of a person's oral cavity and data representing an evaluation value of the state of the oral cavity indicated by the image data. The evaluation value can be assigned according to a predetermined evaluation standard. For example, the Oral Health Assessment Tool (OHAT) can be used as the evaluation criterion. As shown in Figure 5, the OHAT consists of eight evaluation items: lips, sublingual, gums and mucous membranes, saliva, remaining teeth, dentures (if the subject has them), oral hygiene, and toothache. The OHAT evaluation is based on three levels, each of which is assigned a rating of "0," "1," or "2," with the latter level corresponding to a worsening condition. For example, each image data item is assigned a rating of "0," "1," or "2."

[0022] In step S002, training data (image data) is generated in which the above test results are labeled as correct answers. Machine learning is performed based on this training data to generate image-based judgment algorithms A1, A2, A3, A4, A5, A6, A7, and A8 (step S003). For example, a convolutional neural network (CNN) can be used for the machine learning. Alternatively, a method using a neural network such as a multi-layer neural network (MLP), a long short-term memory (LSTM), a gated recurrent unit (GRU), a graph neural network (GNN), or a transformer may be used for the machine learning. The above-mentioned determination algorithms A1 to A8 are algorithms for determining the degree of deterioration of the oral cavity condition for each evaluation item of the OHAT when an image is given. This makes it possible to quantify the degree of deterioration of oral condition for each OHAT evaluation item and display the image-based OHAT evaluation item indices I1, I2, I3, I4, I5, I6, I7, and I8. The control unit 30 stores the image-based OHAT evaluation item indices I1, I2, I3, I4, I5, I6, I7, and I8 in the memory area 24. Instead of outputting the evaluation item indices I1 to I8 using the above-mentioned determination algorithms A1 to A8, the evaluation item indices I1 to I8 may be output using a single determination model (multi-class classification model). In addition to this determination, the reliability of the determination may be quantified, or the degree of certainty that the disease is present may be indicated by a graded evaluation such as high, medium, or low.

[0023] Alternatively, a positive or negative judgment may be made, for example, as "Judgment: very poor oral condition," "Judgment: poor oral condition," "Judgment: normal oral condition," "Judgment: good oral condition," or "Judgment: very good oral condition." In addition to or in addition to this judgment, the reliability of the judgment may be quantified or indicated by a scale such as high, medium, or low. Alternatively, the degree of certainty that the oral condition is a disease may be indicated by a scale such as high, medium, or low. The images used to generate the training data may be either still images or moving images. Furthermore, the training data may be generated using images captured at multiple positions or multiple directions, or may be images captured at the same position or direction. For example, multiple types of images capturing one side and the other side of the oral cavity may be used to generate the training data. Furthermore, for the denture image, for example, an image of only the denture taken in a state where it has been removed from the oral cavity can be used. Furthermore, when evaluating dentures, an AI algorithm for detecting abnormalities can be used as an oral condition evaluation model. For example, an Auto Encoder that has been trained on various images of normal dentures can be used to detect denture abnormalities. Because teeth have a fixed shape, deviations from this (such as chips or cracks) can be detected as abnormalities. In addition to the above-mentioned Auto Encoder, other anomaly detection methods include methods that measure the degree of anomaly based on the distance from the distribution of features obtained from normal images, such as SPADE, Padim, and PatchCore; methods that use restoration errors using dictionaries obtained from normal images, such as Sparse Modeling; and Efficient-GAN, a method of anomaly detection based on restoration errors using GANs (Generative Adversarial Networks). For example, any of these may be used.

[0024] In addition, in this embodiment, an evaluation item index for oral odor is also generated.

[0025] In this case, for example, as shown in Fig. 6, odor information obtained using an odor information acquisition device 3 at a medical institution and associated correct labels indicating a deterioration in oral condition for each item of the OHAT are collected (step S'001). Correct labels are assigned based on, for example, the results of dental examinations, periodontal disease tests, saliva tests, bacterial tests, PCR tests, and virus isolation and culture tests that the patient has undergone, as well as the findings of a doctor or dentist. Although PCR tests and virus isolation and culture tests take several weeks to produce results, they are highly accurate and therefore suitable as correct labels. In step S'002, training data (smell data) is generated in which the test results are labeled as correct answers. Machine learning is performed based on this training data to generate an odor-based determination algorithm B7 (step S'003). Machine learning of odor information can be performed using neural networks or gradient boosting methods. Alternatively, machine learning can be performed using gradient boosting decision trees (GBDTs) such as LightGBM (Light Gradient Boosting Machine), XGBoost, or CatBoost. In response to input of actual patient odor information, the determination algorithm B7 outputs an odor-based OHAT evaluation item index I'7 for the OHAT evaluation item "oral cleanliness." The control unit 30 stores the odor-based OHAT evaluation item index I'7 in the memory area 24. In addition to this determination, the reliability of the determination may be quantified, or the degree of certainty that the disease is present may be indicated by a graded evaluation such as high, medium, or low.

[0026] Alternatively, a positive or negative judgment may be made, for example, as "Judgment: very poor oral condition," "Judgment: poor oral condition," "Judgment: normal oral condition," "Judgment: good oral condition," or "Judgment: very good oral condition." In addition to or in addition to this judgment, the reliability of the judgment may be quantified or indicated by a scale such as high, medium, or low. Alternatively, the degree of certainty that the oral condition is a disease may be indicated by a scale such as high, medium, or low.

[0027] In addition, in this embodiment, evaluation item indices are also generated for medical interviews obtained from medical professionals, including doctors and dentists.

[0028] In this case, for example, as shown in FIG. 7, medical interview information and the associated correct labels indicating the deterioration of the oral condition for each item of the OHAT are collected (step S''001). The correct labels are assigned based on, for example, medical interview information obtained by checking the oral cavity of the patient by a medical professional or the like, findings information, and patient attribute information obtained at a medical institution. Correct labels may be assigned to one patient by multiple medical professionals or the like to collect the labels.

[0029] The method for collecting medical interview information will be explained using an example. For example, regarding toothache, a healthcare professional or other such person may ask a patient, care recipient, or other subject, "Does your tooth hurt?" to confirm the presence and location of toothache. If the healthcare professional or other such person touches the subject's teeth and the subject shows no particular signs, the score is determined to be 0, indicating no verbal or physical signs of pain. On the other hand, if the healthcare professional or other such person touches the subject's teeth and the subject shows verbal or behavioral signs such as a facial twitch, lip biting, or aggressive behavior, or if the subject shows signs such as not eating in daily life at a nursing home or day care center, the score is determined to be 1 (change). Furthermore, if the subject shows swelling of the cheeks or gums, broken teeth, ulcers, subgingival abscess, etc. in addition to verbal and behavioral signs, the score is determined to be 2 (pathological). Note that swelling of the cheeks or gums, broken teeth, ulcers, subgingival abscess, etc. may also be collected as interview information from image data. Regarding saliva, the medical professional will instruct the subject to "open their mouth slightly" to check the condition of the saliva in the oral cavity. If it is found to be moist and serous, a score of 0 will be given. On the other hand, if there is a small amount of saliva and the mucous membrane is sticky or the saliva is foamy, a score of 1 (change) will be given. If the saliva is dried up or the care recipient claims to have a very dry mouth, a score of 2 (pathological) will be given. Note that the stickiness of the mucous membrane can also be measured using a viscometer, etc.

[0030] Regarding the lips, the healthcare professional will carefully examine the subject's lips. The healthcare professional will instruct the subject to gently open the corners of their mouth, saying, "Please open your mouth slightly." If the lips are judged to be normal, moist, and pink, the score will be 0. If the corners of the mouth are dry or cracked, the score will be 1 (changed). If ulcerative lesions or bleeding are found, the score will be 2 (pathological). For the tongue, a healthcare professional or other professional instructs the subject to "open their mouth and stick out their tongue." They carefully compare the visible condition of the tongue with a reference photograph displaying the tongue condition and score, and determine the score for the "tongue" item. If necessary, the tongue may be touched. If the result is a normal, moist, pink appearance, it is scored as 0. If tongue coating is observed, regardless of the amount, nature, or color, it is scored as 1 (change). If ulcerative lesions or associated bleeding are observed, it is scored as 2 (pathological).

[0031] Next, training data is generated in which the above test results are labeled as correct answers (step S''002). Machine learning is performed based on this training data to generate interview-based determination algorithms C1, C2, C3, C4, C5, C6, C7, and C8 (step S''003). Neural networks and gradient boosting methods can be used for machine learning of medical interview information. Furthermore, methods using gradient boosting decision trees (GBDTs), such as LightGBM (Light Gradient Boosting Machine), XGBoost, and CatBoost, can also be used for machine learning. In addition to this determination, the reliability of the determination may be quantified, or the degree of certainty that the disease is present may be indicated by a graded evaluation such as high, medium, or low.

[0032] Alternatively, a positive or negative judgment may be made, for example, as "Judgment: very poor oral condition," "Judgment: poor oral condition," "Judgment: normal oral condition," "Judgment: good oral condition," or "Judgment: very good oral condition." In addition to or in addition to this judgment, the reliability of the judgment may be quantified or indicated by a scale such as high, medium, or low. Alternatively, the degree of certainty that the oral condition is a disease may be indicated by a scale such as high, medium, or low.

[0033] The judgment algorithms C1, C2, C3, C4, C5, C6, C7, and C8 output interview-based OHAT evaluation item indices I''1, I''2, I''3, I''4, I''5, I''6, I''7, and I''8 for each OHAT evaluation item in response to input of interview information, findings information, and attribute information regarding the lips, tongue, gums / mucosa, saliva, remaining teeth, dentures, oral hygiene, and toothache of actual medical professionals, etc. The control unit 30 stores the interview-based OHAT evaluation item indices I''1, I''2, I''3, I''4, I''5, I''6, I''7, and I''8 in the memory area 24. Note that instead of outputting the evaluation item indices I''1 to I''8 using the above-mentioned judgment algorithms C1 to C8, the evaluation item indices I''1 to I''8 may be output using a single judgment model (multi-class classification model). In addition to this determination, the reliability of the determination may be quantified, or the degree of certainty that the disease is present may be indicated by a graded evaluation such as high, medium, or low.

[0034] Furthermore, when the control unit 30 receives the image-based OHAT evaluation item indices I1, I2, I3, I4, I5, I6, I7, and I8, the odor-based OHAT evaluation item index I'7, and the interview-based OHAT evaluation item indices I''1, I''2, I''3, I''4, I''5, I''6, I''7, and I''8 as input, it may read each index from the memory area 24 and perform ensemble processing.As an example of this ensemble processing, the obtained image-based OHAT evaluation item indices, odor-based OHAT evaluation item indices, and interview-based OHAT evaluation item indices are input to a ridge regression model, and a final OHAT score is obtained in which each index is ensembled as an evaluation result for the oral condition. In addition to the ridge regression model described above, the ensemble method may also be, for example, a process for obtaining an average value, a process for obtaining a maximum value, a process for obtaining a minimum value, a weighted summation process, or a process using other machine learning methods such as bagging, boosting, stacking, lasso regression, or linear regression. The ensemble process is performed in multiple stages. The ensemble may combine all three types of judgment algorithms with different inputs (image-based judgment algorithm, odor-based judgment algorithm, and interview-based judgment algorithm), or any two of them. When combining multiple judgment algorithms, first, judgment algorithms with different inputs for each evaluation item are ensembled to create an algorithm for judging each evaluation item. Next, the algorithms that judge each ensembled evaluation item are further ensembled together to obtain an overall OHAT score that takes all evaluation items into consideration. By ensembling an image-based judgment algorithm corresponding to each item of the OHAT with an odor-based judgment algorithm corresponding to each item of the OHAT and / or a questionnaire-based judgment algorithm corresponding to each item of the OHAT, it is possible to make a more accurate judgment than if each item of the OHAT were judged by each judgment algorithm alone.

[0035] As described above, the terminal device 1 can be a mobile information terminal such as a smartphone or tablet PC, and includes a touch panel display and a camera. The touch panel display is a device that combines an input function for accepting user input by touching the screen with a finger or the like and an output function for displaying an image on the screen. The housing of the terminal device 200 is formed with a shooting window that guides light from a subject into the housing. The camera has a photoelectric conversion element that receives light that enters the housing from the shooting window through an optical system, and an A / D conversion circuit that converts an analog signal generated by the element through photoelectric conversion into a digital signal. The terminal device 200 generates image data representing the subject from the digital signal output by the camera.

[0036] Application software (hereinafter referred to as "app") dedicated to the oral cavity condition evaluation support system 100 is installed in the terminal device 1. When the app is started, the terminal device 200 captures an image of the oral cavity of the target in accordance with the started application. The terminal device 200 sends the captured oral cavity image data to the evaluation device 5.

[0037] When an exhalation sensor is used as the information detection sensor, the odor information acquisition device 3 detects and acquires odor information by inserting the exhalation sensor into the oral cavity. Alternatively, an exhalation sensor may be attached to a dental instrument such as a penlight that illuminates the oral cavity, a dental drill, a probe, or a sickle scaler, and odor information may be detected when the instrument is inserted into the oral cavity. The odor information acquisition device 3 sends the acquired odor information to the evaluation device 5.

[0038] Next, the processing flow in this embodiment will be described with reference to Figures 8 and 9. Figure 8 shows the processing flow relating to the generation of an evaluation value, which is information about evaluations, and Figure 9 shows the processing flow relating to the registration of an evaluation value. In the following explanation, an example will be given in which denture images are also used in addition to oral cavity images to generate evaluation values, but this is not limiting, and only oral cavity images may be used as images. Oral cavity images and denture images are collectively referred to as oral cavity images, etc. Similarly to the images used to generate the training data, the target images used to generate the evaluation value may be either still images or moving images. The evaluation value may be generated using images captured in multiple positions or directions, or may be images captured in the same position or direction. For example, multiple types of images capturing one side and the other side of the oral cavity may be used to generate the evaluation value. Furthermore, for the image of the dentures, for example, an image of the dentures taken in a state where they have been removed from the oral cavity can be used.

[0039] First, the processing flow for generating an evaluation value shown in FIG. 8 will be described. First, in step S101, when a user operation (instruction to generate an evaluation value) to execute a program related to evaluation value generation is acquired, the control unit 30 executes the program to enable acceptance of medical interview information, oral cavity images, etc., and odor information.

[0040] In step S102, the control unit 30 acquires the medical interview information input via the operation reception unit 22, and stores it in the oral cavity finding information storage unit 25 of the storage area .

[0041] In step S103, the control unit 30 acquires data such as oral cavity images sent from the terminal device 1 via the communication unit 26 and stores them in the memory area 24. The control unit 30 stores the denture images in the oral cavity finding information memory unit 25 of the memory area 24.

[0042] In step S104, the control unit 30 acquires the odor information sent from the odor information acquisition device 3 via the communication unit 26, and stores it in the oral cavity finding information storage unit 25 of the storage area 24.

[0043] Proceeding to step S105, the control unit 30 generates an evaluation value using an oral cavity condition evaluation model based on the oral cavity image, etc., the medical interview information, and the odor information. The evaluation values ​​generated may be values ​​for each evaluation item (lips, tongue, gums / mucosa, saliva, remaining teeth, dentures, oral hygiene, and toothache), or an overall value (for example, an average value) may be generated.

[0044] In step S106, the control unit 30 determines whether the physical information of the subject is included in the database DB1. Note that this determination can be made, for example, based on information for identifying the individual (such as name) included in the medical interview information.

[0045] If the physical information is stored in database DB1, control unit 30 acquires the physical information via communication unit 26 in step S107. In step S108, control unit 30 causes display unit 22 to display the obtained evaluation score and physical information.

[0046] On the other hand, if it is determined in step S106 that the physical information is not stored in database DB1, the process proceeds to step S109, where control unit 30 causes display unit 22 to display only the obtained evaluation value.

[0047] Next, the processing flow relating to evaluation value registration in FIG. 9 will be described. First, in step S201, when a user operation (registration instruction) to execute a program related to the registration of evaluation values ​​is acquired, the control unit 30 executes the program. In step S202, the control unit 30 determines whether physical information is stored in the database DB1. Note that this determination can be made based on information for identifying an individual (such as name) included in the medical interview information.

[0048] If it is determined that the physical information is not stored, the control unit 30 ends the process. In this case, the control unit 30 may cause the display unit 22 to display a message informing the user that the physical information is not stored.

[0049] On the other hand, if it is determined that the physical information is stored, the control unit 30 associates the evaluation value with the physical information and stores the association in the database DB1.

[0050] As described above, according to this embodiment, it is possible to obtain more appropriate evaluation results for the state of the oral cavity. The present invention is not limited to the above-described embodiment, and other aspects are of course possible. For example, in the above-described embodiment, an oral condition evaluation model is obtained by machine learning using oral cavity images, medical interview information, and odor information, and the control unit 30 generates an evaluation value, which is information about the evaluation result, using the oral cavity images, medical interview information, and odor information of the subject. However, without being limited to this, for example, an oral condition evaluation model may be constructed and information about the evaluation result may be generated using oral cavity images and odor information. In the above-described embodiment, odor-based evaluation item indices are output for oral hygiene in the OHAT. In another embodiment, odor-based evaluation item indices may be output for other evaluation items in addition to or instead of oral hygiene in the OHAT. For example, odor-based evaluation item indices may be output for one or more of the other evaluation items in the OHAT (lips, sublingual, gingiva / mucosa, saliva, remaining teeth, dentures, toothache).

[0051] Also, for example, in the above-described embodiment, it is determined whether physical information is stored in database DB1, and if the physical information is stored, information about the evaluation results and the physical information are displayed on display unit 22, and the information about the evaluation results and the physical information are associated and stored in database DB1. In another embodiment, it is possible to determine whether the target's insurance information is stored in database DB1 or database DB2 (hereinafter simply referred to as database DB1, etc.) instead of or together with the physical information, and if the target's insurance information is stored, display unit 22 correlates the information about the evaluation result with the insurance information, and store the information about the evaluation result correlated with the insurance information in database DB1, etc. In this embodiment, the auxiliary information storage unit has the above-mentioned configuration including database DB1, etc. In this specification, insurance information refers to information about a patient's enrollment in medical insurance (a system that reduces the burden of medical expenses) that applies to the costs of examinations and treatments for illnesses, injuries, etc. at medical institutions, and includes information indicating whether the patient has enrolled in medical insurance, the type of insurance, etc.

[0052] This embodiment or other embodiments may be used to assess each OHAT evaluation item based on subject images, etc., and generate, for example, notification documents based on the assessment results. Such a processing system generates an evaluation result of the oral condition based on, for example, subject images of the oral cavity captured by a photographing device. Then, based on information about the evaluation results, it generates information indicating whether the disease is eligible for insurance coverage, or notification documents required for various institutions such as the national government, local governments, schools, and workplaces. Therefore, users (e.g., medical professionals) who need to prepare such notification documents can more efficiently prepare notification documents. In other words, it is expected that documents related to the oral health coordination enhancement surcharge can be prepared more efficiently.

[0053] Furthermore, other evaluation items may be included in addition to the evaluation items of the OHAT. For example, it would be possible to measure the degree of mouth opening as one of the evaluation items. Specifically, the upper and lower teeth or lips are detected using object detection AI, and the distance between the upper and lower teeth is measured. The magnification of the image is calculated based on the size of the lips and teeth, and the degree of mouth opening is measured. Examples of object detection AI algorithms include YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), DETR (End-to-End Object Detection with Transformers), and segmentation models such as UNet and Mask R-CNN.

[0054] In addition, other evaluation items may be used instead of the evaluation items of the OHAT, and are not particularly limited. For example, an oral condition evaluation model for the evaluation items shown in Figure 10 may be generated, and information on the evaluation results regarding the oral condition of the subject may be generated using the oral condition evaluation model. Furthermore, evaluation items may include swallowing function, chewing function, tongue movement, lip movement, and the moist state of the oral cavity.

Claims

1. an oral cavity image acquisition unit for acquiring an oral cavity image of a subject; an oral cavity finding information storage unit that stores oral cavity finding information of the subject; and an evaluation unit that generates information on the evaluation results regarding the subject's oral condition using an oral condition evaluation model obtained by machine learning based on the oral condition and its evaluation results, based on the oral images acquired by the oral image acquisition unit and the oral finding information.

2. The device further includes an odor information acquisition unit that acquires odor information of the oral cavity of the subject obtained by the sensor, the oral cavity finding information storage unit stores the odor information as the oral cavity finding information, The oral condition evaluation support system of claim 1, wherein the oral condition evaluation model is obtained by machine learning based on oral images and their evaluation results, and machine learning based on oral odor and its evaluation results.

3. a denture image acquisition unit for acquiring a denture image obtained by imaging the target denture, the oral cavity finding information storage unit stores information about a state of a denture based on the denture image as the oral cavity finding information; 2. The oral cavity condition evaluation support system according to claim 1, wherein the oral cavity condition evaluation model is obtained by further performing machine learning based on the state of the dentures and the evaluation results thereof.

4. a medical interview information acquisition unit that acquires medical interview information related to a medical interview result of the subject; the oral cavity finding information storage unit stores the interview information as the oral cavity finding information, The oral cavity condition evaluation support system according to claim 1 , wherein the oral cavity condition evaluation model is obtained by further performing machine learning based on the interview results and the evaluation results.

5. The apparatus further includes a denture image acquisition unit that acquires an image obtained by imaging the denture of the subject, and a medical interview information acquisition unit that acquires medical interview information regarding the result of the medical interview of the subject, the oral cavity finding information storage unit stores, as the oral cavity finding information, information on the state of the dentures based on the denture image and the medical interview information; The oral condition evaluation support system of claim 1, wherein the oral condition evaluation model is obtained by further performing machine learning based on the denture condition and its evaluation results, and machine learning based on the interview results and its evaluation results.

6. The oral cavity condition evaluation support system according to claim 1 , further comprising an auxiliary information storage unit that stores physical information about the subject's body and / or insurance information about the subject's insurance.

7. The oral cavity condition evaluation support system according to claim 6 , further comprising an information output unit that outputs information about the evaluation result generated by the evaluation unit in association with the physical information and / or the insurance information.

8. The oral condition evaluation support system of claim 7, further comprising an evaluation result registration unit that associates information about the evaluation results generated by the evaluation unit with the physical information and / or the insurance information and stores it in the memory unit.

Images