Oral condition assessment support system

Abstract

To provide an oral cavity condition evaluation support system that obtains a more appropriate evaluation result for the condition of an oral cavity.SOLUTION: In an oral cavity condition evaluation support system, an evaluation device 5 comprises: an operation receiving unit 22 corresponding to an oral cavity image acquisition unit that acquires an oral cavity image of a subject; a storage area 24 corresponding to an oral cavity finding information storage unit that stores oral cavity finding information of the subject; and a control unit 30 corresponding to an evaluation unit that, on the basis of the oral cavity image acquired by the oral cavity image acquisition unit and the oral cavity finding information, generates information on an evaluation result related to the oral cavity condition of the subject, by using an oral cavity condition evaluation model obtained by performing machine learning on the basis of the oral cavity condition of the subject and an evaluation result thereof.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a system for assisting in the evaluation of oral conditions.

Background Art

[0002] In order to perform oral care etc., first, an evaluation of the oral condition is carried out (for example, Patent Document 1). In Patent Document 1, first, an assessment is performed on a person requiring oral care to create assessment information, and based on this assessment information, an oral care assessment, an oral care plan, and a denture fabrication plan are created.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present invention is to provide a novel technique capable of obtaining a more appropriate evaluation result regarding the oral condition.

Means for Solving the Problems

[0005] The above-described evaluation of the oral cavity is performed manually by a person. Therefore, variations tend to occur in the evaluation results, and an appropriate evaluation result cannot always be obtained.

[0006] The inventor has found that by using an oral condition evaluation model (neural network) obtained by machine learning based on the oral condition and its evaluation result, and generating information regarding the evaluation result based on the oral cavity image of the subject and sensor information which is information about the oral cavity of the subject acquired by a sensor, a more appropriate evaluation result can be obtained.

[0007] The gist of the present invention is as follows. [1] An oral image acquisition unit that acquires an oral image of the target, An oral cavity findings information storage unit that stores oral cavity findings information of the subject, An oral condition evaluation support system comprising: an evaluation unit that generates information about the evaluation results regarding the target oral condition using an oral condition evaluation model obtained by machine learning based on the oral condition and its evaluation results, based on the oral image acquired by the oral image acquisition unit and the oral findings information. [2] The system further includes an odor information acquisition unit that acquires odor information of the target oral cavity obtained by a sensor, The oral cavity information storage unit stores the odor information as the oral cavity information, The oral condition evaluation model is obtained by machine learning based on oral images and their evaluation results, and by machine learning based on oral odor and its evaluation results, as described in [1], for the oral condition evaluation support system. [3] The system further includes a denture image acquisition unit that acquires a denture image obtained by imaging the aforementioned denture, The oral cavity information storage unit stores information about the condition of the denture based on the denture image as the oral cavity information. The oral condition evaluation model is obtained by further machine learning based on the condition of the dentures and the evaluation results, as described in [1], and is an oral condition evaluation support system. [4] The system further includes a medical information acquisition unit that acquires medical information relating to the results of the aforementioned medical interviews, The oral cavity findings information storage unit stores the interview information as the oral cavity findings information, The oral condition assessment model is obtained by further machine learning based on the results of the medical interview and the assessment results, as described in [1], and is an oral condition assessment support system. [5] The system further comprises a denture image acquisition unit that acquires an image obtained by imaging the aforementioned denture, and a medical interview information acquisition unit that acquires medical interview information relating to the medical interview results of the aforementioned denture. The oral cavity findings information storage unit stores information about the condition of the dentures based on the denture images and the medical interview information as oral cavity findings information. The oral condition evaluation model is obtained by further machine learning based on the condition of dentures and their evaluation results, and by machine learning based on the results of the medical interview and their evaluation results, as described in [1], for the oral condition evaluation support system. [6] The oral condition evaluation support system according to any one of [1] to [5] further comprises an auxiliary information storage unit for storing physical information about the subject's body and / or insurance information about the subject's insurance. [7] The oral condition evaluation support system according to [6] further comprises an information output unit that outputs information about the evaluation results generated by the evaluation unit in association with the physical information and / or the insurance information. [8] The oral condition evaluation support system according to [7], further comprising an evaluation result registration unit that stores information about the evaluation results generated by the evaluation unit in the storage unit in association with the physical information and / or the insurance information. Furthermore, the present invention includes the following methods. In other words, the present invention includes a computer-based information processing method comprising acquiring an oral cavity image of a target, storing oral cavity findings information of the target, and generating information about the evaluation results regarding the oral cavity state of the target using an oral cavity state evaluation model obtained by machine learning based on the oral cavity state and its evaluation results, based on the oral cavity image and the oral cavity findings information. Furthermore, the present invention includes the following programs. In other words, the present invention provides a computer comprising an oral image acquisition unit for acquiring an oral image of a target, An oral finding information storage unit that stores the oral finding information of the subject, and a program for causing a computer to function as an evaluation unit that generates information about an evaluation result regarding the oral state of the subject by using an oral state evaluation model obtained by machine learning based on an oral image acquired by the oral image acquisition unit and the oral finding information.

Advantages of the Invention

[0008] According to the present invention, it is possible to provide a novel technique capable of obtaining a more appropriate evaluation result regarding an oral state.

Brief Description of the Drawings

[0009] [Figure 1] It is a diagram showing an outline of the device configuration of the oral state evaluation support system of the present embodiment. [Figure 2] It is a diagram showing an outline of the evaluation device included in the oral state evaluation support system of the present embodiment. [Figure 3] It is a block diagram of the evaluation device included in the oral state evaluation support system of the present embodiment. [Figure 4] It is a diagram showing a generation procedure of a determination algorithm related to an oral image of the oral state evaluation model according to the present embodiment. [Figure 5] It is a table showing the relationship between the evaluation items and evaluation values (scores) of OHAT (Oral Health Assessment Tool). [Figure 6] It is a diagram showing a generation procedure of a determination algorithm related to odor information of the oral state evaluation model according to the present embodiment. [Figure 7] It is a diagram showing a generation procedure of a determination algorithm related to questionnaire information of the oral state evaluation model according to the present embodiment. [Figure 8] It is a diagram showing a processing flow related to evaluation value generation of the present embodiment. [Figure 9] It is a diagram showing a processing flow related to evaluation value registration of the present embodiment. [Figure 10] It is a table showing the relationship between the evaluation items and evaluation values of other embodiments.

Best Mode for Carrying Out the Invention

[0010] Hereinafter, one embodiment of the present invention will be described in detail. This embodiment relates to an oral condition evaluation support system, and includes an oral image acquisition unit, a sensor information acquisition unit, and an evaluation unit. The oral image acquisition unit acquires an oral image of a target. The sensor information acquisition unit acquires sensor information about the oral cavity of the target obtained by a sensor. The evaluation unit uses an oral condition evaluation model obtained by machine learning based on the oral condition and its evaluation results, based on the oral image acquired by the oral image acquisition unit and the sensor information acquired by the sensor information acquisition unit, to generate information about the evaluation result regarding the oral condition of the target. In this specification, the oral image means an intraoral image, or an intraoral image and an image of the peripheral part of the oral cavity. In the following description of this embodiment, the case where an intraoral image and an image of the peripheral part of the oral cavity are used as the oral image will be taken as an example for explanation. Also, in this embodiment, the sensor information acquisition unit acquires odor information about the odor of the target oral cavity as sensor information (in this embodiment, the sensor information acquisition unit corresponds to an odor information acquisition unit). Note that the sensor information acquisition unit may acquire information obtained by other sensors as sensor information. For example, examples of such sensor information include information about the swallowing sound obtained by a voice sensor, the biting strength obtained by a pressure sensor, and the humidity inside the oral cavity obtained by a humidity sensor. Also, in the following description of this embodiment, the case where an evaluation value (score) is generated as information about the evaluation result will be taken as an example for explanation. Also, in this embodiment, the oral findings information includes the target interview information, the odor of the target oral cavity, the swallowing sound, the biting pressure, the oral humidity, and the evaluation related to the denture.

[0011] FIG. 1 is a diagram showing an overview of the apparatus configuration of the oral condition evaluation support system 100 of this embodiment. The oral condition assessment support system 100 includes terminal devices 1, odor information acquisition devices 3, assessment devices 5, a claims processing device 7, and databases DB1 and DB2, all of which can communicate with each other via a network NW. For example, a personal computer can be used for the evaluation device 5 and the claims processing device 7. Terminal device 1 is a device capable of imaging the oral cavity within the target area, and for example, a mobile information terminal such as a smartphone or tablet PC can be used. The odor information acquisition device 3 can use various information detection sensors capable of acquiring information about odors. For example, a breath sensor can be used as an information detection sensor, and commercially available breath sensors that can communicate via a network (NW) may be used. When using a breath sensor, it is possible to use a sensor device capable of detecting specific gas components such as ammonia and acetone contained in the breath. Here, there is a correlation between the amount of specific gas components such as ammonia and acetone that cause bad breath contained in the breath and the tendency for human diseases and deterioration of health status. Examples include periodontal disease, tooth decay, plaque, tartar, tongue coating, decreased saliva, oral cancer, nasal and throat diseases, respiratory diseases, digestive diseases, and stress. In addition, for example, a sweet smell is associated with infectious diseases, acetone with diabetes, the gangrenous smell of ammonia and protein with certain tumors and liver disease, isoprene with hypoglycemia and sleep disorders, methyl mercapbutane with oral bacteria and liver disease, carbon monoxide with stress, ethanol with alcohol consumption, trimethylamine with kidney disease, and nitric oxide with asthma. Therefore, by detecting the gaseous components that cause these odors in exhaled breath, it is possible to estimate the disease. In other words, gaseous component data measured by breath sensors and disease information estimated from it are used as biodetection information. Furthermore, database DB1 and database DB2 can be configured as appropriate, such as cloud-based or on-premise.

[0012] Figure 2 is a schematic diagram of the evaluation device 5 in this embodiment. The medical billing device 7 can also be configured similarly to the evaluation device 5. The evaluation device 5 includes a processor 11 which is an arithmetic processing unit, memory 12 (such as RAM (Random Access Memory)) which is a main memory, and an SSD (Solid State Drive) 13 which is an auxiliary memory. The oral condition evaluation support system 100 also includes a network interface 14 for controlling communication with external units, a monitor 15, input devices 16 (such as a keyboard and mouse), and a media reading device 17. The SSD13 has programs pre-stored to implement the embodiments of the system. These programs can be installed by reading external installation media (such as a CD-ROM or DVD) with the media reading device 17 and storing them in the SSD13, or by downloading them via the network interface 14 and storing them in the SSD13. Furthermore, although the oral condition evaluation support system 100 of this embodiment is a standalone device such as a PC (Personal Computer) with each component housed within the device, one or more of the components may be placed outside the device and connected via a network NW. For example, programs that can be stored in the SSD 13 may be stored in the cloud.

[0013] Figure 3 is a block diagram relating to the evaluation device 5 of this embodiment. As shown in Figure 3, the evaluation device 5 consists of a display unit 20 that displays various information, an operation reception unit 22 that receives operations from users such as doctors and nurses, a storage area 24 that stores various information, a communication unit 26 that is connected to a network NW, and a control unit 30 that controls the operation of these units. The storage area 24 includes an oral cavity finding information storage unit 25 that stores oral cavity finding information of the target.

[0014] Specifically, the display unit 20 is composed of, for example, a monitor, and has the function of displaying various information based on control signals from the control unit 30. Furthermore, the operation reception unit 22 is composed of an input device 16, such as a keyboard, mouse, or touch panel that also serves as the display unit 20, and has the function of receiving input operations from the user and the function of outputting information indicating the received input content to the control unit 30.

[0015] Furthermore, the memory area 24 is composed of memory 12, SSD 13, etc., and has the function of writing and storing various information, and the function of reading various information. This memory area 24 stores a program that realizes an oral condition evaluation model, a program that realizes physical information output, a program that realizes evaluation value registration, generated evaluation values ​​for each subject, and physical information for each subject. The oral findings information storage unit 25 stores odor information obtained by the odor information acquisition device 3, interview information regarding the results of medical professionals' evaluations of the subject's oral condition through interviews, and information about the condition of dentures based on denture images. The information about the condition of dentures may be image data of dentures, or it may be information about the condition of dentures obtained from the image data of dentures. The odor information, interview information, and information about the condition of dentures can be used for machine learning, which will be described later. Furthermore, the oral cavity findings information storage unit 25 may include information obtained by other sensors in addition to or instead of odor information. Specifically, examples include information about swallowing sounds obtained by the aforementioned voice sensor, the strength of clenching obtained by the pressure sensor, and information about the humidity inside the oral cavity obtained by the humidity sensor.

[0016] Furthermore, the communication unit 26 is composed of network IF 14. Furthermore, the control unit 30 is composed of hardware such as the processor 11 and memory 12, and software such as a control program.

[0017] This control unit 30 has processing functions related to the exchange of various signals with the display unit 20, operation reception unit 22, storage area 24, communication unit 26, etc., and functions to control the operation of each unit connected via a predetermined bus.

[0018] In this embodiment, the control unit 30 and operation reception unit 22 correspond to the oral cavity image acquisition unit, the control unit 30 and operation reception unit 22 correspond to the sensor information acquisition unit, the control unit 30 corresponds to the evaluation unit, the storage area 24 corresponds to the oral cavity finding information storage unit, the storage area 24 corresponds to the auxiliary information storage unit, the control unit 30 and display unit 20 correspond to the information output unit, the control unit 30 and operation reception unit 22 correspond to the denture image acquisition unit, the control unit 30 and operation reception unit 22 correspond to the medical history information acquisition unit, and the control unit 30 corresponds to the evaluation result registration unit. The auxiliary information storage unit can also be configured to include database DB1 and database DB2 connected via the communication unit 26 in addition to the storage area 24.

[0019] As described above, in this embodiment, an oral condition evaluation model obtained by machine learning based on the oral condition and its evaluation results is used to generate evaluation values.

[0020] The judgment algorithm for the oral condition evaluation model may be generated, for example, by the procedure shown in Figures 4-7. Evaluation criteria and indices using oral images and denture images can be generated, for example, as follows: For example, first, as shown in Figure 4, images of the patient's oral cavity and surrounding areas, along with accompanying correct labels indicating deterioration of the oral condition, are collected at a medical institution (Step S001). The correct labels are assigned based, for example, on the results of dental examinations, periodontal disease examinations, saliva tests, bacterial tests, PCR tests, and viral isolation culture tests performed on the patient, as well as the findings of a physician or dentist. PCR tests and viral isolation culture tests, although taking several weeks to obtain results, exhibit extremely high accuracy, making their results suitable as correct labels.

[0021] Furthermore, the correct labels used in this learning process may include, for example, image data of a person's oral cavity and data representing an evaluation value of the oral condition indicated by the image data. Evaluation values ​​can be assigned according to predetermined evaluation criteria. For example, the Oral Health Assessment Tool (OHAT) can be used as an evaluation criterion. As shown in Figure 5, the OHAT evaluation items consist of eight items: lips, sublingual area, gums / mucosa, saliva, remaining teeth, dentures (if the subject has them), oral hygiene, and toothache. The OHAT evaluation is based on three levels, with each level assigned an evaluation value of "0," "1," and "2," where the latter level corresponds to a worse condition. For example, each image data is assigned an evaluation value of either "0," "1," or "2."

[0022] In step S002, training data (image data) is generated in which the above inspection results are labeled as correct answers. Based on this training data, machine learning is performed to generate image-based judgment algorithms A1, A2, A3, A4, A5, A6, A7, and A8 (step S003). For example, a convolutional neural network (CNN) can be used for this machine learning. Alternatively, methods using neural networks such as multilayer herceptons (MLP), long-short-term memory (LSTM), gated recurrent units (GRU), graph neural networks (GNN), and transformers may be used as the machine learning method. The above judgment algorithms A1 to A8 are algorithms for determining the degree of deterioration of oral condition for each evaluation item of OHAT when given an image. This makes it possible to quantify the degree of deterioration of oral condition for each evaluation item of OHAT and display image-based OHAT evaluation item indices I1, I2, I3, I4, I5, I6, I7, and I8. The control unit 30 stores the image-based OHAT evaluation item indices I1, I2, I3, I4, I5, I6, I7, and I8 in the memory area 24. Alternatively, instead of using the above judgment algorithms A1 to A8 to output evaluation item indices I1 to I8, a single judgment model may be used to output evaluation item indices I1 to I8 (multi-class classification model). In addition to this assessment, the reliability of the assessment may be quantified, or the degree of certainty that a disease exists may be indicated using a tiered evaluation such as high, medium, or low.

[0023] Selectively, positive or negative results may be determined, for example, "Judgment: Very poor oral condition," "Judgment: Poor oral condition," "Judgment: Normal oral condition," "Judgment: Good oral condition," and "Judgment: Very good oral condition." In addition to or separate from this judgment, the reliability of the judgment may be quantified or indicated by a graded evaluation such as high, medium, or low. Alternatively, the degree of certainty of disease may be indicated by a graded evaluation such as high, medium, or low. The images used to generate training data may be either still images or moving images. Furthermore, the training data may be generated using images taken from multiple locations and directions, or it may consist of images taken from the same location and direction. For example, multiple types of images, such as images of one side and the other side of the oral cavity, can be used to generate training data. Furthermore, regarding denture images, for example, images of only the denture taken after it has been removed from the oral cavity can be used. Furthermore, for the evaluation of dentures, an AI algorithm for abnormality detection may be used as an oral condition evaluation model. For example, an Autoencoder that has been pre-trained on various normal denture images can be used to detect abnormalities in dentures. Since teeth have a fixed shape, shapes that deviate from this (such as chips or cracks) can be detected as abnormalities. In addition to the reconstruction error method described above for Auto Encoders, other methods for anomaly detection include methods that measure the degree of anomaly by distance from the distribution of features obtained from normal images, such as SPADE, Padim, and PatchCore; methods that use reconstruction error with a dictionary obtained from normal images, such as Sparse Modeling; and Efficient-GAN, which is an anomaly detection method using reconstruction error with GANs (Generative Adversarial Networks). For example, one of these methods may be used.

[0024] Furthermore, in this embodiment, evaluation criteria for oral odor are also generated.

[0025] In this case, for example, as shown in Figure 6, odor information obtained using the odor information acquisition device 3 at a medical institution, along with correct labels indicating deterioration of oral condition for each item of OHAT, are collected (step S'001). The correct labels are assigned based on, for example, the results of dental examinations, periodontal disease examinations, saliva tests, bacterial tests, PCR tests, and viral isolation culture tests performed on the patient, as well as the findings of a physician or dentist. PCR tests and viral isolation culture tests take several weeks to yield results, but they exhibit extremely high accuracy, making their results suitable as correct labels. In step S'002, training data (odor data) is generated, with the above test results labeled as correct answers. Based on this training data, machine learning is performed to generate the odor-based judgment algorithm B7 (step S'003). For machine learning of scent information, neural networks and gradient boosting methods can be used. Furthermore, gradient boosting decision tree (GBDT) techniques such as LightGBM (Light Gradient Boosting Machine), XGBoost, and CatBoost can also be employed. The judgment algorithm B7, in response to the input of actual patient odor information, outputs an odor-based OHAT evaluation item index I'7 for the "oral hygiene" evaluation item of OHAT. The control unit 30 stores the odor-based OHAT evaluation item index I'7 in the memory area 24. In addition to this assessment, the reliability of the assessment may be quantified, or the degree of certainty that a disease exists may be indicated using a tiered evaluation such as high, medium, or low.

[0026] Selectively, positive or negative results may be determined, for example, "Judgment: Very poor oral condition," "Judgment: Poor oral condition," "Judgment: Normal oral condition," "Judgment: Good oral condition," and "Judgment: Very good oral condition." In addition to or separate from this judgment, the reliability of the judgment may be quantified or indicated by a graded evaluation such as high, medium, or low. Alternatively, the degree of certainty of disease may be indicated by a graded evaluation such as high, medium, or low.

[0027] Furthermore, in this embodiment, evaluation criteria indicators for medical interviews obtained from healthcare professionals, including physicians or dentists, are also generated.

[0028] In this case, for example, as shown in Figure 7, medical history information and corresponding correct labels indicating deterioration of oral condition for each item of OHAT are collected (step S''001). The correct labels are assigned based on medical history information obtained by examining the patient's oral cavity by healthcare professionals, findings, and patient attribute information obtained at the medical institution. Correct labels may be assigned to a single patient by multiple healthcare professionals.

[0029] I will explain the methods for collecting medical information through interviews, using examples. For example, regarding toothaches, healthcare professionals ask patients or care recipients, "Do you have any toothaches?" to confirm the presence or absence of toothaches and the location of the pain. If the healthcare professional touches the subject's teeth and the subject shows no particular signs, the score is determined to be 0, indicating no verbal or physical signs of pain. On the other hand, if the healthcare professional touches the subject's teeth and the subject shows verbal signs such as facial twitching, lip biting, or aggressive behavior, or if the subject shows signs such as refusing to eat in daily life at a care facility or day service, the score is determined to be 1 (change). Furthermore, if, in addition to verbal and behavioral signs, there is swelling of the cheeks or gums, tooth fracture, ulcer, subgingival abscess, etc., the score is determined to be 2 (pathological). Note that swelling of the cheeks or gums, tooth fracture, ulcer, subgingival abscess, etc., may also be collected as interview information from image data. Regarding saliva, healthcare professionals instruct subjects to "open your mouth slightly" to check the condition of the saliva in the oral cavity. If the saliva is found to be moist and serous, the score is determined to be 0. On the other hand, if there is a small amount of saliva and the mucous membrane is sticky, or if the saliva is foamy, the score is determined to be 1 (change). If the saliva is dried up or the person being cared for claims to have a significant feeling of thirst, the score is determined to be 2 (pathological). Note that the stickiness of the mucous membrane may be measured using a viscometer or other methods.

[0030] Regarding the lips, medical professionals carefully observe the subjects' lips. They instruct the subjects to "open your mouth slightly," asking them to gently open the corners of their mouths. If the lips are judged to be normal, moist, and pink in appearance, the score is 0. If dryness or cracking is observed at the corners of the mouth, the score is 1 (change). If ulcerative lesions or bleeding resulting from them are observed, the score is 2 (pathological). For the tongue, a medical professional instructs the subject to "open your mouth and stick out your tongue," and carefully observes the visible state of the tongue by comparing it with reference photographs showing the state of the tongue and a score, in order to determine the score for the "tongue" item. If necessary, the tongue may be touched during this process. If the tongue is judged to be normal, moist, and pink in appearance, the score is determined to be 0. If tongue coating is observed, regardless of the amount, nature, or color, the score is determined to be 1 (changed). If ulcerative lesions or bleeding resulting therefrom are observed, the score is determined to be 2 (pathological).

[0031] Next, training data is generated in which the above test results are labeled as correct answers (step S''002). Based on this training data, machine learning is performed to generate the medical history-based judgment algorithms C1, C2, C3, C4, C5, C6, C7, and C8 (step S''003). For machine learning of medical interview data, neural networks and gradient boosting methods can be used. Furthermore, gradient boosting decision tree (GBDT) techniques such as LightGBM (Light Gradient Boosting Machine), XGBoost, and CatBoost can also be employed. In addition to this assessment, the reliability of the assessment may be quantified, or the degree of certainty that a disease exists may be indicated using a tiered evaluation such as high, medium, or low.

[0032] Selectively, positive or negative results may be determined, for example, "Judgment: Very poor oral condition," "Judgment: Poor oral condition," "Judgment: Normal oral condition," "Judgment: Good oral condition," and "Judgment: Very good oral condition." In addition to or separate from this judgment, the reliability of the judgment may be quantified or indicated by a graded evaluation such as high, medium, or low. Alternatively, the degree of certainty of disease may be indicated by a graded evaluation such as high, medium, or low.

[0033] The judgment algorithms C1, C2, C3, C4, C5, C6, C7, and C8, in response to input information from actual medical professionals regarding interviews, findings, and attribute information about the lips, tongue, gums / mucosa, saliva, remaining teeth, dentures, oral hygiene, and toothache, output interview-based OHAT evaluation item indices I''1, I''2, I''3, I''4, I''5, I''6, I''7, and I''8 for each evaluation item of OHAT. The control unit 30 stores the interview-based OHAT evaluation item indices I''1, I''2, I''3, I''4, I''5, I''6, I''7, and I''8 in the memory area 24. Alternatively, instead of using the above judgment algorithms C1 to C8 to output evaluation item indices I''1 to I''8, a single judgment model may be used to output evaluation item indices I''1 to I''8 (multi-class classification model). In addition to this assessment, the reliability of the assessment may be quantified, or the degree of certainty that a disease exists may be indicated using a tiered evaluation such as high, medium, or low.

[0034] Furthermore, when the control unit 30 receives image-based OHAT evaluation item indices I1, I2, I3, I4, I5, I6, I7, and I8, odor-based OHAT evaluation item indices I'7, and interview-based OHAT evaluation item indices I''1, I''2, I''3, I''4, I''5, I''6, I''7, and I''8 as input, it may read each indice from the memory area 24 and perform ensemble processing. As an example of such ensemble processing, the obtained image-based OHAT evaluation item indices, odor-based OHAT evaluation item indices, and interview-based OHAT evaluation item indices are given as input to a ridge regression model, and a final OHAT score is obtained in which each indice is ensembled as an evaluation result for the oral condition. Ensemble methods may include, in addition to the ridge regression model described above, processes such as obtaining the mean, obtaining the maximum, obtaining the minimum, weighted addition, bagging, boosting, stacking, lasso regression, linear regression, and other machine learning techniques. Ensemble processing is performed in multiple stages. The ensemble can combine all three types of judgment algorithms with different inputs (image-based judgment algorithm, odor-based judgment algorithm, and interview-based judgment algorithm), or any two of them can be combined. When combining multiple judgment algorithms, first, judgment algorithms with different inputs are ensembled for each evaluation item to create an algorithm that judges each evaluation item. Next, the ensembled algorithms that judge each evaluation item are further ensembled to obtain an overall OHAT score that takes all evaluation items into consideration. By combining an image-based judgment algorithm corresponding to each item of OHAT with an odor-based judgment algorithm and / or a questionnaire-based judgment algorithm corresponding to each item of OHAT, each item of OHAT can be judged more accurately than by using each judgment algorithm individually.

[0035] As described above, terminal device 1 can be a portable information terminal such as a smartphone or tablet PC, and is equipped with a touch panel display and a camera. The touch panel display is a device that combines an input function that accepts user input by touching the screen with a finger or the like, and an output function that displays an image on the screen. The housing of terminal device 200 has a shooting window formed therein that guides light from the subject into the housing. The camera has a photoelectric conversion element that receives light incident into the housing from the shooting window through an optical system, and an A / D conversion circuit that converts the analog signal generated by the element by photoelectric conversion into a digital signal. Terminal device 200 generates image data representing the subject from the digital signal output by the camera.

[0036] Terminal device 1 has application software (hereinafter referred to as "app") specifically for the oral condition evaluation support system 100 installed. When the app is launched, terminal device 200 captures images of the target oral cavity according to the launched application. The terminal device 200 sends the captured oral cavity image data to the evaluation device 5.

[0037] When the odor information acquisition device 3 uses a breath sensor as an information detection sensor, it detects and acquires odor information by inserting the breath sensor into the oral cavity. Alternatively, a breath sensor may be attached to dental instruments such as a penlight for illuminating the oral cavity, a dental drill, a probe, or a sickle-shaped scaler, so that odor information is detected when these are inserted into the oral cavity. The odor information acquisition device 3 sends the acquired odor information to the evaluation device 5.

[0038] Next, the processing flow in this embodiment will be explained using Figures 8 and 9. Figure 8 shows the processing flow related to the generation of evaluation values, which are information about the evaluation, and Figure 9 shows the processing flow related to the registration of evaluation values. In the following explanation, we will use the example of using both oral images and denture images for generating evaluation values, but the explanation is not limited to this example; it is also acceptable to use only oral images. Oral images and denture images are collectively referred to as oral images, etc. Furthermore, similar to the images used for generating training data, the target images used for generating evaluation values ​​may be either still images or moving images. The evaluation may also be generated using images taken from multiple locations or directions, or images taken from the same location and direction may be used. For example, multiple types of images, such as images of one side and the other side of the oral cavity, can be used to generate evaluation values. Furthermore, regarding images of dentures, for example, images taken of dentures after they have been removed from the oral cavity can be used.

[0039] First, we will explain the processing flow related to the generation of evaluation values ​​shown in Figure 8. First, in step S101, when the control unit 30 receives a user operation (instruction to generate evaluation values) to execute the program related to generating evaluation values, it executes the program to enable the acceptance of medical interview information, oral images, and odor information.

[0040] In step S102, the control unit 30 acquires the medical interview information input via the operation reception unit 22 and stores it in the oral findings information storage unit 25 of the memory area 24.

[0041] In step S103, the control unit 30 acquires data such as oral images sent from the terminal device 1 via the communication unit 26 and stores it in the memory area 24. The control unit 30 stores denture images in the oral findings information storage unit 25 of the memory area 24.

[0042] In step S104, the control unit 30 acquires odor information sent from the odor information acquisition device 3 via the communication unit 26 and stores it in the oral cavity findings information storage unit 25 of the memory area 24.

[0043] Proceeding to step S105, the control unit 30 generates evaluation values ​​using an oral condition evaluation model based on oral images, interview information, and odor information. The generated evaluation values ​​may be for each evaluation item (lips, tongue, gums / mucosa, saliva, remaining teeth, dentures, oral hygiene, and toothache), or they may be combined into a single value (e.g., an average value).

[0044] In step S106, the control unit 30 determines whether the database DB1 contains the subject's physical information. This determination can be made, for example, based on information (such as a name) used to identify the individual included in the medical interview information.

[0045] If physical information is stored in the database DB1, in step S107, the control unit 30 acquires the physical information via the communication unit 26. In step S108, the control unit 30 displays the obtained evaluation value and physical information on the display unit 22.

[0046] On the other hand, if it is determined in step S106 that no physical information is stored in the database DB1, the process proceeds to step S109, and the control unit 30 displays only the obtained evaluation values ​​on the display unit 22.

[0047] Next, we will explain the processing flow related to the registration of evaluation values ​​shown in Figure 9. First, in step S201, when the control unit 30 receives a user operation (registration instruction) to execute the program related to registering evaluation values, it executes the program. Proceeding to step S202, the control unit 30 determines whether physical information is stored in the database DB1. This determination can be made based on information for identifying the individual (such as a name) included in the medical interview information.

[0048] If it is determined that no physical information is stored, the control unit 30 terminates processing. In this case, the control unit 30 may also display a message on the display unit 22 informing the user that no physical information is stored.

[0049] On the other hand, if it is determined that physical information is stored, the control unit 30 associates the evaluation value with the physical information and stores it in the database DB1.

[0050] As described above, this embodiment makes it possible to obtain more appropriate evaluation results regarding the oral condition. It should be noted that the present invention is not limited to the embodiments described above, and other embodiments are of course possible. For example, in the above embodiment, an oral condition evaluation model is obtained by machine learning using oral images, medical history information, and odor information, and the control unit 30 generates evaluation values, which are information about the evaluation results, using the target oral images, medical history information, and odor information. However, the invention is not limited to this, and for example, the configuration of the oral condition evaluation model and the generation of information about the evaluation results may be performed using oral images and odor information. Furthermore, in the above-described embodiment, odor-based evaluation item indices are output for OHAT oral cleaning. In other embodiments, odor-based evaluation item indices may be output for other evaluation items in addition to or instead of OHAT oral cleaning. For example, odor-based evaluation item indices may be output for one or more of OHAT's other evaluation items (lips, sublingual, gingiva / mucosa, saliva, remaining teeth, dentures, toothache).

[0051] Furthermore, in the above embodiment, for example, it is determined whether physical information is stored in the database DB1, and if physical information is stored, the information about the evaluation result and the physical information are displayed on the display unit 22, and the information about the evaluation result and the physical information are associated and stored in the database DB1. As another aspect, instead of or together with the body information, it is determined whether insurance information of the subject is stored in the database DB1 or the database DB2 (hereinafter also simply referred to as the database DB1 etc.). When the insurance information of the subject is stored, the information about the evaluation result and the insurance information are associated and displayed on the display unit 22, and the information about the evaluation result and the insurance information may be associated and stored in the database DB1 etc. In the case of this aspect, the attached information storage unit has the above-described configuration including the database DB1 etc. In this specification, the insurance information is the patient's enrollment information in a medical insurance (a system for reducing the burden of medical expenses) applied to the expenses of examinations and treatments for diseases, injuries, etc. in a medical institution, and includes information indicating the presence or absence of the patient's enrollment in medical insurance, the type of insurance, etc.

[0052] This embodiment or other embodiments may be used to determine each OHAT evaluation item based on a subject image etc. and generate, for example, a notification document based on the determination result. Such a processing system generates an evaluation result of the oral cavity state based on, for example, a subject image etc. having the oral cavity photographed by a photographing device as the subject. Then, based on the information about the evaluation result, information indicating whether it is a disease for which insurance is applicable or notification documents required for various organizations such as countries, local governments, schools, workplaces, etc. are generated. Therefore, a user (for example, a medical worker) who needs to create such a notification document etc. can create the notification document etc. more efficiently. In other words, it can be expected that the creation of documents related to the enhancement of oral cavity cooperation can be performed more efficiently.

[0053] Also, other evaluation items can be included in addition to the evaluation items of OHAT. For example, as one of the evaluation items, the degree of mouth opening may also be measurable. Specifically, the upper and lower teeth or lips are detected by an object detection AI, and the distance between the upper and lower is measured. The magnification rate of the image is calculated based on the size of the lips and teeth to measure the degree of opening. Examples of object detection AI algorithms include YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), DETR (End-to-End Object Detection with Transformers), and segmentation models such as UNet and Mask R-CNN.

[0054] Furthermore, other evaluation items may be used in place of the evaluation items of OHAT, and are not particularly limited. For example, an oral condition evaluation model may be generated for the evaluation items as shown in Figure 10, and information on the evaluation results regarding the subject's oral condition may be generated using this oral condition evaluation model. Furthermore, evaluation items should include swallowing function, chewing function, tongue movement, lip movement, and oral cavity moisture level.

Claims

1. An oral image acquisition unit that acquires an oral image of the target, A medical information acquisition unit that acquires medical information related to the medical interview results of the subject, The oral findings information storage unit stores the oral images and the medical history information of the subject, The system includes an evaluation unit that generates evaluation information regarding the evaluation results of the target oral condition using an oral condition evaluation model obtained by machine learning based on the oral condition and its evaluation results, based on the oral image acquired by the oral image acquisition unit and the medical interview information acquired by the medical interview information acquisition unit, The oral condition evaluation model includes an image-based evaluation algorithm that outputs evaluation information about the evaluation results of the evaluation items corresponding to the acquired oral image using predetermined evaluation criteria for a plurality of evaluation items, and a questionnaire-based evaluation algorithm that outputs evaluation information about the evaluation results of the evaluation items corresponding to the acquired questionnaire information using the evaluation criteria. The oral condition evaluation support system generates evaluation information by performing ensemble processing on the judgment information obtained from the image-based judgment algorithm and the interview-based judgment algorithm.

2. The system further includes a sensor information acquisition unit that acquires odor information of the target oral cavity obtained by the sensor, The oral cavity information storage unit stores the odor information as the oral cavity information, The oral condition evaluation model is obtained by machine learning based on the odor of the mouth and its evaluation results, and further includes an odor-based judgment algorithm that outputs judgment information about the evaluation results of the evaluation items corresponding to the acquired odor information using the evaluation criteria. The oral condition evaluation support system according to claim 1, wherein the evaluation unit generates the evaluation information by performing ensemble processing on the judgment information obtained from the image-based judgment algorithm, the interview-based judgment algorithm, and the odor-based judgment algorithm.

3. An oral image acquisition unit that acquires an oral image of the target, A sensor information acquisition unit that acquires odor information of the target oral cavity obtained by the sensor, The oral findings information storage unit stores the oral images and odor information as the oral findings information of the subject, The system comprises an evaluation unit that generates evaluation information regarding the evaluation results of the target oral condition using an oral condition evaluation model obtained by machine learning based on the oral condition and its evaluation results, based on the oral image acquired by the oral image acquisition unit and the odor information acquired by the sensor information acquisition unit, The oral condition evaluation model includes an image-based judgment algorithm that outputs judgment information about the evaluation results of the evaluation items corresponding to the acquired oral image using predetermined evaluation criteria for a plurality of evaluation items, and an odor-based judgment algorithm that outputs judgment information about the evaluation results of the evaluation items corresponding to the acquired odor information using the evaluation criteria. The oral condition evaluation support system generates evaluation information by performing ensemble processing on the judgment information obtained from the image-based judgment algorithm and the odor-based judgment algorithm.

4. The oral condition evaluation support system according to any one of claims 1 to 3, wherein the ensemble processing is a machine learning-based processing.

5. The system further includes a denture image acquisition unit that acquires a denture image obtained by imaging the aforementioned denture, The oral cavity information storage unit stores information about the condition of the denture based on the denture image as the oral cavity information. The oral condition evaluation support system according to claim 1, wherein the image-based judgment algorithm is obtained by further machine learning based on the condition of the denture and the evaluation result, and the judgment information is output based on the denture image.

6. An auxiliary information storage unit that stores physical information (excluding date of birth) about the subject's body, The oral condition evaluation support system according to any one of claims 1 to 3, further comprising: an information output unit that outputs the evaluation information generated by the evaluation unit in association with the physical information.

7. An auxiliary information storage unit that stores insurance information for the aforementioned insurance policy, The oral condition evaluation support system according to any one of claims 1 to 3, further comprising: an information output unit that outputs the evaluation information generated by the evaluation unit in association with the insurance information.

8. The oral condition evaluation support system according to claim 6, further comprising an evaluation result registration unit that stores the evaluation information generated by the evaluation unit in association with the physical information in the auxiliary information storage unit.

9. The oral condition evaluation support system according to claim 7, further comprising an evaluation result registration unit that stores the evaluation information generated by the evaluation unit in association with the insurance information in the auxiliary information storage unit.

10. The oral condition evaluation support system according to any one of claims 1 to 3, wherein the evaluation criteria include evaluation items relating to the lips, tongue, gums / mucosa, saliva, remaining teeth, dentures (if the subject has them), oral hygiene, and toothache.

11. The oral condition evaluation support system according to claim 2 or 3, wherein the oral findings information includes information on swallowing sounds, bite pressure, and oral humidity of the subject obtained by the sensors, respectively.

Images