Dementia and / or depression prediction AI system and training data creation method

Abstract

To provide an AI system capable of estimating severity of dementia and / or depression in a subject on the basis of information obtained outside a hospital.SOLUTION: A method of creating teacher data for an AI system for estimating severity of dementia and / or depression is provided. Information to be used as teacher data for the AI system can be acquired from a site where a user receives prescribed drugs or from an application on a mobile terminal used by the user. The teacher data at least includes a face picture of the user, and a prescription, instruction, or information that enables estimation of dementia and / or depression in the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to AI for assessing medical conditions, and more particularly to technology for estimating dementia and / or depression using facial photographs. [Background technology]

[0002] Early detection and treatment of dementia and depression are important. One method for detecting symptoms is to use AI to identify the subject's facial photograph and estimate the degree of dementia or depression. However, building such a system generally requires information managed by the medical institution, and it is not easy for other businesses to do so. Therefore, there was a need for a method that uses AI to estimate the degree of dementia or depression of a subject from information obtained outside of a hospital.

[0003] Various technologies have been proposed to address these problems. For example, a method of assisting doctors in diagnosis by image analysis using AI (see Patent Document 1) has been proposed and is a publicly known technology. More specifically, this is a device that uses training data consisting of image data of an image of a subject, observation data, and information on the subject's lifestyle and dietary habits to input image data and subject information, performs machine learning to output observation data, and uses this AI to output the observation data. However, since data acquisition in this prior art is performed at the hospital, other businesses cannot handle it, and the above problem has not been resolved. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-108800 Summary of the Invention [Problem to be solved by the invention]

[0005] In view of the above problems, the present invention aims to build an AI that can estimate the degree of dementia and / or depression of a subject based on information obtained outside of a hospital. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, the present invention provides a method for creating training data to be used by an AI that estimates the degree of dementia and / or depression from a facial photograph of a user, wherein in a system for estimating the degree of dementia and / or depression, information that serves as training data for the AI ​​can be acquired from a location where the user receives prescription medicine or from an application on a mobile device used by the user, and the training data includes at least a facial photograph of the user, a prescription, an instruction sheet, or information that can be used to estimate the user's dementia and / or depression, and the facial photograph of the user is an expressionless face photograph or a smiling face photograph. One or both of the facial photographs are used, facial features are extracted from the facial photographs and generated as example data for training data, the example data for training data includes the user's age, sex, medical history, and family medical history, and the level of dementia and depression is estimated from the type and amount of medication written on the prescription or instructions and used as correct answer data for the training data, and the features expected for a specific level of dementia and depression are estimated and used as the example data, with the predicted level of dementia and depression being used as the correct answer data, and a means is adopted to create the training data for machine learning.

[0007] The present invention also provides an AI system that uses training data created by the training data creation method described above, and employs a means for estimating the level of dementia and depression of a subject based on a facial photograph of the subject.

[0008] Furthermore, the present invention employs a means for taking a photograph of the subject's face at the pharmacy and providing advice based on the results of the AI ​​system.

[0009] Furthermore, the present invention employs a means for taking a photograph of the subject's face and directing them to video content based on the results of the AI ​​system.

[0010] Furthermore, the present invention employs a means of taking a photograph of the subject's face and, depending on the results of the AI ​​system, referring the subject to a specialized medical institution.

[0011] Furthermore, the present invention employs a means for taking a photograph of the subject's face and notifying the family of advice based on the results of the AI ​​system.

[0012] Furthermore, the present invention employs a means for taking a facial photograph of the subject at an OTC medical supply store and providing advice based on the results of the AI ​​system.

[0013] Furthermore, the present invention employs a means for obtaining a facial photograph of the subject taken with a mobile device used by the subject via a network, and for providing advice based on the results of the AI ​​system's analysis of the obtained facial photograph. [Effects of the Invention]

[0014] According to the dementia and / or depression estimation AI system and training data creation method of the present invention, information for constructing an AI that estimates the degree of dementia or depression of a subject can be obtained outside of a hospital, so that an AI that estimates dementia and / or depression can be constructed by anyone, not just hospital personnel. In addition, by utilizing applications on the mobile devices used by users, it is possible to easily obtain a large amount of information needed to build AI that can predict dementia and / or depression. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a flow chart showing an embodiment of a teacher data creation method according to the present invention. [Figure 2] 1 is a schematic diagram showing an embodiment of a teacher data creation method according to the present invention; [Figure 3] 10 is a calculation table showing an example of generating correct answer data in the teacher data generating method according to the present invention. [Figure 4] FIG. 1 is a table showing an example of training data in a training data creation method according to the present invention. [Figure 5] FIG. 1 is a schematic diagram showing an example of the operation of the AI ​​in the dementia and / or depression prediction AI system according to the present invention. [Figure 6] FIG. 10 is a flow chart showing another embodiment of the teaching data creation method according to the present invention. [Figure 7] FIG. 1 is a schematic diagram showing an example of extracting facial features in the dementia and / or depression prediction AI system according to the present invention. [Figure 8] FIG. 10 is a table showing another example of the training data in the training data creation method according to the present invention. [Figure 9] FIG. 1 is a flow chart showing an example of teacher data in a teacher data creation method according to the present invention. [Figure 10] 1 is an explanatory diagram showing an example of teacher data in a teacher data creation method according to the present invention; [Figure 11] FIG. 1 is a flow chart showing an example of the dementia and / or depression prediction AI system according to the present invention at a pharmacy. [Figure 12] FIG. 1 is a flow chart showing an example of monitoring support in the dementia and / or depression prediction AI system according to the present invention. [Figure 13] FIG. 10 is a system diagram of another embodiment of the present invention. [Figure 14] FIG. 10 is a diagram of a screen of a mobile terminal according to another embodiment of the present invention. [Figure 15] 10 is a table showing information for generating training data according to another embodiment of the present invention. [Figure 16] FIG. 10 is a flowchart showing a procedure for acquiring teacher data according to another embodiment of the present invention. [Figure 17] FIG. 1 is a flow chart showing an example of the dementia and / or depression prediction AI system according to the present invention on a mobile terminal. DETAILED DESCRIPTION OF THE INVENTION

[0016] The greatest feature of the dementia and / or depression prediction AI system and training data creation method of the present invention is that it can obtain information for estimating the degree of dementia or depression in a subject at a location other than a hospital, thereby making it possible to build an AI that can predict dementia and / or depression not only for hospital personnel but also for anyone other than hospital personnel. Hereinafter, embodiments of an AI system for predicting dementia and / or depression and a method for creating training data according to the present invention will be described with reference to the accompanying drawings.

[0017] The overall configuration and the configuration of each part of the dementia and / or depression prediction AI system and teacher data creation method described below are not limited to the examples described below, but can be modified as appropriate within the scope of the technical idea of ​​the present invention, i.e., within the scope of a configuration that can achieve the same functional effects. Furthermore, since the present invention is a dementia and / or depression prediction AI system and a training data creation method, it naturally includes a dementia prediction AI system, a depression prediction AI system, and a dementia and depression prediction AI system, but the examples will focus on examples of predicting dementia and depression. In the case of a dementia prediction AI system, the system can be easily constructed by excluding the depression factor, which is another factor. The same is true for a depression prediction AI system. In addition, in the embodiments, a user refers to a person who cooperates in providing training data, and a subject refers to a person whose condition is estimated using AI. [Example]

[0018] The present invention will be described with reference to FIGS. FIG. 1 is a flow diagram showing an embodiment of a training data creation method according to the present invention. FIG. 2 is a schematic diagram showing an embodiment of a training data creation method according to the present invention, where (a) is a schematic diagram of data acquisition work performed in a pharmacy, (b) is a schematic diagram of the work of creating sample data from a user's facial photograph, and (c) is a schematic diagram of the work of creating correct data from a prescription or other information. FIG. 3 is a calculation table showing an embodiment of generating correct data in the training data creation method according to the present invention, where (a) is a table for estimating and calculating dementia and depression levels, and (b) is a calculation table obtained by estimating and calculating dementia and depression levels by actually entering the type and amount of medication. FIG. 4 is a table showing an embodiment of training data in the training data creation method according to the present invention, where (a) is an example in which correct data is in 10% frequency increments, (b) is an example in which the frequency is 1 or 0, and (c) is an example in which age and other factors are included in the sample data. FIG. 5 is a schematic diagram showing an example of the operation of the AI ​​in the dementia and / or depression prediction AI system according to the present invention, where (a) shows an example of using AI to predict dementia, etc., and (b) shows the process of machine learning using AI.

[0019] Dementia and / or Depression Estimation AI1 is an AI that estimates the degree of dementia and depression, and its estimation capabilities are improved through machine learning. One method for building a dementia and / or depression prediction AI1 is to generate training data and have the AI ​​learn a large amount of it to make highly accurate predictions. Training data is data used to train AI in machine learning. Training data is often generated by pairing example data with correct answer data. Example data is data that represents the phenomenon that you want the AI ​​to estimate, and correct answer data is the data that you want the AI ​​to answer.

[0020] When making an AI predict whether or not a person has a disease, for example, the subject's symptoms are used as example data, and the subject's disease name is used as correct answer data. By having the AI ​​learn a large amount of such training data, an AI that can predict disease is constructed.

[0021] When examining illnesses such as dementia and depression, doctors often look at facial expressions as part of their diagnosis. When considering a system that uses AI to predict dementia or depression, the subject's facial expressions can be obtained by taking a photo with a camera, but the subject's illness name is managed by a doctor in a medical record, etc., and it is generally difficult to take this data outside the hospital.

[0022] Therefore, in this embodiment, the name and severity of the subject's illness are estimated from the prescription, etc. at the pharmacy, and training data is obtained. In other words, this is a method for creating training data in which information that serves as training data for AI is obtained at the location where the user receives the prescribed medication, and the training data includes at least a photograph of the user's face and the prescription or instructions. Information that serves as training data includes information that serves as the basis for the training data and information that is used as training data. Prescriptions, etc. include at least prescriptions, medicine notebooks, and instructions. Instructions, also known as medication instructions, are written documents in which a doctor instructs guardians, facilities, etc. on the time and amount of medication to be administered. The procedure for obtaining training data will be explained using Figures 1 and 2. Fig. 1 shows a flow for generating the teacher data 40. Fig. 2 shows a procedure for generating the teacher data 40. The information is collected by the pharmacy 10. Here, the term "pharmacy" refers to both pharmacies that are independent of the hospital and pharmacies that are located within the hospital. The data handled is the same in both cases. First, the contents of the prescription 31 are obtained from the user P at the pharmacy 10. The contents of the medicine notebook 30 and the instructions may also be obtained along with the prescription 31. Since the user P has come to the pharmacy 10 to obtain the medicine listed on the prescription 31 and the medicine notebook 30, obtaining the prescription 31 is easy. Apart from dispensing medication, permission is obtained to use information from prescriptions 31, medication notebooks 30, and instructions to build dementia and / or depression prediction AI 1, and information is acquired (S101, Figure 2(a)). Next, in order to construct the dementia and / or depression prediction AI1, permission is obtained from the user to obtain a facial photograph 12, and the user P is photographed with the photographing device 11 to obtain the facial photograph 12 of the user (S102, Figure 2(a)). As the user's facial photograph 12, a facial photograph 13 of the user with a neutral expression and a facial photograph 14 of the user with a smiling expression are acquired. The neutral face can be said to be a natural, ordinary face. It is said that in cases of dementia or depression, facial expressions tend to be dull, with drooping eyelids and a lack of overall expression. Smiles may also be unnatural or may be no different from a blank smile. Therefore, by using facial photos of users with a neutral expression and a smiling expression, the difference between normal people and those with dementia or depression can be emphasized. When taking a photo with no expression, if it is difficult to say, "Please make sure you have no expression," you can use a phrase such as, "Please let me take a photo of your face in a natural, normal way." By steps S101 and S102, the acquisition of information from the user is completed.

[0023] Next, the presence and severity of dementia or depression are estimated from the type and amount of medication listed on the prescription 31, etc. (S103, Figure 2(b)). While a pharmacist or other professional can make a certain degree of estimation, a calculation table 43 is used to reduce variability. An example of the calculation table is shown in Figure 3(a). Correct answer data is generated using the calculation table 43. Next, facial photograph data 20 is generated from the user's facial photograph 12. Generally, facial photographs taken outside a studio or the like have different angles of view, resolutions, backgrounds, etc., so by performing image correction so that the angles of view, resolutions, and backgrounds are similar, appropriate facial photograph data 20 can be generated as example data 41 (S104, FIG. 2(c)). The facial photograph data 20 consists of facial photograph data 21 of a face with no expression and facial photograph data 22 of a face with a smile. A series of operations completes one piece of training data (S105). In other words, the process involves using a photograph of the user's face as example data for the training data, and using the level of dementia and depression estimated from the type and amount of medication listed on the prescription or instructions as correct answer data for the training data, to generate training data for machine learning. Increasing the amount of training data can improve the accuracy of machine learning.

[0024] A calculation table 43 is used to estimate the degree of dementia and depression from the prescription 31, medicine notebook 30, and instructions. Calculation Table 43 has a column for recording the type and amount of medication, which is filled out according to the user's information. There are columns for dementia and depression, and coefficients are listed for each type of medication. Medications that are closely related to each disease have high coefficients, and medications that are less related have low coefficients. An evaluation score is determined by entering the type and amount of medication. The severity of each illness can be determined by adding up the evaluation scores. To obtain correct data, normalization is required, such as from 0 to 100%, so normalization is performed using a predetermined number. There are two ways to calculate correct data: by expressing it as a percentage, or by expressing it as 1 or 0. Figure 3(b) shows a calculation example using specific numbers. In this example, since there are many medications related to dementia, the total evaluation score for dementia is 23 points, and the total evaluation score for depression is 4 points. As an example, normalization is performed by 30, and the evaluation for dementia is 76.7% and the evaluation for depression is 13.3%. As correct answer data, correct answer data 1 with 10 levels and correct answer data 2 with 2 levels (1, 0) are generated. Correct data 1 has a dementia level of 70% and a depression level of 10%, and correct data 2 has a dementia level of 1 and a depression level of 0. This process generates correct data on the dementia level and depression level.

[0025] 4 shows examples of training data. In FIG. 4(a), facial photo data 20 is used as example data of the training data, and as the facial photo data 20, expressionless facial photo 21 and smiling facial photo data 22 are used. The correct answer data for both dementia and depression is 10-level data. The correct answer data has 100 correct answer elements on a 10-level x 10-level scale, so a correspondingly large amount of training data is required for machine learning. Figure 4(b) shows the case where two levels of data are used as the correct answer data for both dementia and depression. The correct answer data consists of four correct answer elements, with two levels x two levels, so the amount of training data required for machine learning may be smaller than in the case of Figure 4(a). Figure 4(c) shows an example where the user's age, gender, and medical history are added in addition to facial photo data. Family medical history can also be added. Using a large amount of data can sometimes make estimation easier. However, when using AI, it is necessary to input the same data as during learning, so in this case, the user's age, gender, and medical history must be input in addition to facial photo data, which reduces versatility.

[0026] An overview of the operation of AI during learning and utilization will be described with reference to Figure 5. A machine learning system 50 shows an example of deep learning using a neural network. The dementia and / or depression estimation AI1 consists of an input section 51, an intermediate section 52, and an output section 53. During machine learning, a correct answer section 54 is added. The input section 51 has elements X1 to Xn, which correspond to the information to be input into the AI. The intermediate section 52 is the section that inherits information from the input layer and performs various calculations. The more layers the intermediate section 52 has, the more complex the analysis that can be performed. The calculation results are sent to the output section 53. The output section 53 is the section that outputs the answer and has elements Z1 to Zm, which correspond to the answer. The value of each element is, for example, between 0 and 1, and the element with the highest value is estimated to be the answer (Figure 5(a)). The correct answer unit 54 is the part that sets the correct answer data 42 to be given to the dementia and / or depression prediction AI 1 during machine learning, and has elements A1 to Am. The number of elements is the same as that of the output unit 53. Of A1 to Am, the numerical value of the correct answer element is set to 1, and the numerical values ​​of the other elements are set to 0. The difference between the numerical values ​​of Z1 to Zm and A1 to Am is calculated, and the sum of these values ​​becomes the error D. Minimizing the error D improves the accuracy of the AI ​​(Figure 5(b)).

[0027] The operation of the dementia and / or depression prediction AI 1 will be simplified for the sake of explanation, and an example of an AI that predicts whether or not a person has dementia will be described. Regarding the output unit 53, the AI's answer is either dementia or no dementia. Therefore, the elements of the output unit 53 are, for example, Z1 "has dementia" and Z2 "does not have dementia." When facial photograph data 20 including expressionless facial photograph data 21 and smiling facial photograph data 22 is input, the number of elements X of input unit 51 is an element according to the resolution. When facial photo data 20 is input to the AI ​​input unit 51, elements Z1 and Z2 of the output unit 53 each output a numerical value between 0 and 1. The higher the numerical value, the more accurate the answer is estimated by the AI. If the difference between the numerical values ​​is large, the accuracy of the estimation is high, and if the difference between the numerical values ​​is small, the accuracy of the estimation is low. For example, if Z1 is 0.1 and Z2 is 0.9, Z2 is the higher value, so the AI's answer will be "not dementia," which is Z2. Because the difference between the values ​​of Z1 and Z2 is large, the accuracy can be said to be high.

[0028] When the training data in Figure 4(b) is used, the output unit 53 of the dementia and / or depression estimation AI 1 will have four levels: dementia level 0 and depression level 0, dementia level 0 and depression level 1, dementia level 1 and depression level 0, and dementia level 1 and depression level 1. Therefore, Z1 to Z4 can be assigned, respectively. After the AI ​​is constructed, facial photo data 20, which consists of expressionless facial photo data 21 and smiling facial photo data 22, is input into dementia and / or depression prediction AI 1, and processed by intermediate section 52 consisting of a neural network, and the results are output as numbers between 0 and 1 for Z1 to Z4 of output section 53. For example, if Z1 is 0.1, Z2 is 0.6, Z3 is 0.2, and Z4 is 0.1, the highest value, Z2, will be adopted. Z2 has a dementia level of 0 and a depression level of 1, so the AI's answer will be, "There is a possibility of depression." You can also use the highest value as the probability. In this case, the value of Z2 is 0.6, so you can say "There is a 60% chance of depression."

[0029] When using the training data in Figure 4(a), the output unit 53 of the dementia and / or depression prediction AI 1 can be divided into 100 elements, in 10% increments, from "dementia level 0% - depression level 0%" to "dementia level 90% - depression level 90%." In this case, the output unit 53 has 100 elements, Z1 to Z100. Z1 to Z100 of the output unit 53 for the input are checked, and the highest one is the answer. If a certain element of Z is the highest, the corresponding dementia level XX% and depression level XX% are the answers.

[0030] Another method for simplifying the output unit 53 is to separate it into a dementia estimation AI and a depression estimation AI. By doing so, the amount of data in the input unit 51, intermediate unit 52, and output unit 53 can be reduced. In this case, the final output of the dementia and / or depression prediction AI1 will be the combination of the answers from the two AIs.

[0031] Next, we will explain the machine learning method of the dementia and / or depression estimation AI 1 with reference to Figure 5(b). Facial photo data 20, consisting of expressionless facial photo data 21 and smiling facial photo data 22, is input to the input unit 51 of the dementia and / or depression estimation AI 1, and the output unit 53 has 100 elements ranging from a dementia level of 0% and a depression level of 0% to a dementia level of 90% and a depression level of 90%. First, one of the training data 40 is used for the dementia and / or depression estimation AI1. Example data 41 is input to the input unit 51. Correct answer data 42 is set in the correct answer section. If the correct answer data 42 of the training data 40 is a dementia level of 70% and a depression level of 10%, Aj (j is the element number corresponding to a dementia level of 70% and a depression level of 10%) corresponding to a dementia level of 70% and a depression level of 10% is set to 1, and A1 to A100 other than Aj are set to 0. When example data 41 is input to dementia and / or depression estimation AI1, estimated values ​​are output as numbers between 0 and 1 in Z1 to Z100 of output unit 53. The sum of the differences between Z1 to Z100 and the corresponding A1 to A100 becomes error value D. The neural network of intermediate unit 52 is adjusted to reduce this error value D. By performing this process on all of the large amounts of training data, the error value D is reduced and the accuracy of the AI ​​is improved.

[0032] In this way, according to this embodiment, information for constructing an AI that estimates the degree of dementia and / or depression in a subject can be obtained outside of a hospital, making it possible to construct an AI for estimating dementia and / or depression not only for hospital personnel but also for anyone else. In addition, by using a facial photograph of the person being diagnosed, it will be possible to operate and utilize AI to estimate the degree of dementia and / or depression of the person.

[0033] Furthermore, the dementia and / or depression estimation AI1 can be used at the pharmacy 10 to give advice about dementia and depression to pharmacy customers. A facial photograph of the person requesting AI advice is taken, and advice on dementia and depression is provided by the dementia and / or depression estimation AI 1. This will be explained along the flow in Figure 11. 1) Take a photo of the subject's face (S401). Take photos of both a neutral and smiling face. If this is difficult, just one of them will suffice. This can be done in parallel with acquiring training data 40 to improve the accuracy of the dementia and / or depression estimation AI 1. 2) Image processing is performed from the subject's expressionless facial photo 13 and the subject's smiling facial photo 14 to generate expressionless facial photo data 21 and smiling facial photo data 22, which are facial photo data 20 to be input into the dementia and / or depression prediction AI 1 (S402). 3) Input to AI (S403). If only one photo is acquired, input two identical photos. 4) The value of the output section 53 of the dementia and / or depression estimation AI1 is confirmed, and the estimated values ​​of the dementia degree and depression degree are determined (S404). 5) The degree of dementia and depression of the subject is estimated from the prescription 31 and used as the estimated prescription value (S405). 6) If the dementia or depression level estimated by the AI ​​is worse than the prescription estimate, the subject is notified of advice (S406). The same is true if the AI ​​estimates a certain degree of dementia or depression even though Prescription 31 does not contain any prescriptions for dementia or depression. In addition, in the case of hospital pharmacies that are not related to dementia or depression, such as dermatology or surgery, advice may be given without checking prescriptions, etc.

[0034] By performing such tasks, the subject's potential level of dementia and depression can be confirmed without burdening the subject, which can contribute to the early detection of dementia and depression.

[0035] The dementia and / or depression estimation AI 1 can also be used in a health check system, which estimates the user's level of dementia and depression during a health check. 1) Take a photograph of the face of the person undergoing the health checkup. Take two photographs: one with a blank expression and one with a smile. This may be easy if you explain that the photographs will be used for the health checkup, but if this is difficult, just one of the photographs will suffice. 2) Generate 20 facial photo data for AI. 3) Input the image into the AI. If only one image is needed, input two identical images. 4) The estimated levels of dementia and depression are determined from the AI ​​output results. 5) The target person will be notified of video content that corresponds to the estimated level of dementia and depression, and will be referred to specialized medical institutions. Notification may be provided on paper with a QR code (registered trademark) along with the health check results, or by sending a URL to the subject via email.

[0036] In this way, by adding the dementia and / or depression estimation AI1 to the health checkup, the subject can obtain a diagnosis result for dementia and depression without any special burden.

[0037] The dementia and / or depression estimation AI1 can also be used in a monitoring support system. When the monitoring support system is used, it estimates the degree of dementia and depression of the person being monitored. This will be explained along the flow in Figure 12. A monitoring support system is a system that uses cameras and sensors to check on elderly family members and relatives who live far away, and provides support or contacts family members if an emergency occurs. 1) In the monitoring support system, a facial photograph of the target person is acquired from video footage used to confirm the target person's safety (S501). In a monitoring support system, since it is necessary to constantly check the target person's condition, the target person is often constantly photographed with a camera, and it is easy to acquire facial photographs of the target person with a blank expression and a smiling expression. For example, a facial photograph 13 of the user with a blank expression and a facial photograph 14 of the user with a smiling expression may be acquired based on the target person's behavior, voice, etc. 2) Generate facial photo data 20 for AI from a facial photo of the subject (S502). 3) Input into AI (S503). 4) Based on the output results of the AI, estimates of the degree of dementia and depression are determined (S504). 5) If the estimated level of dementia or depression is worse than expected, the AI ​​estimate of the subject's level of dementia or depression and corresponding advice will be notified to the subject's relatives or family members (S505). Specifically, if a subject who is not diagnosed with dementia is suspected to have a high level of dementia, the system will notify pre-registered relatives and family members of the subject's possible dementia via notifications, etc. At that time, the system will also refer the subject to a medical institution to confirm dementia and provide advice on how to respond to the subject.

[0038] In this way, by adding the dementia and / or depression estimation AI1 to the monitoring support system, it is possible to estimate the possibility of dementia or depression in the subject without placing any special burden on the subject, and to take early action by contacting relatives or family members.

[0039] The dementia and / or depression prediction AI1 can also be used for over-the-counter (OTC) medical supply stores. When dealing with customers who visit the store, the AI ​​estimates the degree of dementia and depression of the target customer. 1) Take a photograph of the subject's face at the OTC. The subject can be encouraged to take a photograph as it allows for simple confirmation of dementia and depression. Take a photograph of the user's face with a neutral expression13 and a photograph of the subject's face with a smiling face14. If this is difficult, just one of the photographs will suffice. 2) Image processing is performed on the subject's facial photograph 12 so that it becomes facial photograph data to be input into the dementia and / or depression prediction AI 1. 3) Input the image into the AI. If only one image is needed, input two identical images. 4) Check the value of the output section 53 of the dementia and / or depression estimation AI1 and determine the estimated values ​​of the dementia level and depression level. 5) The subject will be informed that the results are merely estimates, and will be notified of the estimated level of dementia and depression, along with advice based on those estimates. Specifically, if a subject who is not aware of dementia or depression is suspected to have a high level of dementia or depression, the system will refer them to a medical institution to confirm dementia or depression and provide advice such as guidelines on how to respond.

[0040] In this way, by utilizing the dementia and / or depression prediction AI1 over-the-counter, subjects can receive a simple diagnosis and advice about dementia and depression without any special burden.

[0041] It can also be used when renewing your license. When renewing a driver's license, a cognitive test may be conducted in the form of questions to check the onset and severity of dementia, etc. However, as this takes a certain amount of time, it is often considered a roundabout way of doing things by those who are eligible. By using the dementia and / or depression prediction AI1 instead of or as a complement to cognitive testing, the accuracy of cognitive testing can be improved, or the amount of cognitive testing can be reduced or eliminated. Specifically, this is done as follows. 1) When a driver's license photo is taken at a license renewal venue, the photo data is acquired as a facial photo 12 of the subject. As in the past, only a driver's license photo may be taken, or a smiling facial photo 14 may be acquired with the explanation that it is for dementia confirmation. 2) Image processing is performed on the subject's facial photograph 12 to create facial photograph data 20 to be input into the dementia and / or depression prediction AI 1. 3) Input the data into the AI. If you only have a driver's license photo, input two copies of the photo data. 4) Check the value of the output section 53 of the dementia and / or depression estimation AI1 and determine the estimated values ​​of the dementia level and depression level. 5) Subjects will be notified of the test results, including AI-based estimates of the degree of dementia and depression, as a complement to the question-based cognitive test.

[0042] In this way, using AI as a means to complement cognitive tests will contribute to improving the accuracy of tests, etc. Furthermore, using AI instead of cognitive tests will reduce the burden on users.

[0043] Although an example has been described in which expressionless face photograph 13 and smiling face photograph 14 are used as sample data, training data can also be generated from only one of the photographs. Generally, many identification photos, such as driver's licenses, are expressionless, and when assessing dementia or depression from such photos, it may be necessary to use only expressionless photos. Regarding the construction of the training data, the only difference is that the two facial photographs, one with a blank expression and one with a smiling face, are replaced with one blank facial photograph 13, and the other data is the same. The advantage of using only blank facial photograph 13 is that it can be easily acquired without requiring the user to "please smile."

[0044] It is also possible to use only one smiling face photograph 14. This is because when using photographs of the subject, there may be many of them smiling. Dementia and depression can sometimes show specific characteristics when smiling, so photos of smiling faces only are also effective. By creating training data of smiling faces only and performing machine learning, it is possible to estimate illnesses from photos of smiling faces alone.

[0045] Regarding how to utilize the acquired data, a facial photo 13 of the user with a blank expression and a facial photo 14 of the user with a smiling expression can be acquired as facial photo 12 of the user, and facial photo data 20 can be generated as facial photo data 21 of the blank expression and facial photo data 22 of the smiling expression. After that, AI can be constructed by deliberately using only one of the photo data. In this case, it is possible to build three AIs from a single data set: an AI that distinguishes between expressionless and smiling faces, an AI that distinguishes from expressionless faces, and an AI that distinguishes from smiling faces.When using the system, if photos of the user with both expressionless and smiling faces can be obtained, the AI ​​that distinguishes between expressionless and smiling faces will be used, and if only one photo, such as an expressionless face, can be obtained, the AI ​​that distinguishes from expressionless faces will be used, allowing the AI ​​to be used according to the situation. [Example]

[0046] Another embodiment will be described with reference to Figures 6 to 8. The same parts as in the first embodiment will be omitted. Fig. 6 is a flow diagram showing a method for creating training data for a dementia and / or depression diagnostic AI according to this embodiment. Fig. 7 is a schematic diagram showing how facial features are extracted for a dementia and / or depression diagnostic AI, where (a) is a schematic diagram showing the area where facial features are extracted, and (b) is a diagram explaining the rising and falling of the corners of the mouth. Fig. 8 is a table showing an example of training data in the method for creating training data for a dementia and / or depression diagnostic AI according to this embodiment.

[0047] Example 1 makes it possible to build an AI for dementia and / or depression without obtaining information from medical institutions. However, in order to build AI from facial photo data of subjects, a relatively large number of samples and training data are required. Therefore, there is a need for a method to reduce the workload of building AI by using features closely related to dementia and depression obtained from facial photo data.

[0048] Specialists have pointed out that the facial expressions of patients with dementia or depression have specific characteristics. Fig. 7 shows a schematic diagram of a user's face 60. As an example for discriminating patients with dementia and depression, the degree of drooping of the corners of the mouth 61, the degree of drooping of the eyelids 62, the degree of drooping of the cheeks 63, the degree of drooping of the eyebrows 64, the degree of mouth opening 65, etc. may be considered. By quantifying each feature, it can be used as input data for AI. For example, with regard to the degree of downward movement of the corners of the mouth 61, when the corners of the mouth are at their highest point as shown in Figure 7(b), the degree of downward movement of the corners of the mouth is set to 0. When the corners of the mouth are at their lowest point as shown in Figure 7(d), the degree of downward movement of the corners of the mouth is set to 9. When the corners of the mouth are somewhat downward as shown in Figure 7(c), the degree of downward movement of the corners of the mouth is set to 5. In this way, by defining and quantifying the degree of decline for each feature 70, it becomes possible to input the data to the dementia and / or depression estimation AI 1. In addition, by quantifying data from expressionless and smiling faces, it is possible to build more accurate AI.

[0049] An example of the training data in this embodiment will be described with reference to the table in FIG. In Example 1, facial photo data 20, which consisted of expressionless facial photo data 21 and smiling facial photo data 22, were the example data, and the recognition and depression levels were the correct data. The correct data was the same as in Example 1, but the example data was feature quantity 70 extracted from the user's facial photo. Extracted from the facial photo data 20, the degree of drooping of the mouth corners 61, the degree of drooping of the eyelids 62, the degree of drooping of the cheeks 63, the degree of drooping of the eyebrows 64, and the degree of mouth opening 65 were determined as numerical values. In this example, there are 10 levels ranging from 0 to 9. As the feature amount 70, a group of feature amounts 71 of neutral expressions and a group of feature amounts 72 of smiling expressions are used. The feature amount 70 is extracted only from the facial photograph data 20, so the burden on the user P is not increased.

[0050] This embodiment will be described along the flow of FIG. The process of obtaining the contents of the prescription 31 etc. from the user (S201), obtaining a facial photograph 12 of the user (S202), and estimating the degree of dementia and depression from the type and amount of medication listed on the prescription 31 (S203) is the same as in Example 1. Facial feature amounts 70 are extracted from the facial photograph and generated as example data (S204). In this step, similar to the first embodiment, facial photograph data 20 is generated from the user's facial photograph 12. Thereafter, facial feature amounts 70 are extracted from expressionless facial photograph data 21 and smiling facial photograph data 22, which are the facial photograph data 20, and generated as example data. For example, in the example of training data using feature quantity 70 in FIG. 8, in the case of number 1, first, image processing is performed on expressionless face photo data 21, which is the source of the example data, to obtain a mouth corner drooping degree of 8, eyelid drooping degree of 5, cheek drooping degree of 3, eyebrow drooping degree of 5, and mouth opening degree of 2. Next, image processing is performed on smiling face photo data 22 to obtain a mouth corner drooping degree of 7, eyelid drooping degree of 5, cheek drooping degree of 2, eyebrow drooping degree of 5, and mouth opening degree of 2. The method of generating correct answer data is the same as in Example 1. In this way, one piece of training data 40 is completed (S205). The same process is performed on the other data to generate training data 40.

[0051] In this way, by using facial features 70 as example data, the example data becomes specialized for dementia and / or depression, and the amount of input data is reduced, so the accuracy of the AI ​​can be improved with a small amount of training data.

[0052] Furthermore, by using the facial feature amount 70 as the example data, the generation of the training data 40 can be partially simplified. Dementia and depression experts have extensive knowledge and experience regarding facial features70 and the degree of dementia and depression. Therefore, with the assistance of experts, it is possible to create a certain amount of training data without using user photos or prescriptions31. In other words, by estimating the feature values ​​70 that are expected at a specific dementia level and depression level, using the estimated feature values ​​70 as example data, and then using the predicted dementia level and depression level as correct answer data and performing machine learning, it becomes easier to obtain data.

[0053] In other words, by asking experts to list the characteristics that are likely to be present in dementia or depression when the face is expressionless, it is possible to build up a certain amount of data without collecting photographs. Similarly, in the case of smiles, by asking people to list the characteristics that distinguish them from a smile and a neutral expression, it is possible to build up some data.

[0054] In AI machine learning, there may be a large discrepancy between the AI ​​output and the correct answer in the early stages of learning. By performing machine learning using training data created in advance with the support of experts, when machine learning is performed using user data, the discrepancy between the AI ​​output and the correct answer can be reduced, allowing the AI ​​to converge more quickly.

[0055] The explanation will be given along the flow in FIG. The facial feature amount 70 and the degree of dementia or depression are quantified by an expert (S301). For example, the facial feature amount 70 may be a combination of the degree of drooping of the mouth corners 61, the degree of drooping of the eyelids 62, the degree of drooping of the cheeks 63, the degree of drooping of the eyebrows 64, and the degree of mouth opening 65 when the face is expressionless, and the degree of drooping of the mouth corners 61, the degree of drooping of the eyelids 62, the degree of drooping of the cheeks 63, the degree of drooping of the eyebrows 64, and the degree of mouth opening 65 when the face is smiling, and the degree of dementia or depression is quantified between 0 and 100%. An example is shown in Figure 10(a). This example is a table showing the estimated dementia level when the change in the degree of drooping of the corners of the mouth and the degree of drooping of the eyelids when smiling from a neutral expression is changed on a 10-point scale, assuming that the cheeks are drooped to a degree of 5, the eyebrows are drooped to a degree of 4, the mouth opening is open to a degree of 2, the eyelids are drooped to a degree of 4, and the corners of the mouth are drooped to a degree of 5 when smiling from a neutral expression. The range of change in the degree of drooping of the mouth corners is from -5 to 4, since the degree of drooping of the mouth corners when the face is expressionless is 5. The range of change in the degree of drooping of the eyelids is from -4 to 5, since the degree of drooping of the eyelids when the face is expressionless is 4. For example, if the change in the degree of eyelid downturn when smiling is 0 and the change in the degree of mouth corner downturn is -3, the smile is properly expressed and the recognition rate is considered low. As an example, the maximum and minimum values ​​of the change are estimated, and a proportional calculation is performed from there, resulting in an estimated value of 34%. Similarly, if the change in the degree of downward movement of the corners of the mouth when smiling is +2, the corners of the mouth are further downwards despite the smile, and it is considered that the smile is not being expressed appropriately, so the recognition rate is considered to be high. As an example, the maximum and minimum values ​​of the amount of change are estimated, and a proportional calculation is performed from there, resulting in an estimated value of 59%.

[0056] Such a table is created for all the feature quantities 70. Using these, the feature levels are set as example data, and the estimated dementia and depression levels are set as correct data (S302). Figure 10(b) shows an example of a training data group. When all facial feature values ​​70 are changed from 0 to 9, the correct data is listed as dementia level XX% and depression level XX%. The example in Figure 10(b) is an example of a single feature group. If a neutral feature value 71 and a smiling feature value 72 are used, the number of classifications will be even larger. The training data is completed through expert guidance (S303). Next, experts are asked to generate dementia and depression levels, focusing on combinations of 70 feature values ​​that are likely to actually exist (S304). This process increases the amount of training data that is closer to reality. This completes the initial training data for machine learning (S305).

[0057] By generating training data in this way, the amount of data to be collected can be reduced and the convergence of AI machine learning can be accelerated. [Example]

[0058] Another embodiment will be described with reference to Figures 13 to 17. The same parts as in the first embodiment will be omitted. Fig. 13 is a system diagram of Example 3 according to the present invention. Fig. 14 is a diagram of the screen of a mobile terminal according to Example 3 according to the present invention, in which (a) is a screen for asking a question to confirm the level of energy level, (b) is a screen for requesting permission to acquire teacher data, (c) is a screen for displaying guidance for acquiring a facial photograph of the user by the mobile terminal, and (d) is a screen for displaying guidance for confirming the medications taken by the user by the mobile terminal. Fig. 15 is a table showing information for generating training data according to Example 3 of the present invention. Fig. 16 is a flow diagram showing a training data acquisition procedure according to Example 3 of the present invention. Fig. 17 is a flow diagram showing an example of a mobile terminal in the dementia and / or depression prediction AI system according to the present invention.

[0059] According to Example 1, it is possible to build an AI for dementia and / or depression by generating training data from information at pharmacies without obtaining information from medical institutions. However, in order to build AI from facial photo data of subjects, a relatively large number of samples and training data are required. Therefore, there is a need for a method to obtain training data on dementia and depression, not just for pharmacies.

[0060] In recent years, apps for mobile devices such as smartphones have been developed that provide advice on diet and exercise to prevent dementia. Therefore, in this embodiment, it is possible to obtain information that serves as training data for AI from an application on a mobile terminal used by a user. The training data includes at least a facial photograph of the user, a prescription, instructions, or information that can be used to infer the user's dementia and / or depression. The information that can be used to infer the user's dementia and / or depression is obtained by asking the user questions. By obtaining information related to training data from a mobile device app in this way, it is possible to obtain sample data from many people, not just pharmacies.

[0061] FIG. 13 shows a system diagram of this embodiment. The dementia and / or depression estimation AI 1 in this embodiment is a system that obtains training data from a mobile terminal of a user via a network. The dementia and / or depression prediction AI1 comprises a machine learning system 50, a management unit 80, a server 81, and a mobile terminal 90.

[0062] The machine learning system 50 is the core part of the AI, as in the first embodiment. There is a learning phase where the AI ​​learns from training data, and a utilization phase where the intermediate part constructed through learning is used to provide an AI answer. During learning, training data sent from the management unit 80 is used to learn, and intermediate adjustments are made to improve the accuracy of the AI. When in use, the system estimates the level of awareness and depression from the photograph data sent from the management unit 80 and returns the estimated values ​​to the management unit 80. The management unit 80 is a part that acts as a bridge between the mobile terminal 90 and the machine learning system 50. The management unit 80 writes data related to the generation of teacher data to and reads data from the server 81. The information necessary for generating teacher data is received from the mobile terminal 90 via the Internet N, and some of it is processed and adjusted, and then stored in the server 81, including the information that has been converted into teacher data. The information to be processed may be, for example, a photograph of a medicine taken by the user. If the received photograph of a medicine is a medicine notebook, the management unit 80 reads the name of the medicine written in the notebook and sets it as the medicine content. If the received photograph of a medicine is a medicine bag or the medicine itself, the name of the medicine is identified from the name on the surface of the medicine bag or the name on the medicine sheet. Thereafter, similarly to the first embodiment, the recognition level and the depression level are estimated and calculated using the recognition level and depression level calculation table for the drug name as shown in FIG. Facial photographs are also processed and adjusted in the same way. Since facial photographs taken by users may not be suitable as training data depending on the shooting environment and the user's shooting method, the management unit 80 performs the same processing as in the first embodiment, such as simplifying the size of the face and the background. The processed photographs are stored in the server 81. The management unit 80 reads out the values ​​of the facial photograph, recognition level, and depression level, which are the training data stored in the server 81, and sends the training data to the machine learning system 50 that is currently learning.

[0063] The server 81 mainly stores user information 83, information acquired from the user 84, and information related to the training data that has been processed to store example data 41 of the training data and correct answer data 42. These are collectively referred to as server management data 82. The server management data 82 will be described with reference to FIG. A user ID is assigned to each user, and information is managed by the user ID. For example, the user with user ID 1001 is Yamada Taro, and the user with user ID 1056 is Suzuki Hanako. Each person's date of birth, gender, and medical history are recorded, and this data is generally not changed after registration. A data number and the date of data acquisition are assigned to each collection of information 84 acquired from the user by the management unit 80. The data number is unique and there are no duplicates. Age, a photo of a face with a neutral expression, a photo of a face with a smiling expression, a photo related to medication, and awareness and depression levels estimated from questions posed by the app are sent to the management unit 80 and recorded in the server 81. Medications may be photographs of a medicine notebook, a medicine packet, or a medicine sheet. The management unit 80 adjusts and processes the information from the mobile terminal 90, records the adjusted facial photograph as example data 41, and records the contents of the medicine and the recognition and depression values ​​estimated from the medicine as correct answer data in the server 81. Information managed under one data number corresponds to one piece of training data. As information related to training data, multiple data numbers may be stored for one user. This is because the user's status may change depending on the time of acquisition.

[0064] The server management data 82 records the awareness and depression level information generated from the drug information, and the awareness and depression level estimated from the questions posed by the application. The awareness and depression level information generated from the drug information may be used as the correct answer data 42, or the awareness and depression level estimated from the questions posed by the app may be used as the correct answer data. Alternatively, the correct answer data may be the average of the data generated from the drug information and the data estimated from the questions posed by the app. In addition, if the photo of the medication taken by the user is unclear, the awareness and depression levels estimated from the questions posed by the app may be used as an alternative.

[0065] The mobile terminal 90 connects each user with the management unit 80 . The mobile terminal 90 mainly comprises a display unit 91, a control unit, a storage unit, an operation unit, a communication unit, and a camera unit, all of which are not shown. The control unit is a part that controls the entire mobile terminal 90. While the application 92 is running, it also performs control according to the content of the application. The control unit acquires the user's operation details from the operation unit, displays characters, symbols, images, etc. on the display unit 91, takes pictures with the camera, and instructs the communication unit to send and receive data to and from the network. It also writes and reads temporary data to and from the memory unit, and reads out applications 92. The storage unit stores programs for operating the control unit and temporarily stores variables, data, etc. used for control. The application 92 of the embodiment is also stored as one of the programs.

[0066] The camera is a camera that takes pictures of the surroundings of the mobile terminal 90. In this embodiment, the camera takes pictures of the user's face and the medicine notebook. The communication unit is a communication unit that communicates with the Internet N. It connects to the Internet N using a mobile communication system such as WiFi or 4G. The operation unit is the mechanism that allows the user to operate the terminal. It is operated using buttons and a touch panel. The display unit 91 is a part that displays text and images such as guidance and messages, and serves as a screen display for applications and a monitor for captured images.

[0067] In this embodiment, the application 92 is assumed to be an exercise recommendation application for preventing dementia in the elderly, or a health check application. Some of these apps are configured to have a section that acquires information about training data. Users who install dementia prevention apps are people who are concerned about their risk of developing dementia. In other words, they are people who are at a high risk of developing dementia or depression. Therefore, training data obtained from mobile devices with such apps installed is highly likely to be training data that indicates the early stages of dementia and depression, and by using it for AI training, the accuracy of AI in estimating dementia and depression can be improved.

[0068] The flow of acquiring information related to the training data will be described with reference to the flow in FIG. In this embodiment, the device functions as a part of an application 92 in a mobile terminal 90. Examples of the application 92 include an exercise recommendation app for preventing dementia in the elderly and a health check app. When a user launches application 92, it is checked using in-app data, etc., whether this is the first time the application has been launched or whether three months have passed since the last request to obtain information related to teacher data (S601). If not, the result is NO and the application proceeds to normal operation (S608). If so, an information provision request screen 94 for AI learning is displayed on the screen to request cooperation in AI learning in S602 (FIG. 14(b)).

[0069] If the user selects "Yes" in S603, the information acquisition proceeds. If the user selects "No," the application proceeds to normal operation (S608).

[0070] As a guidance for taking a face photo, a face photo guidance screen 95 is displayed on the display unit 91, and a face photo with a neutral expression and a face photo with a smile are taken (S604, FIG. 14(c)). If it is difficult to specify a face with a neutral expression, a face photo with a normal, natural face may be specified. As guidance for obtaining information about medicine, a medicine information photographing guidance screen 96 is displayed on the screen, and a photograph of the page of the medicine notebook showing the medicine information, the medicine envelope, etc. is taken (S605, FIG. 14(d)).

[0071] Questions related to dementia and depression (Figure 14(a)) are asked, and the awareness and depression levels estimated from the user's answers are acquired as awareness and depression levels according to the questions (S606). The awareness and depression levels are estimated simply using an app on the mobile device. Alternatively, the content of the question may be notified to the management unit 80 via the Internet, and an appropriate estimation result may be obtained from the management unit 80. The user's ID number, a facial photo, a photo related to medication, and the level of awareness and depression estimated from answers to questions, all of which are managed under a single data number, are transmitted to the management unit 80 via the Internet N (S607). Once the information has been acquired and transmitted, the application proceeds to normal operation (S608). By performing this series of operations, information related to the training data can be obtained extremely easily via the mobile terminal 90.

[0072] Furthermore, the dementia and / or depression estimation AI1 can be used on the mobile terminal 90 to give advice to the person to be diagnosed. The system is designed so that a photograph of the subject's face taken with the subject's mobile device can be obtained via the network, and advice is provided based on the results of the AI ​​system's analysis of the obtained photograph. For example, a photo of the target person's face is taken by calling from an application 92 on a mobile terminal 90, and advice on dementia and depression is given based on the estimated dementia and depression scores by AI. This will be explained along the flow in Figure 17.

[0073] 1) The application 92 in the smartphone, which is the mobile terminal 90, proposes a simple diagnosis of dementia and depression to the subject (S701). The suggestions are from an exercise recommendation app to prevent dementia in the elderly and a health check app, so subjects can receive a simple diagnosis without feeling uncomfortable.

[0074] 2) The subject takes a photo of their face according to the guidance of the application 92 (S702). It is preferable to take two photos of the face, one with a neutral expression and one with a smiling face, as this can improve the accuracy of estimating dementia and depression. By taking a picture while viewing an image of the user on the mobile terminal 90, the user can take a picture while being aware of their own expressionless and smiling faces, and thus can take a more appropriate facial photograph.

[0075] 3) The application 92 transmits the facial photograph to the management unit 80 via the Internet N. The management unit 80 processes the facial photograph of the subject by removing the background, adjusting the size of the face, etc., so that the facial photograph data becomes the same as the sample data (S703).

[0076] 4) The management unit 80 inputs the photo data to the input unit 51 of the machine learning system 50, which is the core of the AI ​​(S704). 5) The values ​​of the output section 53 of the machine learning system 50 are checked, and the estimated values ​​of the dementia level and depression level are determined (S705). 6) The estimated value is transmitted to the mobile terminal 90 via the Internet N (S706).

[0077] 7) The subject will be notified of the AI-estimated level of dementia and depression, and advice based on the AI ​​estimate will be displayed on the screen. For example, the advice may read, "The dementia level is estimated to be 70%. We recommend that you seek medical advice as soon as possible." The content of the advice may be generated by the management unit 80, or the content may be displayed on the mobile terminal 90 in advance according to the estimated value. Furthermore, in the case of the second or subsequent simple diagnosis, the previous estimated values ​​may also be displayed, and advice may be given depending on the increase or decrease in the value (S707). For example, the advice may be, "The degree of dementia is estimated to be 30%, which is better than the previous estimated value of 40%. Please continue to exercise to prevent dementia."

[0078] By making such a suggestion, the subject's potential level of dementia and depression can be confirmed without burdening the subject, which can contribute to the early detection of dementia and / or depression. [Industrial Applicability]

[0079] The dementia and / or depression prediction AI system and training data creation method according to the present invention can be used not only as a technology for building dementia and / or depression prediction AI outside of hospitals, but also for other cases in which the disease manifests as a characteristic facial expression. Therefore, the present invention is believed to have great industrial applicability. [Explanation of symbols]

[0080] 1. Dementia and / or depression prediction AI 10. Pharmacy 11. Filming equipment 12 User's photo 13 Expressionless Face Photos 14 Smiling Face Photos 20 Facial photo data 21 Expressionless face photo data 22 Smiling face photo data 30 Medicine notebook 31 Prescription 32 Drug Data 40 Training data 41 Example data 42 Correct data 43 Calculation table 50 Machine Learning Systems 51 Input section 52 Middle section 53 Output section 54 Correct Answer 60 User's face 61 Degree of downward movement of the corners of the mouth 62 Eyelid droopiness 63 Cheek droopiness 64 Eyebrow droop 65 Opening 70 features 71 Neutral features 72 Smile Features 80 Management Department 81 servers 82 Server Management Data 83 User information 84 Information obtained from users 90 Mobile Devices 91 Display section 92 Applications 93 Health Check Questions 94 Information request screen for AI learning 95 Face photo guidance screen 96 Drug Information Photo Guidance Screen P User D error N Internet

Claims

1. A method for creating training data to be used in AI that estimates the degree of dementia and / or depression from a user's facial photograph, comprising:

1. A system for estimating the degree of dementia and / or depression, The information that serves as training data for the AI ​​can be obtained from the location where the user picks up the prescription medicine or from an application on the mobile device used by the user. The training data includes at least a facial photograph of the user, a prescription, an instruction sheet, or information that can be used to estimate the user's dementia and / or depression, Using either a facial photograph of the user with a neutral expression or a facial photograph of a smiling expression, or both, and extracting facial features from the facial photograph to generate example data of training data; The example data of the training data includes the user's age, sex, medical history, and family medical history, The degree of dementia and the degree of depression are estimated from the type and amount of the medicine written in the prescription or instruction sheet and used as correct answer data for training data. Furthermore, the feature amounts expected at a specific dementia level and depression level are estimated, and the estimated feature amounts are used as the example data. At this time, the predicted dementia level and depression level are used as the correct answer data. A teacher data creation method for creating teacher data for machine learning.

2. An AI system using teacher data created by the teacher data creation method according to claim 1, A dementia and / or depression estimation AI system that estimates the degree of dementia and depression of a subject based on a facial photograph of the subject.

3. The dementia and / or depression prediction AI system described in claim 2, characterized in that a facial photo of the subject is taken at the pharmacy and advice is provided based on the results of the AI ​​system's response to the photo.

4. The dementia and / or depression prediction AI system described in claim 2, characterized in that a photo of the subject's face is taken and, depending on the results of the AI ​​system, the subject is guided to video content.

5. The dementia and / or depression prediction AI system described in claim 2, characterized in that a facial photo of the subject is taken and, depending on the results of the AI ​​system's analysis, a referral to a specialized medical institution is made.

6. The dementia and / or depression prediction AI system of claim 2, characterized in that a photograph of the subject's face is taken and advice is provided to the family based on the results of the AI ​​system's analysis.

7. The dementia and / or depression prediction AI system described in claim 2, characterized in that a facial photo of the subject is taken at an OTC medical supply store and advice is provided based on the results of the AI ​​system's response to the photo.

8. The dementia and / or depression prediction AI system described in claim 2, characterized in that a facial photograph of the subject taken with a mobile device used by the subject can be obtained via a network, and advice is provided depending on the results of the AI ​​system's analysis of the obtained facial photograph.

Images