Ai system for estimating dementia and / or depression, and method of creating teacher data

Abstract

To provide an AI system capable of estimating severity of dementia and / or depression in a subject on the basis of information obtained outside a hospital.SOLUTION: A method of creating teacher data for an AI system for estimating severity of dementia and / or depression is provided. Information to be used as teacher data for the AI system can be acquired from a site where a user receives prescribed drugs or from an application on a mobile terminal used by the user. The teacher data at least includes a face picture of the user, and a prescription, instruction, or information that enables estimation of dementia and / or depression in the user.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to AI for diagnosing medical conditions, and more particularly to technology for estimating dementia and / or depression based on facial photographs. [Background technology]

[0002] Early detection and treatment of dementia and depression are important. One method for detecting symptoms is to use AI to identify the subject's facial photograph and estimate the degree of dementia or depression. However, building such a system generally requires information managed by the medical institution, and it is not easy for other businesses to do so. Therefore, there was a need for a method that uses AI to estimate the degree of a subject's dementia or depression from information obtained outside of a hospital.

[0003] To address such problems, various technologies have been proposed in the past. For example, a method of assisting doctors in diagnosis by image analysis using AI (see Patent Document 1) has been proposed and is a publicly known technology. More specifically, this is a device that uses training data consisting of image data of an image of a subject, findings data, and lifestyle and dietary habits information of the subject, inputs image data and subject information, performs machine learning to output findings data, and outputs the findings data using this AI. However, since data acquisition in this prior art is carried out at the hospital, other businesses cannot handle it, and the above problem is not resolved. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-108800 Summary of the Invention [Problem to be solved by the invention]

[0005] In consideration of the above problems, the present invention aims to build an AI that can estimate the degree of dementia and / or depression of a subject based on information obtained at a location other than a hospital. [Means for solving the problem]

[0006] In order to solve the above problems, the present invention provides a method for creating training data used by an AI that estimates the degree of dementia and / or depression from a facial photograph of a user, wherein in a system for estimating the degree of dementia and / or depression, information that serves as training data for the AI ​​can be acquired from a location where a user receives medicine according to a prescription, or from an application on a mobile device used by the user, and the training data includes at least a facial photograph of the user, and a prescription, an instruction sheet, or information that can estimate the dementia and / or depression of the user, A method is adopted in which either one or both of a photograph of a face with a neutral expression or a photograph of a face with a smiling expression are used, facial features are extracted from the photograph to generate example data of training data, the features having at least the degree of drooping of the corners of the mouth, the degree of drooping of the eyelids, the degree of drooping of the cheeks, the degree of drooping of the eyebrows, and the degree of mouth opening, and further the example data of the training data includes the user's age, sex, medical history, and family medical history, and the level of dementia and depression are estimated from the type and amount of medication written on the prescription or instructions and used as correct answer data for the training data, thereby creating the training data for machine learning.

[0007] Furthermore, the present invention estimates the features expected at a specific level of dementia and depression, uses the estimated features as the example data, and employs a means for performing machine learning by using the predicted level of dementia and depression as the correct answer data.

[0008] Furthermore, the present invention provides an AI system that uses training data created by the training data creation method, and employs a means for estimating the degree of dementia and depression of a subject based on a facial photograph of the subject.

[0009] Furthermore, the present invention employs a means for taking a facial photo of the subject at the pharmacy and providing advice based on the results of the AI ​​system's response to the photo.

[0010] Furthermore, the present invention employs a means for taking a facial photo of the subject and directing the subject to video content depending on the results of the AI ​​system.

[0011] Furthermore, the present invention employs a means for taking a facial photo of the subject and, depending on the results of the AI ​​system, referring the subject to a specialized medical institution.

[0012] Furthermore, the present invention employs a means for taking a facial photo of the subject and notifying the family of advice based on the results of the AI ​​system.

[0013] Furthermore, the present invention employs a means for taking a facial photo of the subject at an OTC medical store and notifying the subject of advice based on the results of the AI ​​system's analysis.

[0014] Furthermore, the present invention employs a means for acquiring a facial photograph of the subject taken with a mobile device used by the subject via a network, and for providing advice based on the results of the AI ​​system's application of the acquired facial photograph. Effect of the Invention

[0015] According to the dementia and / or depression prediction AI system and teacher data creation method of the present invention, information needed to build an AI that predicts the level of dementia or depression in a subject can be obtained outside of a hospital, so that an AI that predicts dementia and / or depression can be built by anyone, not just hospital personnel. In addition, by utilizing applications on the mobile devices used by users, it is possible to easily obtain a large amount of information needed to build AI that can predict dementia and / or depression. [Brief description of the drawings]

[0016] [Figure 1] FIG. 2 is a flow chart showing an embodiment of a teacher data creation method according to the present invention. [Diagram 2] FIG. 1 is a schematic diagram showing an embodiment of a teacher data creation method according to the present invention. [Diagram 3] 1 is a calculation table showing an example of generating correct answer data in the teaching data generating method according to the present invention. [Figure 4] 1 is a table showing an example of teacher data in a teacher data creation method according to the present invention; [Diagram 5] FIG. 2 is a schematic diagram showing an example of the operation of the AI ​​in the dementia and / or depression prediction AI system according to the present invention. [Figure 6] FIG. 11 is a flow chart showing another embodiment of the teaching data creation method according to the present invention. [Figure 7] FIG. 1 is a schematic diagram showing an embodiment of extracting facial features in the dementia and / or depression prediction AI system according to the present invention. [Figure 8] FIG. 11 is a table showing another embodiment of the teacher data in the teacher data creation method according to the present invention. [Figure 9] FIG. 2 is a flow diagram showing an example of teacher data in the teacher data creation method according to the present invention. [Figure 10] FIG. 2 is an explanatory diagram showing an example of teacher data in the teacher data creation method according to the present invention; [Figure 11] FIG. 1 is a flow diagram showing an example of the dementia and / or depression prediction AI system according to the present invention at a pharmacy. [Figure 12] FIG. 1 is a flow diagram showing an embodiment of monitoring support in the dementia and / or depression prediction AI system according to the present invention. [Figure 13] FIG. 11 is a system diagram of another embodiment of the present invention. [Figure 14] FIG. 11 is a diagram of a screen of a mobile terminal according to another embodiment of the present invention. [Figure 15] 13 is a table showing information for generating training data in another embodiment of the present invention. [Figure 16]FIG. 11 is a flowchart showing a procedure for acquiring teacher data according to another embodiment of the present invention. [Figure 17] FIG. 1 is a flow chart showing an example of the dementia and / or depression prediction AI system according to the present invention on a mobile terminal. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] The greatest feature of the dementia and / or depression prediction AI system and training data creation method of the present invention is that it can obtain information for estimating the level of dementia or depression in a subject in a location other than a hospital, thereby making it possible to build an AI that can predict dementia and / or depression not only for people affiliated with hospitals, but also for anyone other than hospital personnel. Hereinafter, embodiments of an AI system for predicting dementia and / or depression and a method for creating teacher data according to the present invention will be described with reference to the drawings.

[0018] Note that the overall configuration and the configuration of each part of the dementia and / or depression prediction AI system and teacher data creation method described below are not limited to the examples described below, and can be modified as appropriate within the scope of the technical idea of ​​the present invention, i.e., within the scope of configurations that can achieve the same functional effects. In addition, as the present invention is a dementia and / or depression prediction AI system and a teacher data creation method, it naturally includes a dementia prediction AI system, a depression prediction AI system, and a dementia and depression prediction AI system; however, the embodiments will focus on examples of cases where dementia and depression are predicted. In the case of a dementia prediction AI system, the system can be easily constructed by excluding the depression factor, which is another factor. The same is true for a depression prediction AI system. In addition, in the embodiments, a user refers to a person who cooperates in providing training data, and a subject refers to a person whose medical condition is estimated using AI. EXAMPLES

[0019] The present invention will be described with reference to FIGS. FIG. 1 is a flow diagram showing an embodiment of the teacher data creation method according to the present invention. FIG. 2 is a schematic diagram showing an embodiment of the teacher data creation method according to the present invention, where (a) is a schematic diagram of data acquisition work performed in a pharmacy, (b) is a schematic diagram of the work of creating example data from a user's face photo, and (c) is a schematic diagram of the work of creating correct data from a prescription or the like. FIG. 3 is a calculation table showing an embodiment of generating correct answer data in the teacher data creation method according to the present invention, where (a) is a table for estimating and calculating the dementia degree and depression degree, and (b) is a calculation table for estimating and calculating the dementia degree and depression degree by actually inputting the type and amount of medicine. FIG. 4 is a table diagram showing an embodiment of teacher data in the teacher data creation method according to the present invention, where (a) is an example in which the correct answer data is in 10% frequency increments, (b) is a case where the frequency is 1 or 0, and (c) is a case where age, etc. are included in the example data. FIG. 5 is a schematic diagram showing an example of the operation of the AI ​​in the dementia and / or depression prediction AI system according to the present invention, where (a) shows an example of predicting dementia, etc. using AI, and (b) shows the process of machine learning using AI.

[0020] Dementia and / or Depression Prediction AI1 is an AI that estimates the degree of dementia and depression, and its prediction capabilities are improved through machine learning. One method for building a dementia and / or depression prediction AI1 is to generate training data and have the AI ​​learn a large amount of it to make highly accurate predictions. Training data is data used to train AI in machine learning. Training data is often generated by pairing example data with correct answer data. Example data is data that represents the phenomenon that you want the AI ​​to estimate, and answer data is the data that you want the AI ​​to answer.

[0021] When making an AI predict whether or not a person has a disease, for example, the subject's symptoms are used as example data and the subject's disease name is used as correct answer data. By having the AI ​​learn a large amount of such training data, an AI that can predict diseases is constructed.

[0022] When examining illnesses such as dementia or depression, doctors often look at facial expressions as part of their diagnosis. When considering a system that uses AI to predict dementia or depression, the subject's facial expressions can be obtained by photographing them with a camera, but the subject's illness name is managed by a doctor in a medical record, etc., and it is generally difficult to take that data outside the hospital.

[0023] Therefore, in this embodiment, the name and severity of the subject's illness is estimated from the prescription, etc. at the pharmacy, and training data is obtained. In other words, this is a method for creating training data in which information that serves as training data for the AI ​​is obtained at the location where the user receives the prescription medication, and the training data includes at least a photograph of the user's face and the prescription or instructions. The information that serves as the teacher data includes information that serves as the basis of the teacher data and information that is used as the teacher data. Prescriptions, etc. include at least prescriptions, medicine notebooks, and instructions. Instructions, also called medication instructions, are documents in which a doctor instructs a guardian or facility on the time and amount of medication to be administered. The procedure for obtaining training data will be explained using Figures 1 and 2. Fig. 1 shows a flow for generating teacher data 40. Fig. 2 shows a procedure for generating teacher data 40. The information is collected by the pharmacy 10. Here, the pharmacy includes both pharmacies that are independent of the hospital and pharmacies that are installed within the hospital. The data handled is the same in both cases. First, the contents of the prescription 31 are obtained from the user P at the pharmacy 10. The contents of the medicine notebook 30 and the instruction sheet may be obtained together with the prescription 31. Since the user P has come to the pharmacy 10 to obtain the medicine listed on the prescription 31 and the medicine notebook 30, obtaining the prescription 31 can be done easily. Apart from dispensing medication, permission is obtained to use information from prescriptions 31, medication records 30, and instructions to build dementia and / or depression prediction AI 1, and information is acquired (S101, Figure 2(a)). Next, in order to construct the dementia and / or depression prediction AI1, permission is obtained to obtain a facial photograph 12 of the user P, a photograph of the user P is taken with a photographing device 11, and the facial photograph 12 of the user is obtained (S102, FIG. 2(a)). As the user's facial photograph 12, a facial photograph 13 of the user with a neutral expression and a facial photograph 14 of the user with a smiling expression are acquired. An expressionless face can be said to be a natural, ordinary face. It is said that in cases of dementia or depression, the eyes tend to look lifeless, the eyelids droop, and the overall expression tends to be lacking. In addition, the smile may be unnatural or may be no different from an expressionless face. Therefore, by using facial photos of the user with both a blank expression and a smiling expression, the difference between normal people and those with dementia or depression can be emphasized. When taking a photo with a blank expression, if it is difficult to say, "Please make sure you have a blank expression," you can use a phrase such as, "Please let me take a photo of your face in a natural, normal way." By carrying out S101 and S102, the acquisition of information from the user is completed.

[0024] Next, the presence and severity of dementia or depression is estimated from the type and amount of medicine written on the prescription 31, etc. (S103, FIG. 2(b)). A pharmacist or the like can make an estimate to some extent, but to reduce variability, a calculation table 43 is used. An example of the calculation table is shown in FIG. 3(a). Correct answer data is generated by the calculation table 43. Next, facial photo data 20 is generated from the user's facial photo 12. Generally, facial photos taken outside a studio or the like have different angles of view, resolutions, backgrounds, etc., so by performing image correction so that the angles of view, resolutions, and backgrounds are similar, appropriate facial photo data 20 can be generated as example data 41 (S104, FIG. 2(c)). The facial photograph data 20 consists of facial photograph data 21 of a face with no expression and facial photograph data 22 of a face with a smiling face. Through this series of steps, one piece of training data is completed (S105). In other words, the process involves using a photo of the user's face as example data for the training data, and using the level of dementia and depression estimated from the type and amount of medication listed on the prescription or instructions as correct answer data for the training data, to generate training data for machine learning. The accuracy of machine learning can be improved by increasing the amount of training data.

[0025] A calculation table 43 is used to estimate the degree of dementia and depression from the prescription 31, medicine notebook 30, and instructions. Calculation Table 43 has a column for recording the type and amount of medication, which is to be filled in according to the user's information. There are columns for dementia and depression, and coefficients are listed for each type of medication. Drugs that are closely related to each disease have high coefficients, and drugs that are less related have low coefficients. An evaluation score is determined by entering the type and amount of medication. The severity of each illness can be determined by adding up the evaluation scores. To make the data correct, normalization is required, such as from 0 to 100%, so normalization is performed using a predetermined number. There are two ways to calculate correct data: by expressing it as a percentage, or by expressing it as 1 or 0. Figure 3(b) shows a calculation example using specific numbers. In this example, since there are many medications related to dementia, the total evaluation score for dementia is 23 points, and the total evaluation score for depression is 4 points. As an example, normalization is performed by 30, and the evaluation of dementia is 76.7% and the evaluation of depression is 13.3%. As correct answer data, correct answer data 1 with 10 points and correct answer data 2 with two points, 1 and 0, are generated. Correct data 1 has a dementia level of 70% and a depression level of 10%, and correct data 2 has a dementia level of 1 and a depression level of 0. Through this process, correct data on the dementia level and depression level are generated.

[0026] Fig. 4 shows an example of training data. Fig. 4(a) uses facial photo data 20 as example data of the training data, and as the facial photo data 20, expressionless facial photo data 21 and smiling facial photo data 22 are used. The correct answer data for both dementia and depression is 10-level data. The correct answer data has 100 correct answer elements, 10 levels x 10 levels, so a correspondingly large amount of training data is required for machine learning. Figure 4(b) shows the case where the correct answer data for the training data is two-level data for both dementia and depression. The correct answer data is two levels x two levels, resulting in four correct answer elements, so the amount of training data for machine learning may be less than that in the case of Figure 4(a). Figure 4(c) shows an example where the user's age, gender, and medical history are added in addition to the facial photo data. Family medical history can also be added. Using a lot of data can make estimation easier. However, when using AI, it is necessary to input the same data as during learning, so in this case, it is necessary to input the user's age, gender, and medical history in addition to the facial photo data, which reduces versatility.

[0027] An overview of the operation of AI learning and utilization will be described with reference to Figure 5. A machine learning system 50 shows an example of deep learning using a neural network. The dementia and / or depression prediction AI1 consists of an input section 51, an intermediate section 52, and an output section 53. During machine learning, a correct answer section 54 is added. The input section 51 has elements X1 to Xn, which are the amount of information to be input to the AI. The intermediate section 52 is a section that inherits information from the input layer and performs various calculations. The more layers the intermediate section 52 has, the more complex the analysis that can be performed. The calculation results are sent to the output section 53. The output section 53 is a section that outputs the answer, and has elements Z1 to Zm, which are the amount of the answer. The value of each element is, for example, between 0 and 1, and the element with the highest value is estimated to be the answer (Figure 5(a)). The correct answer unit 54 is a part that sets the correct answer data 42 to be given to the dementia and / or depression prediction AI 1 during machine learning, and has elements A1 to Am. The number of elements is the same as that of the output unit 53. Of A1 to Am, the numerical value of the correct answer element is set to 1, and the numerical value of the other elements is set to 0. The differences between the numerical values ​​of Z1 to Zm and A1 to Am are calculated, and the sum of these values ​​becomes the error D. Minimizing the error D improves the accuracy of the AI ​​(Figure 5(b)).

[0028] The operation of the dementia and / or depression prediction AI 1 will be simplified for the purpose of explanation, and an example of an AI that predicts whether or not a person has dementia will be described. Regarding the output unit 53, the AI's answer is either dementia or no dementia. Therefore, the elements of the output unit 53 are, for example, Z1 "has dementia" and Z2 "does not have dementia". When face photograph data 20 including expressionless face photograph data 21 and smiling face photograph data 22 is input, the number of elements X of input unit 51 becomes an element according to the resolution. When face photo data 20 is input to the AI's input unit 51, elements Z1 and Z2 of the output unit 53 each output a numerical value between 0 and 1. The higher the numerical value, the more likely the AI ​​is to guess. If the difference between the numerical values ​​is large, the accuracy of the guess is high, and if the difference between the numerical values ​​is small, the accuracy of the guess is low. For example, if Z1 is 0.1 and Z2 is 0.9, Z2 is the higher value, so the AI's answer will be "not dementia," which is Z2. The difference between the values ​​of Z1 and Z2 is large, so the accuracy is high.

[0029] When the training data in Fig. 4(b) is used, the output unit 53 of the dementia and / or depression estimation AI 1 will have four levels: dementia level 0·depression level 0, dementia level 0·depression level 1, dementia level 1·depression level 0, and dementia level 1·depression level 1. Therefore, Z1 to Z4 can be assigned, respectively. After the AI ​​is constructed, facial photo data 20, which consists of expressionless facial photo data 21 and smiling facial photo data 22, is input into the dementia and / or depression prediction AI 1, and processed by intermediate section 52 consisting of a neural network, and the results are output as numbers between 0 and 1 for Z1 to Z4 of output section 53. For example, if Z1 is 0.1, Z2 is 0.6, Z3 is 0.2, and Z4 is 0.1, the highest value, Z2, will be adopted. Since Z2 has a dementia level of 0 and a depression level of 1, the AI's answer will be "There is a possibility of depression." You can also use the highest value as the probability. In this case, the value of Z2 is 0.6, so you can say "There is a 60% chance of depression."

[0030] When using the training data in Fig. 4(a), the output unit 53 of the dementia and / or depression estimation AI 1 can be divided into 10% increments from "Dementia level 0% Depression level 0%" to "Dementia level 90% Depression level 90%." In this case, the output unit 53 has 100 elements Z1 to Z100. Check Z1 to Z100 of the output section 53 for the input, and the highest one will be the answer. If a certain element of Z is the highest, the corresponding dementia level XX% and depression level XX% will be the answer.

[0031] Another method for simplifying the output unit 53 is to separate it into a dementia estimation AI and a depression estimation AI. By doing so, the amount of data in the input unit 51, intermediate unit 52, and output unit 53 can be reduced. In this case, the final output of the dementia and / or depression prediction AI1 will be the combination of the answers from the two AIs.

[0032] Next, the machine learning method of the dementia and / or depression estimation AI 1 will be explained with reference to Fig. 5(b). The input unit 51 of the dementia and / or depression estimation AI 1 receives facial photo data 20, which are expressionless facial photo data 21 and smiling facial photo data 22, and the output unit 53 has 100 elements ranging from dementia degree 0% / depression degree 0% to dementia degree 90% / depression degree 90%. First, one of the training data 40 is used for the dementia and / or depression prediction AI 1. Example data 41 is input to the input section 51. Correct answer data 42 is set in the correct answer section. If the correct answer data 42 of the training data 40 is a dementia level of 70% and a depression level of 10%, Aj (j is the element number corresponding to a dementia level of 70% and a depression level of 10%) corresponding to a dementia level of 70% and a depression level of 10% is set to 1, and A1 to A100 except for Aj are set to 0. As a result of inputting example data 41 into dementia and / or depression estimation AI1, estimated values ​​are output as numbers between 0 and 1 in Z1 to Z100 of output section 53. The sum of the differences between Z1 to Z100 and the corresponding A1 to A100 becomes error value D. The neural network of intermediate section 52 is adjusted to lower this error value D. By performing this process on all of the large amounts of training data, the error value D is reduced and the accuracy of the AI ​​is improved.

[0033] In this way, according to this embodiment, since information for constructing an AI that estimates the degree of dementia and / or depression in a subject can be obtained outside of a hospital, it becomes possible to construct an AI for estimating dementia and / or depression not limited to hospital personnel. In addition, by using a facial photo of the person being diagnosed, it will be possible to operate and utilize AI that estimates the degree of dementia and / or depression of the person.

[0034] In addition, the dementia and / or depression prediction AI1 can be used by the pharmacy 10 to provide advice about dementia and depression to pharmacy customers. A facial photo of the person who wishes to receive AI advice is taken, and advice on dementia and depression is given by the dementia and / or depression prediction AI 1. The explanation will be given following the flow in Figure 11. 1) Take a photograph of the subject's face (S401). Take photographs of a face with both a blank expression and a smiling expression. If this is difficult, one of them will suffice. This may be done in parallel with the acquisition of teacher data 40 for improving the accuracy of the dementia and / or depression estimation AI1. 2) Image processing is performed from the subject's expressionless facial photo 13 and the subject's smiling facial photo 14 to generate expressionless facial photo data 21 and smiling facial photo data 22, which are facial photo data 20 to be input into the dementia and / or depression prediction AI 1 (S402). 3) Input the image into the AI ​​(S403). If only one image is acquired, input two identical images. 4) The value of the output section 53 of the dementia and / or depression estimation AI1 is confirmed, and the estimated values ​​of the dementia degree and depression degree are determined (S404). 5) The degree of dementia and depression of the subject is estimated from the prescription 31, and this is used as the estimated prescription value (S405). 6) If the degree of dementia or depression estimated by the AI ​​is worse than the prescription estimate, the subject is notified of advice (S406). The same would happen if the AI ​​estimated a certain degree of dementia or depression even though Prescription 31 did not contain any prescription for dementia or depression. In addition, in the case of hospital pharmacies for departments not related to dementia or depression, such as dermatology or surgery, advice may be given without checking prescriptions, etc.

[0035] By performing such tasks, the subject's potential level of dementia and depression can be confirmed without burdening the subject, which can contribute to the early detection of dementia and depression.

[0036] The dementia and / or depression estimation AI 1 can also be used in a health check system, which estimates the user's level of dementia and depression during a health check. 1) Take a photograph of the face of the person undergoing the health check. Take two photographs, one with a blank expression and one with a smile. This may be easy if you explain that the photographs will be used for a health check, but if this is difficult, it is fine to take just one of the photographs. 2) Generate 20 facial photo data for the AI. 3) Input the image into the AI. If only one image is required, input two identical images. 4) The estimated degree of dementia and depression will be determined from the output results of the AI. 5) Provide the target person with a guide to watch video content according to the estimated level of dementia and depression. Also, introduce them to specialized medical institutions. The notification may be provided on paper with a QR code (registered trademark) along with the health check results, or a URL may be sent to the subject via email.

[0037] In this way, by adding the dementia and / or depression prediction AI1 to a health check, the subject can obtain a diagnosis result for dementia and depression without any special burden.

[0038] In addition, the dementia and / or depression estimation AI1 can be used in a monitoring support system. When the monitoring support system is used, it estimates the degree of dementia and depression of the person being watched over. This will be explained along the flow of Figure 12. The monitoring support system is a system that uses cameras and sensors to check on the condition of elderly family members and relatives who live far away, and provides support or contacts family members if an emergency occurs. 1) In the monitoring support system, a facial photograph of the target is obtained from a video image for confirming the safety of the target (S501). In the monitoring support system, since it is necessary to constantly check the condition of the target, the target is often photographed by a camera at all times, and it is easy to obtain facial photographs of the target with a blank expression and a smiling face. For example, a facial photograph 13 of the user with a blank expression and a facial photograph 14 of the user with a smiling face may be obtained based on the behavior, voice, etc. of the target. 2) Facial photo data 20 for AI is generated from a facial photo of the subject (S502). 3) Input into AI (S503). 4) From the output results of the AI, estimates of the degree of dementia and depression are determined (S504). 5) If the estimated level of dementia or depression is worse than expected, the AI-estimated level of dementia or depression of the subject and corresponding advice will be provided to the relatives or family members (S505). Specifically, if a subject who is not diagnosed with dementia is suspected to have a high level of dementia, the system will notify pre-registered relatives and family members of the subject's possible dementia. At that time, the system will also introduce medical institutions to confirm dementia and provide advice on how to treat the subject.

[0039] In this way, by adding the dementia and / or depression prediction AI1 to the monitoring support system, it is possible to predict the possibility of the subject having dementia or depression without placing any special burden on the subject, and to contact relatives or family members, thereby taking early action.

[0040] In addition, the dementia and / or depression prediction AI 1 can be used for OTC. When dealing with customers who visit an OTC medical store, the AI ​​estimates the degree of dementia and depression of the target customer. 1) Take a photograph of the subject's face at the OTC. The subject can be encouraged to take a photograph as it allows for simple confirmation of dementia and depression. Take a photograph of the user's face with no expression 13 and a photograph of the subject's face with a smiling face 14. If this is difficult, either one will suffice. 2) Image processing is performed on the subject's facial photo 12 so that it becomes facial photo data to be input into the dementia and / or depression prediction AI 1. 3) Input the image into the AI. If only one image is required, input two identical images. 4) Check the value of the output section 53 of the dementia and / or depression estimation AI1, and determine the estimated values ​​of the dementia degree and depression degree. 5) The subjects will be informed that these are merely estimates and will be informed of the estimated level of dementia and depression, as well as advice based on those estimates. Specifically, if a subject who is unaware of dementia or depression is suspected to have a high level of dementia or depression, the system will provide advice such as a referral to a medical institution to confirm dementia or depression and guidelines on how to respond.

[0041] In this way, by using the dementia and / or depression prediction AI1 over-the-counter, subjects can receive a simple diagnosis and advice about dementia and depression without any special burden.

[0042] It can also be used when renewing a driver's license. When renewing a driver's license, a cognitive test may be conducted in the form of questions to check the onset and severity of dementia, etc. However, as this takes a certain amount of time, it is often considered a roundabout way of doing things by those who are eligible. By using the dementia and / or depression prediction AI1 instead of or as a complement to cognitive testing, the accuracy of cognitive testing can be improved, or the amount of cognitive testing can be reduced or eliminated. Specifically, the procedure is as follows. 1) When a driver's license photograph is taken at a license renewal venue, the photographic data is acquired as the subject's facial photograph 12. As in the past, only the driver's license photograph may be taken, or a smiling facial photograph 14 may be acquired, with the explanation that this is for dementia confirmation. 2) Image processing is performed on the subject's facial photo 12 to generate facial photo data 20 to be input into the dementia and / or depression prediction AI 1. 3) Input the data into the AI. If you only have a driver's license photo, input two copies of the photo data. 4) Check the value of the output section 53 of the dementia and / or depression estimation AI1, and determine the estimated values ​​of the dementia degree and depression degree. 5) Subjects will be notified of the test results, including AI-based estimates of their dementia and depression levels, as a complement to the question-based cognitive test.

[0043] In this way, using AI as a means to complement cognitive tests will contribute to improving the accuracy of tests. In addition, using AI instead of cognitive tests will reduce the burden on users.

[0044] Although an example has been described in which expressionless face photograph 13 and smiling face photograph 14 are used as sample data, training data can also be generated from only one of the photographs. Generally, photos taken for identification purposes such as driver's licenses often have a blank expression on the face, and when trying to infer dementia or depression from such photos, it may be possible to use only photos with a blank expression. The construction of the training data is the same as the other data, except that the two facial photographs, one with a blank expression and one with a smile, are reduced to one blank facial photograph 13. The advantage of using only blank facial photograph 13 is that it can be easily obtained without requiring the user to "smile", etc.

[0045] It is also possible to use only one smiling face photograph 14. This is because when using photographs of the subject, there may be many photographs in which the subject is already smiling. Dementia and depression may show distinctive characteristics when smiling, so photos of smiling faces only are also effective. By creating training data of smiling faces only and performing machine learning, it is possible to infer illness from photos of smiling faces alone.

[0046] Regarding the method of utilizing the acquired data, a facial photo 12 of the user may be acquired as a facial photo of the user with a blank expression 13 and a facial photo 14 of the user with a smiling expression. As facial photo data 20, facial photo data 21 of the blank expression and facial photo data 22 of the smiling expression may be generated. Then, an AI may be constructed by using only one of the photographic data. In that case, from one set of data, it is possible to build AI that distinguishes between a neutral expression and a smiling face, AI that distinguishes from a neutral expression, and AI that distinguishes from a smiling face. When using the system, if photos of the user with a neutral expression and a smiling face can be obtained, the AI ​​that distinguishes between a neutral expression and a smiling face will be used, and if only one photo of a neutral expression is obtained, the AI ​​that distinguishes from a neutral expression will be used, allowing the AI ​​to be used according to the situation. EXAMPLES

[0047] Another embodiment will be described with reference to Figures 6 to 8. The same parts as in the first embodiment will be omitted. Fig. 6 is a flow diagram showing a method for creating training data for a dementia and / or depression diagnostic AI according to this embodiment. Fig. 7 is a schematic diagram showing how facial features are extracted for a dementia and / or depression diagnostic AI, where (a) is a schematic diagram showing the part where facial features are extracted, and (b) is a diagram explaining the rise and fall of the corners of the mouth. Fig. 8 is a table showing an example of training data in the method for creating training data for a dementia and / or depression diagnostic AI according to this embodiment.

[0048] Example 1 makes it possible to build an AI for dementia and / or depression without obtaining information from medical institutions. However, in order to build an AI from facial photo data of a subject, a relatively large amount of sample and training data is required. Therefore, there is a need for a method to reduce the workload of building AI by using features closely related to dementia and depression obtained from facial photo data.

[0049] Specialists have pointed out that there are distinctive features in the facial expressions of patients with dementia or depression. Fig. 7 shows a schematic diagram of a user's face 60. As an example for identifying patients with dementia and depression, the degree of drooping of the corners of the mouth 61, the degree of drooping of the eyelids 62, the degree of drooping of the cheeks 63, the degree of drooping of the eyebrows 64, the degree of mouth opening 65, etc. may be considered. By quantifying each feature, they can be used as input data for AI. For example, with respect to the degree of downward movement of the corners of the mouth 61, when the corners of the mouth are at their highest point, as shown in Fig. 7(b), the degree of downward movement of the corners of the mouth is set to 0. When the corners of the mouth are at their lowest point, as shown in Fig. 7(d), the degree of downward movement of the corners of the mouth is set to 9. When the corners of the mouth are somewhat downward, as shown in Fig. 7(c), the degree of downward movement of the corners of the mouth is set to 5. In this way, by defining and quantifying the degree of decline for each feature 70, it becomes possible to input this to the dementia and / or depression prediction AI 1. In addition, by quantifying data from expressionless and smiling faces, it is possible to build a more accurate AI.

[0050] An example of the training data in this embodiment will be described with reference to the table in FIG. In the first embodiment, facial photo data 20, which consisted of expressionless facial photo data 21 and smiling facial photo data 22, were the example data, and the recognition and depression levels were the correct data. The correct data was the same as in the first embodiment, but the example data was feature amount 70 extracted from the facial photo of the user. Extracted from the facial photo data 20, the degree of descent of the mouth corners 61, the degree of descent of the eyelids 62, the degree of descent of the cheeks 63, the degree of descent of the eyebrows 64, and the degree of mouth opening 65 were determined as numerical values. In this example, there are 10 levels from 0 to 9. As the feature amount 70, a group of feature amounts 71 of neutral expressions and a group of feature amounts 72 of smiling expressions are used. Since the feature amount 70 is extracted only from the facial photograph data 20, the burden on the user P is not increased.

[0051] The present embodiment will be described along the flow of FIG. The parts of obtaining the contents of the prescription 31 etc. from the user (S201), obtaining a facial photo 12 of the user (S202), and estimating the degree of dementia and depression from the type and amount of medication listed on the prescription 31 (S203) are the same as in Example 1. Facial feature values ​​70 are extracted from the facial photograph and generated as example data (S204). In this step, similar to the first embodiment, facial photograph data 20 is generated from the user's facial photograph 12. After that, facial feature values ​​70 are extracted from expressionless facial photograph data 21 and smiling facial photograph data 22, which are the facial photograph data 20, and generated as example data. For example, in the case of number 1 in the example of training data using feature amount 70 in FIG. 8, first, expressionless face photo data 21, which is the source of the example data, is image-processed to obtain a mouth corner descent degree of 8, eyelid descent degree of 5, cheek descent degree of 3, eyebrow descent degree of 5, and mouth opening degree of 2. Next, smiling face photo data 22 is image-processed to obtain a mouth corner descent degree of 7, eyelid descent degree of 5, cheek descent degree of 2, eyebrow descent degree of 5, and mouth opening degree of 2. The method of generating the correct answer data is the same as in the first embodiment. In this way, one piece of teacher data 40 is completed (S205). The same process is carried out for the other data, and teacher data 40 is generated.

[0052] In this way, by using facial feature 70 as example data, the example data becomes specialized for dementia and / or depression, and the amount of input data is reduced, making it possible to improve the accuracy of the AI ​​with a small amount of training data.

[0053] Furthermore, by using facial features 70 as example data, the generation of the training data 40 can be partially simplified. Dementia and depression experts have a wealth of knowledge and experience about facial features70 and the degree of dementia and depression. Therefore, with the support of experts, it is possible to create a certain amount of training data without using photos or prescriptions31 of users. In other words, by estimating feature value 70 that is expected at the time of a specific dementia level and depression level, and using the estimated feature value 70 as example data, and at that time, using the predicted dementia level and depression level as correct answer data, it is possible to easily obtain data by performing machine learning.

[0054] In other words, by asking experts to list the characteristics that are likely to be present in dementia or depression when the face is expressionless, it is possible to build up a certain amount of data without collecting photographs. Similarly, in the case of smiling, by asking subjects to list the characteristics that distinguish them from a smile and a neutral expression, it is possible to build up some data.

[0055] In AI machine learning, in the early stages of learning, there may be a large discrepancy between the AI ​​output and the correct answer. By performing machine learning using training data created in advance with the assistance of experts, when machine learning is performed using user data, the discrepancy between the AI ​​output and the correct answer can be reduced, allowing the AI ​​to converge more quickly.

[0056] The explanation will be given following the flow in FIG. The facial feature amount 70 and the degree of dementia or depression are quantified by an expert (S301). For example, the facial feature amount 70 is a combination of the degree of drooping of the corners of the mouth 61, the degree of drooping of the eyelids 62, the degree of drooping of the cheeks 63, the degree of drooping of the eyebrows 64, and the degree of mouth opening 65 when the face is expressionless, and the degree of drooping of the corners of the mouth 61, the degree of drooping of the eyelids 62, the degree of drooping of the cheeks 63, the degree of drooping of the eyebrows 64, and the degree of mouth opening 65 when the face is smiling, and the degree of dementia or depression is quantified between 0 and 100%. An example is shown in Figure 10(a). This example is a table showing the estimated dementia level when the change in the degree of drooping of the corners of the mouth and the degree of drooping of the eyelids from a neutral expression to a smiling expression is changed on a 10-point scale, with the cheeks drooping at a level of 5, the eyebrows drooping at a level of 4, the mouth opening at a level of 2, the eyelids drooping at a level of 4, and the corners of the mouth drooping at a level of 5 when the face is expressionless. The range of change in the degree of descent of the mouth corners when the face is expressionless is 5, so the range of change is -5 to 4. The range of change in the degree of descent of the eyelids when the face is expressionless is 4, so the range of change in the degree of descent of the eyelids when the face is expressionless is -4 to 5. For example, if the change in the degree of eyelid descent is 0 and the change in the degree of mouth corner descent is -3 when smiling, the smile is appropriately expressed and the recognition rate is considered to be low. As an example, the maximum and minimum values ​​of the change are estimated, and a proportional calculation is performed from there, resulting in an estimated value of 34%. Similarly, if the change in the degree of downward movement of the corners of the mouth when smiling is +2, the corners of the mouth are further downward despite the smile, and it is considered that the smile is not being expressed appropriately, so the recognition rate is considered to be high. As an example, the maximum and minimum values ​​of the amount of change are estimated, and a proportional calculation is performed from there, resulting in an estimated value of 59%.

[0057] Such a table is created for all the features 70. Using these, the feature levels are treated as example data, and the estimated dementia and depression levels are treated as correct answer data (S302). Figure 10(b) shows an example of a training data group. When all facial feature values ​​70 are changed from 0 to 9, dementia level XX% and depression level XX% are listed as correct answer data. The example in Figure 10(b) is an example of the case of one feature group. If a neutral feature value 71 and a smiling feature value 72 are used, the number of cases will increase. Teacher data is completed under the guidance of experts (S303). Next, we have experts generate dementia and depression levels by focusing on combinations of 70 features that are likely to actually exist (S304). This process increases the amount of training data that is closer to reality. Through the above steps, the initial training data for machine learning is complete (S305).

[0058] By generating training data in this way, the amount of data to be collected can be reduced and the convergence of AI machine learning can be accelerated. EXAMPLES

[0059] Another embodiment will be described with reference to Figures 13 to 17. The same parts as in the first embodiment will be omitted. Fig. 13 is a system diagram of the third embodiment of the present invention. Fig. 14 is a diagram of a screen of a mobile terminal of the third embodiment of the present invention, in which (a) is a screen for a question to confirm the level of vitality, (b) is a screen for requesting permission to obtain teacher data, (c) is a screen for displaying guidance for obtaining a face photo of the user by the mobile terminal, and (d) is a screen for displaying guidance for confirming the medicine taken by the user by the mobile terminal. Fig. 15 is a table showing information for generating teacher data according to the third embodiment of the present invention. Fig. 16 is a flow diagram showing a teacher data acquisition procedure according to the third embodiment of the present invention. Fig. 17 is a flow diagram showing an embodiment of the dementia and / or depression prediction AI system according to the present invention on a mobile terminal.

[0060] According to Example 1, it is possible to build an AI for dementia and / or depression by generating training data from information at pharmacies without obtaining information from medical institutions. However, in order to build an AI from facial photo data of a subject, a relatively large amount of sample and training data is required. Therefore, there is a need for a method to obtain supervisory data on dementia and depression, not limited to pharmacies.

[0061] In recent years, apps for mobile devices such as smartphones have been developed that provide advice on diet and exercise to prevent dementia. Therefore, in this embodiment, it is made possible to obtain information that serves as training data for the AI ​​from an application on a mobile terminal used by a user. The training data includes at least a facial photograph of the user, a prescription, an instruction sheet, or information that can be used to infer the user's dementia and / or depression. The information that can be used to infer the user's dementia and / or depression is obtained by asking the user questions. In this way, by obtaining information related to teacher data from an app on a mobile device, it is possible to obtain sample data from many people, not just pharmacies.

[0062] FIG. 13 shows a system diagram of this embodiment. The dementia and / or depression prediction AI 1 in this embodiment is a system that obtains training data from a mobile terminal of a user via a network. The dementia and / or depression prediction AI 1 consists of a machine learning system 50, a management unit 80, a server 81, and a mobile terminal 90.

[0063] The machine learning system 50 is the core part of the AI, similar to the first embodiment. There is a learning phase where the AI ​​learns from teacher data, and a utilization phase where the intermediate part constructed through learning is used to provide an AI answer. During learning, training data sent from the management unit 80 is used to learn, intermediate adjustments are made, and the accuracy of the AI ​​is improved. When in use, the system estimates the level of awareness and depression from the photographic data sent from the management unit 80 and returns the estimated values ​​to the management unit 80. The management unit 80 is a part that acts as an intermediary between the mobile terminal 90 and the machine learning system 50. The management unit 80 writes data related to the generation of teacher data to the server 81 and reads it out. The information necessary for generating teacher data is received from the mobile terminal 90 via the Internet N, some of which is processed and adjusted, and the rest, including that which has been converted into teacher data, is stored in the server 81. The information to be processed may be, for example, a photograph of a medicine taken by a user. If the received medicine photograph is a medicine notebook, the management unit 80 reads the name of the medicine written in the notebook and sets it as the medicine contents. If the received medicine photograph is a medicine bag or the medicine itself, the medicine name is identified from the name on the surface of the medicine bag or the name on the medicine sheet. Thereafter, similarly to the first embodiment, the recognition degree and the depression degree are estimated and calculated using the recognition degree and depression degree calculation table for the drug name as shown in FIG. Facial photographs are also processed and adjusted in the same manner. Since facial photographs taken by users may not be suitable as training data depending on the shooting environment and the user's shooting method, the management unit 80 performs the same operations as in the first embodiment, such as simplifying the size of the face and simplifying the background. The processed photographs are stored in the server 81. The management unit 80 reads out the facial photograph, recognition level, and depression level values, which are the training data stored in the server 81, and sends the training data to the machine learning system 50 that is currently learning.

[0064] The server 81 mainly processes user information 83, information acquired from the user 84, and information related to the teacher data, and holds information that has been set as example data 41 of the teacher data and information that has been set as answer data 42. These are collectively referred to as server management data 82. The server management data 82 will be described with reference to FIG. A user ID is assigned to each user, and information is managed by the user ID. For example, the user with the user ID 1001 is Yamada Taro, and the user with the user ID 1056 is Suzuki Hanako. Each person's date of birth, gender, and medical history are recorded. This data is generally not changed after registration. A data number and a data acquisition date are assigned to each collection of information 84 acquired from the user by the management unit 80. The data number is unique and there is no duplication. Age, a face photo with a neutral expression, a face photo with a smiling expression, a photo related to medication, awareness level estimated from questions posed by the app, and depression level are transmitted to the management unit 80 and recorded in the server 81. Photographs of medications may be of a medicine book, a medicine packet, or a medicine sheet. The management unit 80 records in the server 81 the adjusted and processed data from the mobile terminal 90, the adjusted facial photograph as example data 41, the contents of the medicine, and the recognition level and depression level values ​​estimated from the medicine as correct answer data. Information managed under one data number corresponds to one piece of teacher data. As information related to teacher data, multiple data numbers may be stored for one user. This is because the user's status may change depending on the time of acquisition.

[0065] The server management data 82 records awareness and depression level information generated from drug information, and awareness and depression level estimated from questions posed by the app. The awareness and depression level information generated from the drug information may be used as the correct answer data 42, or the awareness and depression level estimated from the questions posed by the application may be used as the correct answer data. In addition, the correct answer data may be an average of the data generated from the drug information and the data estimated from the questions posed by the app. In addition, if the photo of the medication taken by the user is unclear, the awareness and depression levels estimated from the questions posed by the app may be used as an alternative.

[0066] The mobile terminal 90 connects each user with the management unit 80 . The mobile terminal 90 mainly comprises a display unit 91, a control unit, a storage unit, an operation unit, a communication unit, and a camera unit, all of which are not shown. The control unit is a part that controls the entire mobile terminal 90. While the application 92 is running, the control unit also performs control according to the content of the application. The control unit acquires the operation contents of the user from the operation unit, displays characters, symbols, images, etc. on the display unit 91, takes pictures with the camera, and instructs the communication unit to transmit and receive data to and from the network. It also writes and reads temporary data to and from the memory unit, and reads out applications 92. The storage unit stores a program for operating the control unit and temporarily stores variables, data, etc. used for control. An application 92 according to the embodiment is also stored as one of the programs.

[0067] The camera is a camera that captures images of the surroundings of the mobile terminal 90. In this embodiment, the camera captures a photograph of the user's face and a photograph of the user's medicine notebook. The communication unit is a communication unit that communicates with the Internet N. The communication unit connects to the Internet N using a mobile communication system such as WiFi or 4G. The operation unit is a mechanism that allows the user to operate the terminal using buttons and a touch panel. The display unit 91 is a part that displays text and images such as guidance and messages, and serves as a display screen for applications and a monitor for captured images.

[0068] In this embodiment, the application 92 is assumed to be an exercise recommendation application for preventing dementia in the elderly and a health check application. Some of these apps are configured to have a section that acquires information about teacher data. Users who install dementia prevention apps are people who are concerned about their risk of developing dementia. In other words, they are people who are at high risk of developing dementia or depression. Therefore, the training data obtained from mobile devices on which such apps are installed is highly likely to be training data that indicates the early stages of dementia and depression, and by using it for AI training, the accuracy of AI in estimating dementia and depression can be improved.

[0069] The flow of acquiring information related to the teacher data will be described with reference to the flow in FIG. In this embodiment, the function is an example of a part of an application 92 in a mobile terminal 90. Examples of the application 92 include an exercise recommendation app for preventing dementia in the elderly and a health check app. When a user launches application 92, it is confirmed by data within the application whether this is the first time the application has been launched or whether three months have passed since the last request to obtain information related to teacher data (S601). If not, the result is NO and the application proceeds to normal operation (S608). If so, an information provision request screen 94 for AI learning is displayed on the screen as a request for cooperation in the AI ​​learning of S602 (FIG. 14(b)).

[0070] If the user selects "Yes" in S603, the information acquisition proceeds, whereas if the user selects "No", the application proceeds to normal operation (S608).

[0071] As a guidance for taking a face photo, a face photo guidance screen 95 is displayed on the display unit 91, and a face photo with no expression and a face photo with a smile are taken (S604, FIG. 14(c)). If it is difficult to specify a face photo with no expression, a natural, normal face photo may be specified. As guidance for obtaining information about medicine, a medicine information photographing guidance screen 96 is displayed on the screen, and a photograph of the page of the medicine notebook showing the medicine information, the medicine envelope, etc. is taken (S605, FIG. 14(d)).

[0072] Questions related to dementia and depression (Figure 14(a)) are asked, and the awareness and depression levels estimated from the user's answers are obtained as awareness and depression levels according to the questions (S606). The awareness and depression levels are estimated simply using an app on the mobile device. Also, the content of the question may be notified to the management unit 80 via the Internet, and an appropriate estimation result may be obtained from the management unit 80. The user's ID No., a face photo, a photo related to medication, and the degree of awareness and depression estimated from answers to questions, all of which are managed under a single data No., are transmitted to the management unit 80 via the Internet N (S607). Since the information acquisition and transmission is completed, the application proceeds to normal operation (S608). Through this series of operations, information related to the teacher data can be obtained extremely easily via the mobile terminal 90.

[0073] In addition, the dementia and / or depression prediction AI1 can be used on the mobile terminal 90 to give advice to the person to be diagnosed. The system is designed to enable the capture of a person's face using a mobile device via the network, and then provide advice based on the results of the AI ​​system's analysis of the captured face photo. For example, a facial photograph of a target person is taken by calling from an application 92 on a mobile terminal 90, and advice on dementia and depression is given based on the estimated dementia and depression score by AI. This will be explained along with the flow in Fig. 17.

[0074] 1) The application 92 in the smartphone, which is the mobile terminal 90, proposes a simple diagnosis for dementia and depression to the subject (S701). The suggestions will come from exercise recommendation apps to prevent dementia in the elderly and health check apps, so subjects can receive a simple diagnosis without feeling uncomfortable.

[0075] 2) The subject follows the guidance of the application 92 to take a photograph of his / her face (S702). It is preferable to take two photographs of the face, one with a blank expression and one with a smiling face. This is because it is possible to improve the accuracy of estimating dementia and depression. By taking a picture while viewing an image of the user on the mobile terminal 90, the user can take a picture while being conscious of his or her own expressionless face and smiling face, and therefore can take a more appropriate facial picture.

[0076] 3) The application 92 transmits the facial photograph to the management unit 80 via the Internet N. The management unit 80 processes the facial photograph of the target person by removing the background, adjusting the size of the face, etc., so that the facial photograph data becomes similar to the sample data (S703).

[0077] 4) The management unit 80 inputs the photo data to the input unit 51 of the machine learning system 50, which is the core of the AI ​​(S704). 5) Check the values ​​of the output section 53 of the machine learning system 50 and determine the estimated values ​​of the dementia level and depression level (S705). 6) The estimated value is transmitted to the mobile terminal 90 via the Internet N (S706).

[0078] 7) The subject is notified of the degree of dementia and depression estimated by the AI, and advice based on the AI ​​estimate is displayed on the screen. For example, the advice may read, "The degree of dementia is estimated to be 70%. We recommend that you seek medical attention as soon as possible." The contents of the advice may be generated by the management unit 80, or the contents may be displayed on the mobile terminal 90 that are preset according to the estimated values. In addition, in the case of the second or subsequent simple diagnosis, the previous estimated value may also be displayed, and advice may be given according to the increase or decrease in the value (S707). For example, the advice may be, "The degree of dementia is estimated to be 30%. This is better than the previous estimated value of 40%. Please continue to exercise to prevent dementia."

[0079] By making such a suggestion, the subject's potential level of dementia and depression can be confirmed without burdening the subject, which can contribute to the early detection of dementia and / or depression. [Industrial Applicability]

[0080] The dementia and / or depression estimating AI system and the teacher data creation method according to the present invention can be used as a technology for constructing dementia and / or depression estimating AI outside of hospitals, and can also be applied to other cases in which the illness is manifested as a characteristic facial expression. Therefore, the present invention is considered to have great industrial applicability. [Explanation of symbols]

[0081] 1. Dementia and / or depression prediction AI 10. Pharmacy 11. Filming equipment 12 User's photo 13. Expressionless Face Photo 14 Smiling Face Photo 20 Facial photo data 21 Expressionless face photo data 22 Smiling face photo data 30 Medicine notebook 31 Prescription 32 Drug Data 40 Teacher Data 41 Example Data 42 Correct data 43 Calculation table 50 Machine Learning Systems 51 Input section 52 Middle section 53 Output section 54 Answers 60 User's face 61. Mouth corner droop 62 Eyelid drooping 63 Cheek droopiness 64 Eyebrow droop 65 Opening 70 Features 71 Neutral facial expression features 72 Smile Features 80 Management Department 81 Server 82 Server Management Data 83 User information 84 Information obtained from users 90 Mobile Devices 91 Display section 92 Applications 93 Health Check Questions 94 Information request screen for AI learning 95 Face photo guidance screen 96 Drug information shooting guidance screen P User D error N Internet

Claims

1. A method for creating training data used in an AI that estimates the degree of dementia and / or depression from a user's facial photograph, comprising:

1. A system for estimating the degree of dementia and / or depression, The information that serves as training data for the AI ​​can be obtained from the location where the user receives the medicine according to the prescription or from an application on the mobile device used by the user. The teacher data includes at least a facial photograph of the user, a prescription, an instruction sheet, or information that can estimate the user's dementia and / or depression, Using either a facial photograph of the user with a neutral expression or a facial photograph with a smiling expression or both, extracting facial features from the facial photograph and generating them as example data of training data; The feature amount has at least one of the degree of drooping of the corners of the mouth, the degree of drooping of the eyelids, the degree of drooping of the cheeks, the degree of drooping of the eyebrows, and the degree of mouth opening, Furthermore, the example data of the training data includes the user's age, sex, medical history, and family medical history, The degree of dementia and the degree of depression are estimated from the type and amount of the medicine described in the prescription or instruction sheet and used as correct answer data for training data; A teacher data creation method, comprising creating teacher data for machine learning.

2. Estimating the feature amount predicted at a specific dementia level and depression level; The teacher data creation method described in claim 1, characterized in that the estimated features are used as the example data, and the predicted dementia level and depression level are used as the correct answer data, and machine learning is performed.

3. An AI system using teacher data created by the teacher data creation method according to claim 1 or 2, A dementia and / or depression estimation AI system that estimates the degree of dementia and depression of a subject based on a facial photograph of the subject.

4. The dementia and / or depression prediction AI system of claim 3, characterized in that a facial photo of the subject is taken at the pharmacy and advice is provided based on the results of the AI ​​system's analysis of the photo.

5. The dementia and / or depression prediction AI system as described in claim 3, characterized in that a facial photo of the subject is taken and, depending on the results of the AI ​​system's response to the photo, the subject is guided to video content.

6. The dementia and / or depression prediction AI system of claim 3, characterized in that a facial photo of the subject is taken and, depending on the results of the AI ​​system's analysis, a referral to a specialized medical institution is made.

7. The dementia and / or depression prediction AI system according to claim 3, characterized in that a facial photo of the subject is taken and advice is provided to the family depending on the results of the AI ​​system's response to the photo.

8. The dementia and / or depression prediction AI system of claim 3, characterized in that a facial photo of the subject is taken at an OTC medical product store, and advice is provided based on the results of the AI ​​system's analysis of the photo.

9. The dementia and / or depression prediction AI system of claim 3, characterized in that a facial photograph of the subject taken with a mobile device used by the subject can be obtained via a network, and advice is provided depending on the results of the AI ​​system's processing of the obtained facial photograph.

Images