Health state determination apparatus and health state determination method

Abstract

To provide a health state determination apparatus and the like capable of accurately determining and providing a user's health state.SOLUTION: In a server 3, a storage unit 31 stores a fatigue state estimation model, an estimation score acquisition unit 32 acquires an estimation score using the fatigue state estimation model and a still image of the face of a user M, an autonomic nervous state score acquisition unit 33 acquires an autonomic nervous state score of the user from a moving image of the face of the user M, a determination unit 34 determines the user's health state on the basis of a combination of the estimation score and the autonomic nervous state score, and an output unit 35 outputs determination data.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a health condition determination device for determining the health condition of a user. [Background technology]

[0002] A conventional estimation device for estimating a user's mental fatigue level, etc., is described in Patent Document 1. This estimation device estimates the user's mental fatigue level, etc., using a fatigue state estimation model that receives an image of the user's facial skin as input and outputs the user's mental fatigue level, etc. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-85856 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, there has been a demand for a device that can assess a user's health condition and provide the user with the assessment results. However, while the conventional estimation device can estimate the user's mental fatigue level, the estimation results are not sufficient for assessing the user's health condition.

[0005] The present invention has been made to solve the above-mentioned problems, and aims to provide a health condition determination device and the like that can accurately determine and provide the health condition of a user. [Means for solving the problem]

[0006] In order to achieve the above object, the health condition determination device of claim 1 is characterized by comprising: a memory unit that stores a plurality of fatigue state estimation models that receive a still image of a user's face as input and output a plurality of different estimated scores that represent estimated results of the fatigue state of at least one of the user's mind and body; an estimated score acquisition unit that acquires at least one estimated score for the user from the still image of the user's face using at least one of the plurality of fatigue state estimation models; an autonomic nervous system state score acquisition unit that acquires an autonomic nervous system state score that represents the state of the user's autonomic nervous system from a moving image of the user's face using a predetermined acquisition method; a determination unit that determines the user's health condition based on a combination of the at least one estimated score and the autonomic nervous system state score; and an output unit that outputs determination data that represents the determination result by the determination unit.

[0007] This health condition assessment device assesses a user's health condition based on a combination of at least one estimated score representing the user's mental and physical fatigue state and an autonomic nervous state score. Here, the at least one estimated score is obtained from a still image of the user's face and therefore represents the state of fatigue accumulated in the user's mind and body. Meanwhile, the autonomic nervous state score is obtained from a moving image of the user's face and therefore represents the state of the user's autonomic nervous system at the time the moving image was acquired. Therefore, when the user's health condition is assessed based on a combination of the at least one estimated score and the autonomic nervous state score, the user's health condition can be accurately assessed by taking into account the current state of the user's autonomic nervous system in addition to the state of fatigue accumulated in the user's mind and body up to that point. Furthermore, the assessment result can be provided to the user by outputting assessment data representing such assessment result from an output unit. Note that, in this specification, the "estimated result of at least one of the user's mental and physical fatigue state" also includes the estimated results of the user's insomnia state and the user's irritability state.

[0008] In the present invention, the at least one estimated score is preferably a score representing at least one of the Chalder fatigue scale, OSI (Oxidative stress index), isoprostane, 8OHdG (8-hydroxy-2'-deoxyguanosine), homovanillic acid, vanillylmandelic acid, dROMs (Diacron Reactive Oxygen Metabolites), insomnia scale, and stress level.

[0009] Experiments conducted by the present applicant have confirmed that model parameters of multiple fatigue state estimation models that estimate the Chalder fatigue scale, OSI, isoprostane, 8OHdG, homovanillic acid, vanillylmandelic acid, dROMs, insomnia scale, and stress level can be accurately learned using still images of a user's face. Therefore, by using such multiple fatigue state estimation models, this health condition determination device can accurately estimate estimated scores that represent the estimated results of the Chalder fatigue scale, OSI, isoprostane, 8OHdG, homovanillic acid, vanillylmandelic acid, dROMs, insomnia scale, and stress level. Thus, by determining the user's health condition based on a combination of at least one of these estimated scores and the autonomic nervous state score, the accuracy of determining the user's health condition can be improved.

[0010] In the present invention, the autonomic nervous state score is preferably a score representing at least one of CVR-R (Coefficient of Variation of RR intervals) and LF / HF.

[0011] It has been known that CVR-R and LF / HF can be accurately estimated from a moving image of the skin of a user's face. Therefore, this health condition determination device can improve the accuracy of determining the user's health condition by determining the user's health condition based on a combination of such an autonomic nervous state score and at least one estimated score.

[0012] In the present invention, it is preferable that at least one estimated score is composed of a fatigue score representing an estimated result of the user's fatigue state and an insomnia score representing an estimated result of the user's insomnia state, and that the judgment unit judges the user's health state based on a combination of the fatigue score, insomnia score, and autonomic nervous state score.

[0013] This health condition determination device determines the user's health condition based on a combination of the fatigue score, insomnia score, and autonomic nervous state score. Therefore, the user's health condition can be accurately determined by taking into account the current state of the user's autonomic nervous system in addition to the state of fatigue accumulated in the user's mind and body up to that point and the user's current state of insomnia.

[0014] In the present invention, it is preferable that the at least one estimated score is composed of an estimated score representing one of the Chalder fatigue scale, OSI (Oxidative stress index), isoprostane, 8OHdG (8-hydroxy-2'-deoxyguanosine), homovanillic acid, vanillylmandelic acid, dROMs (Diacron Reactive Oxygen Metabolites), an insomnia scale, and stress level, the autonomic nervous state score is composed of CVR-R (Coefficient of Variation of RR intervals) and LF / HF, and the determination unit determines the user's health state based on a combination of the estimated score, CVR-R, and LF / HF.

[0015] This health condition determination device determines the user's health condition based on a combination of an estimated score representing one of the Chalder fatigue scale, OSI, isoprostane, 8OHdG, homovanillic acid, vanillylmandelic acid, dROMs, insomnia scale, and stress level, CVR-R, and LF / HF. Therefore, the user's health condition can be accurately determined by taking into account the current state and autonomic nervous balance of the user in addition to the state of fatigue accumulated in the user's mind and body up to that point.

[0016] In the present invention, the autonomic nervous state score is preferably composed of LF / HF, which indicates whether the user's autonomic nervous balance is in a sympathetic dominant state, a parasympathetic dominant state, or a normal state.

[0017] According to this health condition determination device, by using LF / HF as the autonomic nervous state score, it is possible to accurately determine whether the user's autonomic nervous balance is in a sympathetic nervous system dominant state, a parasympathetic nervous system dominant state, or a normal state.

[0018] In the present invention, it is preferable that at least one estimated score is composed of a fatigue score representing an estimated result of the user's fatigue state, an insomnia score representing an estimated result of the user's insomnia state, and an irritability level representing an estimated result of the user's irritability state, and that the judgment unit judges the user's health state based on a combination of the fatigue score, insomnia score, irritability level, and autonomic nervous state score.

[0019] This health condition determination device determines the user's health condition based on a combination of the fatigue score, insomnia score, irritability level, and autonomic nervous state score. Therefore, the user's health condition can be accurately determined by taking into account the current state of the user's autonomic nervous system in addition to the state of fatigue accumulated in the user's mind and body up to that point, the user's current insomnia state, and the user's current irritability level.

[0020] In order to achieve the above-mentioned object, the health condition determination method of claim 8 is characterized in that it is executed by a processing device, and includes the following steps: a storage step for storing a plurality of fatigue state estimation models that input a still image of a user's face and output a plurality of different estimated scores that represent estimated results of at least one of the user's mental and physical fatigue states; an estimation score acquisition step for acquiring at least one estimated score for the user using at least one of the plurality of fatigue state estimation models and a still image of the user's face; a score acquisition step for acquiring an autonomic nervous state score that represents the state of the user's autonomic nervous system from a moving image of the user's face using a predetermined acquisition method; a determination step for determining the user's health condition based on a combination of the at least one estimated score and the autonomic nervous state score; and an output step for outputting the determination result in the determination step. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a diagram showing the configuration of a health condition determination system including a health condition determination device according to a first embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing the functional configuration of a server. [Figure 3] 10A to 10C are diagrams showing the determination results in a determination unit, etc. FIG. [Figure 4] FIG. 2 is a sequence diagram showing a control process of the health condition determination system. [Figure 5] 10A to 10C are diagrams showing the determination results in a determination unit of the second embodiment. [Figure 6A] 11A and 11B are diagrams showing the determination results of the determination unit of the third embodiment for the time period from 8 pm to before going to bed. [Figure 6B] FIG. 6B is a diagram showing a continuation of FIG. 6A. [Figure 7A] 10A and 10B are diagrams showing the determination results of the determination unit of the third embodiment for the time period from wake-up to 8 pm. [Figure 7B] FIG. 7B is a diagram showing a continuation of FIG. 7A. [Figure 8A] 10A and 10B are diagrams showing the determination results in a determination unit of the fourth embodiment. [Figure 8B]FIG. 8B is a diagram showing a continuation of FIG. 8A. DETAILED DESCRIPTION OF THE INVENTION

[0022] A health condition determination system including a health condition determination device according to a first embodiment of the present invention will be described below with reference to the drawings. The health condition determination device according to this embodiment is for determining a user's health condition based on still and moving images of the user's face, as described below.

[0023] As shown in Fig. 1, the health condition determination system 1 of this embodiment includes a plurality of user terminals 2 (only one is shown) and a server 3 as a health condition determination device. In this health condition determination system 1, the user terminals 2 and server 3 are configured to be able to communicate with each other via a network 4. This network 4 is configured by a wireless communication network, a wired communication network, etc.

[0024] This health condition determination system 1 is equipped with multiple user terminals 2, but the following explanation will be given taking as an example the user terminal 2 owned by user M. The user terminal 2 is a smartphone type, and is equipped with a controller (not shown), a display 2a, a camera 2b, and a wireless communication device (not shown). Various application software is installed in this controller.

[0025] The display 2a is configured with a capacitance type touch panel and is disposed so that its front surface faces the surface of the user terminal 2. Although not shown, various icons corresponding to various application software are displayed on the display 2a.

[0026] Furthermore, when an input operation (for example, an operation such as tapping, swiping, pinching out, or pinching in) is performed by the user M on the display 2a, an operation signal representing the operation is output to the controller. Based on the operation signal, the controller executes various control processes, as will be described later.

[0027] Camera 2b is also disposed facing the surface of user terminal 2. When user M performs an input operation for photographing, camera 2b captures still images and moving images of user M's face. In this case, the still images and moving images of user M's face are captured with the distance from camera 2b set to approximately a predetermined value (for example, 30 cm).

[0028] In addition, when capturing a moving image of the face, after capturing a moving image of the face for a predetermined shooting time (for example, one minute), the first captured portion of the moving image is deleted to obtain a moving image of the face for a predetermined time (for example, 30 seconds). This predetermined time is set so that the autonomic nervous state score, which will be described later, can be appropriately estimated.

[0029] As described above, after the still image and moving image of the face of the user M are captured by the camera 2b, the image data including these images is transmitted from the user terminal 2 to the server 3.

[0030] In addition, the image data may be configured so that only a moving image of user M's face is sent from user terminal 2 to server 3, in which case server 3 may be configured to use the images of each frame cut out from the moving image of user M's face as still images.

[0031] The server 3 is used to execute various arithmetic processes, which will be described later, including communication processes with the user terminal 2, and is equipped with a control unit, a storage unit, and a communication unit (none of which are shown). The control unit is made up of a CPU, memory, an I / O interface, etc., and the storage unit is made up of storage devices such as ROM, RAM, and HDD. The storage unit stores an estimation model, which will be described later. The communication unit is made up of a communication circuit.

[0032] As described below, when this server 3 receives image data including still and moving images of a face from the user terminal 2, an estimated score is calculated using an estimation model in the storage unit, the health condition of user M is determined, and the determination result data is sent to the user terminal 2.

[0033] Next, the functional configuration of the server 3 will be described with reference to Fig. 2. As shown in Fig. 2, the server 3 has functions as a storage unit 31, an estimated score acquisition unit 32, an autonomic nerve state score acquisition unit 33, a determination unit 34, and an output unit 35.

[0034] The storage unit 31 stores a machine learning model that receives a still image of a person's face as input and outputs an estimated score. This estimated score represents a person's mental and physical fatigue state and is configured to take a value within a range of 0 to 1. In this embodiment, the Chalder fatigue scale is used as the estimated score.

[0035] Furthermore, for example, ResNet is used as the machine learning model, and an image of the mouth is extracted from a still image of a person's face using a predetermined image processing method (for example, high-resolution networks for facial landmark detection, etc.), and by using this image as training data, the model parameters are fully learned and stored in the memory unit 31.

[0036] When a still image of user M's face is input from the user terminal 2, the estimated score acquisition unit 32 acquires an estimated score of user M using the machine learning model in the storage unit 31. That is, an image of the mouth of user M's face is input to the machine learning model from the still image of user M's face using the above-mentioned predetermined method, and the estimated score of user M is acquired as the output.

[0037] On the other hand, when a moving image of the face of user M is input from user terminal 2, autonomic nervous state score acquisition unit 33 calculates CVR-R (Coefficient of Variation of RR intervals) as the autonomic nervous state score of user M using the method described below. This CVR-R represents the state of the autonomic nervous system of user M, and is calculated as a value within the range of 0 to 100%.

[0038] First, in each frame of the video, a face is detected using a face detection algorithm (e.g., face landmark detection), and the change in brightness of a specific color component (e.g., G (green)) contained in the information on the change in brightness between frames of each RGB color component in the detected area is calculated. This brightness change data is then Fourier transformed to calculate the mode frequency of the color component as the heart rate.

[0039] Next, the heart rate data is filtered using a band-pass filter and then converted into equal-time interval data (RR interval) using spline interpolation. Using the mean value μ and standard deviation σ of this equal-time interval data, CVR-R is calculated using the following equation (1).

[0040] CVR-R=σ / μ×100(%) (1) The determination unit 34 first determines whether the user M has a high level of fatigue and is in a dangerous state based on the estimated score (Chalder fatigue scale). Specifically, if the estimated score is equal to or greater than a threshold value of 0.5, the determination unit 34 determines that the user M has a high level of fatigue and is in a dangerous state, and otherwise determines that the user M has a low level of fatigue and is in a normal state.

[0041] The threshold value of the estimated score is not limited to 0.5, but can be set to an appropriate value other than 0.5 based on the age group of user M. For example, if user M is in a relatively young age group, the threshold value of the estimated score may be set to a value greater than 0.5 (for example, 0.6).

[0042] Furthermore, based on the autonomic nerve state score (CVR-R), it is determined whether the autonomic nerves of user M are in a dangerous state. Specifically, if the autonomic nerve state score is equal to or less than a threshold value of 2%, it is determined that the autonomic nerves of user M are in a dangerous state (disturbed state), and otherwise it is determined that the autonomic nerves of user M are in a normal state.

[0043] The threshold value of the autonomic nervous state score is not limited to 2%, but can be set to an appropriate value other than 2% based on the age group of user M. For example, if user M is in a relatively young age group, the threshold value of the autonomic nervous state score may be set to a value smaller than 2% (for example, 1.8%).

[0044] Then, the determination unit 34 determines the health condition (risk level) of the user M based on a combination of the determination results of the autonomic nervous state score and the estimated score, as shown in Fig. 3. As shown in the figure, when the determination results of the autonomic nervous state score and the estimated score are both "normal", the health condition (risk level) is determined to be "risk level 1: no problem".

[0045] Furthermore, if the autonomic nervous system status score is judged to be "dangerous" and the estimated score is judged to be "normal," the result will be "Risk level 2: The autonomic nervous system is somewhat disturbed, but it is not affecting the body. It is necessary to adopt daily lifestyle habits that regulate the autonomic nervous system."

[0046] The health condition of user M is determined in this manner for the following reason. That is, the estimated score is obtained from a still image and therefore represents the state of fatigue accumulated in the mind and body of user M, whereas the autonomic nervous state score is obtained from a moving image and therefore represents the state of the autonomic nervous system at that time. Therefore, if the estimated score is in a normal state, it can be estimated that although a temporary disturbance of the autonomic nervous system has occurred, it will be resolved by improving lifestyle habits.

[0047] On the other hand, if the autonomic nervous state score is judged to be "normal" and the estimated score is judged to be "dangerous," then for the reasons mentioned above, the result will be "Risk level 3: Fatigue and other symptoms have accumulated in the body, and immediate action is required."

[0048] Furthermore, if the results of both the autonomic nervous state score and the estimated score are "dangerous," the result will be "Risk level 4: There is a high possibility that you have a serious illness, so medical testing should be suggested."

[0049] Next, the output unit 35 sets a judgment comment based on the judgment result of the judgment unit 34 as described above, and judgment comment data including this judgment comment is output to the user terminal 2. This judgment comment represents the judgment result of the health condition of user M, and specifically, as shown in Fig. 3, the following four types of comments are set corresponding to risk levels 1 to 4.

[0050] Risk Level 1: No health issues. Risk Level 2: Your autonomic nervous system is somewhat out of whack, so please adopt daily habits that will regulate your autonomic nervous system. Risk Level 3: Fatigue and other physical symptoms have accumulated, and immediate action is required. Risk Level 4: There is a high possibility of illness, so please seek medical attention immediately.

[0051] Next, the control process of the health condition determination system 1 of this embodiment will be described with reference to Fig. 4. As shown in Fig. 4, first, an image acquisition process is executed in the user terminal 2 by operation of the user M (Fig. 4 / STEP 1).

[0052] In this image acquisition process, still images and moving images of the face of user M are captured by camera 2b using the method described above, and then image data including these images is transmitted from user terminal 2 to server 3.

[0053] When the server 3 receives this image data, the server 3 executes a health condition determination process (FIG. 4 / STEP 2). In this health condition determination process, the health condition of user M is determined based on still images and moving images of the face of user M using the method described above in the description of FIG. 3, and then determination comment data including a determination comment is transmitted from the server 3 to the user terminal 2.

[0054] When this comment data is received by the user terminal 2, a judgment comment display process is executed in the user terminal 2 (FIG. 4 / STEP 3). In this judgment comment display process, the judgment comment (see FIG. 3) is displayed on the display 2a of the user terminal 2. For example, if the judgment result of user M's health condition is risk level 1, a comment saying "There is no problem with your health condition" is displayed. This allows user M to understand his or her own health condition.

[0055] In the above-described judgment comment display process, in addition to the judgment comment, audio data may be output from the user terminal 2. Furthermore, when comment data is received by the user terminal 2, instead of the judgment comment display process, the user terminal 2 may be configured to execute a judgment comment output process that converts the judgment comment into audio data and outputs it.

[0056] As described above, the server 3 serving as the health condition determination device of the first embodiment determines the health condition of user M based on a combination of the estimated score and the autonomic nervous state score, which represent the state of mental and physical fatigue of user M. Here, the estimated score is obtained from a still image of user M's face and therefore represents the state of fatigue accumulated in user M's body and mind. Meanwhile, the autonomic nervous state score is obtained from a moving image of user M's face and therefore represents the state of user M's autonomic nervous system at the time the moving image was acquired. Therefore, when determining user M's health condition based on a combination of the estimated score and the autonomic nervous state score, the health condition of user M can be accurately determined by taking into account the state of fatigue accumulated in user M's body and mind up to the present time, as well as the current state of user M's autonomic nervous system. Furthermore, determination data representing such a determination result is output from the output unit 35 to the user terminal 2, whereby the determination result can be provided to user M.

[0057] Although the first embodiment is an example in which the health condition determination device is configured as the server 3, the health condition determination device may also be configured as a personal computer or a smartphone. For example, when the health condition determination device is configured as a smartphone, the smartphone camera may be used to capture still images and moving images of the user's face, and based on the data, a program within the smartphone may obtain the user's estimated score and autonomic nervous state score, and then a judgment comment may be displayed on the smartphone display.

[0058] Furthermore, if the health condition assessment device is configured as a personal computer, a smartphone can be connected to the personal computer, and the smartphone's camera can be used to take still and moving images of the user's face.Based on this data, a program within the personal computer can obtain the user's estimated score and autonomic nervous state score, and then a assessment comment can be displayed on the display of the personal computer or smartphone.

[0059] In addition, the first embodiment is an example in which the Chalder fatigue scale is used as the estimated score, but instead, any one of OSI, isoprostane, 8OHdG, homovanillic acid, vanillylmandelic acid, dROMs, insomnia scale, and irritability level may be used as the estimated score.

[0060] In that case, a machine learning model that outputs an estimated score of any one of OSI, isoprostane, 8OHdG, homovanillic acid, vanillylmandelic acid, dROMs, insomnia scale, and irritability level may be configured to be stored in the storage unit 31. Furthermore, when using such a machine learning model, any one of the eyes, cheeks, mouth, and entire face in a still image of a face may be configured to be used as input to the machine learning model depending on the type of estimated score.

[0061] In this case, the machine learning model of irritability is created as follows. That is, by conducting a survey on a large number of subjects (e.g., several hundred people), the subjective irritability symptom score of each subject and a facial image at that time are obtained in a linked state. Then, the irritability score is used as a label, and the acquired facial images are used as training data, whereby model parameters of the machine learning model are learned. As a result, a machine learning model is created that uses facial images as input and outputs the level of irritability as a value within a range of 0 to 1.

[0062] Alternatively, two or more of the estimated scores of the Chalder fatigue scale, OSI, isoprostane, 8OHdG, homovanillic acid, vanillylmandelic acid, dROMs, insomnia scale, and irritability level may be used as the estimated score, or the average value of these nine estimated scores may be used. In this case, two or more machine learning models or nine machine learning models that output these estimated scores may be stored in the storage unit 31.

[0063] Even when using the estimated score as described above, the threshold of the estimated score is not limited to 0.5 and can be set to an appropriate value other than 0.5 based on the age group of user M, etc. For example, when the age group of user M is relatively young, it may be configured such that the threshold of the estimated score is set to a value greater than 0.5 (for example, 0.6).

[0064] Also, in the first embodiment, CVR-R is used as the autonomic nerve state score, but instead, the heart rate or LF / HF may be used. In that case, the heart rate is calculated by the method described above using the moving image of the user's face.

[0065] Also, LF / HF represents the autonomic nerve balance of user M (that is, the balance between the sympathetic nerve and the parasympathetic nerve) and is calculated as follows. That is, isochronous interval data (R-R interval) is calculated from the moving image of the face by the method described above, and power spectrum analysis of this isochronous interval data (R-R interval) is performed. Then, from the result of this power spectrum analysis, LF / HF is calculated as the ratio of the total power in the low frequency band to the total power in the high frequency band in the heart rate variation. This LF / HF is calculated as a value within the range of 0 to ∞.

[0066] And in the determination unit 34, based on LF / HF, the autonomic nerve balance of user M is determined. Specifically, when LF / HF < 2, the autonomic nerve balance is determined to be parasympathetic nerve dominant, when LF / HF ≥ 5, the autonomic nerve balance is determined to be sympathetic nerve dominant, and when 2 < LF / HF < 5, the autonomic nerve balance is determined to be normal. Note that the above thresholds of LF / HF can be appropriately changed according to characteristics such as the age and gender of the assumed user.

[0067] Furthermore, in the first embodiment, a smartphone type is used as the user terminal 2, but instead, a notebook personal computer or a tablet personal computer, etc. may be used as the user terminal 2.

[0068] Next, a health condition determination device according to a second embodiment of the present invention will be described. The health condition determination device according to this embodiment is configured with a server 3 in terms of hardware, similar to the health condition determination device according to the first embodiment. However, some of the functions of this server 3 are different from those of the server 3 according to the first embodiment, and therefore the following description will focus on the differences.

[0069] In this embodiment, the storage unit 31 stores two machine learning models that respectively output a fatigue score and an insomnia score as estimated scores. The fatigue score is data that represents an estimated result of the user's fatigue state, and the insomnia score is data that represents an estimated result of the user's insomnia state. In this embodiment, the Chalder fatigue scale is used as the fatigue score, and the Pittsburgh Sleep Quality Index (PSQI), which is an insomnia scale, is used as the insomnia score.

[0070] The fatigue score may be any of OSI, isoprostane, 8OHdG, homovanillic acid, vanillylmandelic acid, dROMs, and irritability, or the average of these and the Chalder Fatigue Scale. The insomnia score may be the Athens Insomnia Scale instead of the PSQI.

[0071] Furthermore, when a still image of user M's face is input from the user terminal 2, the estimated score acquisition unit 32 acquires user M's fatigue score and insomnia score using two machine learning models in the memory unit 31.

[0072] Meanwhile, the autonomic nervous state score acquisition unit 33 calculates CVR-R as the autonomic nervous state score of the user M using the method described above.

[0073] Furthermore, the judgment unit 34 judges whether user M is highly fatigued and in a dangerous state based on the fatigue score and insomnia score, and judges whether user M's autonomic nerves are in a dangerous state based on the autonomic nerve state score.

[0074] Specifically, if the fatigue score is equal to or greater than a threshold value of 0.5, it is determined that user M has a high level of fatigue and is in a dangerous state, and in other cases it is determined that user M has a low level of fatigue and is in a normal state. Furthermore, if the insomnia score is equal to or greater than a threshold value of 0.5, it is determined that user M is in an insomnia state and is in a dangerous state of fatigue, and in other cases it is determined that user M is in a normal sleeping state and is in a low level of fatigue and is in a normal state.

[0075] Furthermore, if the autonomic nerve state score is below the threshold value of 2%, it is determined that user M's autonomic nerves are in a dangerous state, and otherwise it is determined that user M's autonomic nerves are in a normal state.

[0076] Then, the determination unit 34 determines the health condition of the user M based on a combination of the determination results of the autonomic nervous state score, fatigue score, and insomnia score, as shown in Fig. 5. As shown in the figure, when the determination results of the autonomic nervous state score, fatigue score, and insomnia score are all "normal," the user M is determined to be at "risk level A1: no problem."

[0077] Furthermore, when the results of the autonomic nervous system state score, fatigue score, and insomnia score are "normal," "normal," and "risk," respectively, the patient is assessed as "risk level A2: simply lack of sleep." This is because, although the patient is temporarily in a state of insomnia, it can be estimated that the autonomic nervous system is in a normal state and the level of fatigue is low.

[0078] Furthermore, when the results of the autonomic nervous system state score, fatigue score, and insomnia score are "normal," "risk," and "normal," respectively, the system determines the "risk level A3: temporary problem that can be resolved with adequate rest." This is because, although the level of fatigue is high, it can be assumed that the autonomic nervous system is in a normal state and the person is sleeping well.

[0079] On the other hand, if the results of the autonomic nervous system state score, fatigue score, and insomnia score are "dangerous," "normal," and "normal," respectively, the person is judged to have "Risk level A4: The autonomic nervous system is somewhat disturbed, but it has no effect on the body. It can be dealt with by adopting lifestyle habits that regulate the autonomic nervous system." This is because, although the autonomic nervous system is in a state of disturbance, the fatigue level is low and the sleeping state is estimated to be normal.

[0080] Furthermore, when the results of the autonomic nervous system state score, fatigue score, and insomnia score are "normal," "dangerous," and "dangerous," respectively, the system determines the "risk level A5: possible overwork." This is because it is estimated that although the autonomic nervous system is in a normal state, the level of fatigue is high and the person is in a state of insomnia.

[0081] Furthermore, when the results of the autonomic nervous system state score, fatigue score, and insomnia score are "dangerous," "normal," and "dangerous," respectively, the system determines that the patient is at "Risk Level A6: The risk of overwork is low, but other factors are affecting sleep. A medical diagnosis is necessary." This is because, although the patient's fatigue level is low, it is estimated that the patient has a disturbed autonomic nervous system and is in a state of insomnia.

[0082] On the other hand, if the results of the autonomic nervous system state score, fatigue score, and insomnia score are "dangerous," "dangerous," and "normal," respectively, the system will determine "Risk level A7: There is a risk of overwork. The autonomic nervous system is also being affected, and medical examinations should be suggested." This is because, although there is no problem with the sleep state, it can be estimated that the autonomic nervous system is in a disturbed state and the level of fatigue is high.

[0083] Furthermore, if the autonomic nervous system state score, fatigue score, and insomnia score are all judged to be "dangerous," the system will judge the patient as "Risk level A8: Risk of overwork. Not getting enough sleep and not being able to recover. There is a high possibility of a serious illness, and we recommend immediate medical examination." This is because it is estimated that the autonomic nervous system, fatigue level, and sleep state are all in a dangerous state.

[0084] Next, the output unit 35 sets a judgment comment based on the judgment result of the judgment unit 34 as described above, and outputs judgment comment data including the judgment comment to the user terminal 2. Specifically, as the judgment comment, the following various comments are set corresponding to the risk levels A1 to A8, as shown in FIG.

[0085] Risk level A1: No health problems. Risk level A2: You are likely not getting enough sleep, so please get enough sleep. Risk Level A3: Adequate rest is required to recover. Risk level A4: Your autonomic nervous system is somewhat out of whack, so please adopt daily habits that will regulate your autonomic nervous system. Risk level A5: You may be tired due to a heavy workload, so we recommend a moderate workload. Risk level A6: You are having trouble sleeping, so please visit a doctor. Risk level A7: You may have a medical problem, so please get checked at a hospital. Risk level A8: There is a high possibility of illness, so please get examined at a hospital as soon as possible.

[0086] The judgment comment data is then received by the user terminal 2 and displayed on the display 2a of the user terminal 2, allowing the user M to properly know his or her own health condition.

[0087] As described above, the server 3 serving as the health condition determination device of the second embodiment determines the health condition of user M based on a combination of the fatigue score, insomnia score, and autonomic nerve state score. Therefore, the health condition of user M can be accurately determined by taking into account the current state of user M's autonomic nerves in addition to the state of fatigue accumulated in user M's mind and body up to the present time and user M's current state of insomnia.

[0088] Next, a health state determination device according to the third embodiment of the present invention will be described. As hardware, the health state determination device of the present embodiment is configured by the server 3 in the same manner as the health state determination device of the first embodiment. In the case of this server 3, since some of its functions are different from those of the server 3 of the first embodiment, the following description will focus on the differences.

[0089] In the case of the present embodiment, when a moving image of the face of user M is input from the user terminal 2, the autonomic nerve state score acquisition unit 33 described above calculates, as the autonomic nerve state score of user M, in addition to CVR-R, the aforementioned LF / HF.

[0090] On the other hand, when a still image of the face of user M is input from the user terminal 2, the estimation score acquisition unit 32 uses the machine learning model in the storage unit 31 to acquire the estimation score (Chalder fatigue scale) of user M.

[0091] Furthermore, as described above, the determination unit 34 determines whether user M is in a dangerous state with high fatigue based on the estimated score, and determines whether the autonomic nerve of user M is in a dangerous state based on CVR-R.

[0092] In addition to this, the determination unit 34 determines the autonomic nerve balance of user M based on LF / HF. Specifically, as described above, when LF / HF < 2, the autonomic nerve balance is determined to be parasympathetic nerve dominant, when LF / HF ≧ 5, the autonomic nerve balance is determined to be sympathetic nerve dominant, and when 2 < LF / HF < 5, the autonomic nerve balance is determined to be normal.

[0093] Then, the determination unit 34 determines the health state of user M based on the combination of the determination results of the above CVR-R, LF / HF, and estimated score and the time at the time of determination execution. Specifically, when the time at the time of determination execution is in the time zone from 8 pm to before going to bed, it is determined as shown in FIGS. 6A to 6B, and when the time at the time of determination execution is in the time zone from waking up to 8 pm, it is determined as shown in FIGS. 7A to 7B.

[0094] The reason why the determination result differs depending on the time when the determination is made is as follows: The time period from 8 pm to before going to bed is assumed to be a time period when user M is likely to go to bed and user M should rest both physically and mentally, whereas the time period from waking up to 8 pm is assumed to be a time period when user M is likely to be physically active for work or the like and user M should be alert both physically and mentally. In this embodiment, data on the time when the determination is made is transmitted from user terminal 2 to server 3 together with image data.

[0095] First, with reference to Figures 6A and 6B, the results of the assessment when the assessment is performed between 8 pm and before bedtime will be described. As shown in Figure 6A, when the assessment results of CVR-R, LF / HF, and estimated score are all "normal," the assessment is "risk level B1: no problem."

[0096] Furthermore, when the results of CVR-R, LF / HF, and estimated score are "normal," "parasympathetic dominant," and "normal," respectively, the patient is assessed as "risk level B2: ideal state of being energetic and moderately relaxed." This is because the patient is estimated to be in a relaxed state due to a low level of fatigue and a parasympathetic dominant state.

[0097] Furthermore, when the results of the CVR-R, LF / HF, and estimated score are "normal," "sympathetic dominant," and "normal," respectively, the patient is assessed as "Risk level B3: A little excited, but the body is in a healthy state. However, it may be difficult to fall asleep." This is because the patient is estimated to be in a slightly excited state due to a low level of fatigue and a state in which the sympathetic nervous system is dominant.

[0098] On the other hand, when the results of CVR-R, LF / HF, and estimated score are "Danger," "Normal," and "Normal," respectively, the patient is assessed as "Risk Level B4: Autonomic nervous system is slightly disturbed, but not yet at a level that can be noticed (hidden fatigue)." This is because although the autonomic nervous system is slightly disturbed, the level of fatigue is estimated to be low.

[0099] Furthermore, when the results of the CVR-R, LF / HF, and estimated score are "dangerous," "parasympathetic dominant," and "normal," respectively, the patient is assessed as "Risk level B5: autonomic nervous system is somewhat disturbed, but not yet at a level that can be noticed (hidden fatigue)." This is because although the autonomic nervous system is somewhat disturbed, the level of fatigue is estimated to be low.

[0100] Furthermore, when the results of the CVR-R, LF / HF, and estimated score are "dangerous," "sympathetic dominant," and "normal," respectively, the patient is assessed as "Risk level B6: autonomic nervous system is slightly disturbed, but the effect on the body has not yet been felt. The patient is slightly excited and has difficulty falling asleep." This is because, although the patient's fatigue level is low, the autonomic nervous system is slightly disturbed, and although the patient's fatigue level is low, the patient is estimated to be slightly excited.

[0101] On the other hand, as shown in Figure 6B, when the results of CVR-R, LF / HF, and estimated score are "normal," "parasympathetic dominant," and "risk," respectively, the patient is judged to be at "risk level B7: Feeling fine today, but fatigue is building up." This is because it can be estimated that the autonomic nervous system is not in a disordered state, but the patient is in a state of high fatigue.

[0102] Furthermore, when the results of the CVR-R, LF / HF, and estimated score are "normal," "sympathetic dominant," and "risk," respectively, the patient is assessed as "Risk level B8: Difficulty falling asleep and fatigue. Possibility of further fatigue." This is because it can be estimated that the patient is in a slightly excited state and has a high level of fatigue.

[0103] Furthermore, when the results of the CVR-R, LF / HF, and estimated score are "Dangerous," "Normal," and "Dangerous," respectively, the system determines the patient as "Risk Level B9: Autonomic nervous system is somewhat disturbed and the body is tired. Consultation with a medical institution is recommended." This is because it is estimated that the autonomic nervous system is in a state of disorder and the patient is in a state of high fatigue.

[0104] On the other hand, if the results of the CVR-R, LF / HF, and estimated score are "dangerous," "parasympathetic dominant," and "dangerous," respectively, the patient is assessed as "Risk level B10: Autonomic nervous system is somewhat disturbed and the body is tired. Consultation with a medical institution is recommended." This is because it is estimated that the autonomic nervous system is in a state of disorder and the patient is in a state of high fatigue.

[0105] Furthermore, when the results of the CVR-R, LF / HF, and estimated score are "dangerous," "sympathetic dominant," and "dangerous," respectively, the patient is assessed as "Risk level B11: Autonomic nervous system is somewhat disturbed and the body is tired. Consultation with a medical institution is recommended." This is because it is estimated that the autonomic nervous system is in a state of disorder and the patient is in a state of high fatigue.

[0106] Next, the output unit 35 sets a judgment comment based on the judgment result of the judgment unit 34 as described above, and outputs judgment comment data including the judgment comment to the user terminal 2. Specifically, as the judgment comment, the following various comments are set corresponding to the risk levels B1 to B11, as shown in FIGS. 6A to 6B.

[0107] Risk level B1: No health problems. Risk Level B2: Ideally relaxed state. Risk level B3: You are a little excited. You may have difficulty falling asleep. Risk level B4: Your autonomic nervous system is somewhat out of whack. Please be mindful of your lifestyle habits to regulate your autonomic nervous system. Risk level B5: Your autonomic nervous system is somewhat out of whack. Please be mindful of your lifestyle habits to regulate your autonomic nervous system. Risk level B6: Your autonomic nervous system is somewhat disturbed. You may have difficulty falling asleep. Please be mindful of your lifestyle habits to regulate your autonomic nervous system.

[0108] Risk level B7: You seem to be a little tired. Please try to recover from fatigue. Risk level B8: You seem to be fatigued. Please try to recover from fatigue. Risk level B9: You may have a medical problem, so please get checked at a hospital. Risk level B10: There is a possibility that you have a medical problem, so please get checked at a hospital. Risk level B11: You may have a medical problem and should be examined at a hospital.

[0109] Next, the results of the determination when the determination is made in the time period from wake-up to 8 pm will be described with reference to Figures 7A and 7B. As shown in Figure 7A, when the determination results of CVR-R, LF / HF, and estimated score are all "normal," the result is determined to be "risk level C1: no problem."

[0110] Furthermore, when the results of the CVR-R, LF / HF, and estimated score are "normal," "sympathetic dominant," and "normal," respectively, the patient is assessed as "Risk level C2: Highly motivated, but be careful not to be too enthusiastic." This is because the patient is estimated to be in a slightly excited state due to a low level of fatigue and a sympathetic dominant state.

[0111] Furthermore, when the results of the CVR-R, LF / HF, and estimated score are "normal," "parasympathetic dominant," and "normal," respectively, the patient is assessed as "risk level C3: physical condition is OK, but being too relaxed may interfere with work. Motivation is low." This is because the patient is estimated to be in a relaxed state due to a low level of fatigue and a parasympathetic dominant state.

[0112] On the other hand, when the results of CVR-R, LF / HF, and estimated score are "Danger," "Normal," and "Normal," respectively, the patient is assessed as "Risk level C4: Autonomic nervous system is slightly disturbed, but not yet at a level that can be noticed (hidden fatigue)." This is because although the autonomic nervous system is slightly disturbed, the level of fatigue is estimated to be low.

[0113] Furthermore, when the results of the CVR-R, LF / HF, and estimated score are "danger," "sympathetic dominant," and "normal," respectively, the patient is assessed as "Risk level C5: The autonomic nervous system is somewhat disturbed, but the physical effects have not yet been felt. The patient is motivated, but may not have the stamina to endure. Be careful not to push themselves too hard." This is because the autonomic nervous system is somewhat disturbed and the patient is in a slightly excited state, but the fatigue level is estimated to be low.

[0114] Furthermore, when the results of the CVR-R, LF / HF, and estimated score are "dangerous," "parasympathetic dominant," and "normal," respectively, the patient is assessed as "Risk level C6: Autonomic nervous system slightly disturbed, and motivation is low." This is because although the fatigue level is low, the autonomic nervous system is slightly disturbed, and although the fatigue level is low, it is estimated that the patient is in a slightly excited state.

[0115] On the other hand, as shown in Figure 7B, when the results of CVR-R, LF / HF, and estimated score are "normal," "parasympathetic dominant," and "risk," respectively, the patient is determined to be at "risk level C7: tired and unmotivated." This is because the patient is in a state of high fatigue and is estimated to have a decreased state of physical and mental activity.

[0116] Furthermore, when the results of the CVR-R, LF / HF, and estimated score are "normal," "sympathetic dominant," and "dangerous," respectively, the system determines the risk level as "C8: Possibility of abnormalities in the body due to overexertion. Be careful not to overexert yourself." This is because it is estimated that the patient is in a state of slight arousal and high fatigue.

[0117] Furthermore, when the results of the CVR-R, LF / HF, and estimated score are "Dangerous," "Normal," and "Dangerous," respectively, the patient is assessed as "Risk level C9: Autonomic nervous system is somewhat disturbed and the body is tired. Consultation with a medical institution is recommended." This is because it is estimated that the autonomic nervous system is in a state of disorder and the patient is in a state of high fatigue.

[0118] On the other hand, if the results of the CVR-R, LF / HF, and estimated score are "dangerous," "parasympathetic dominant," and "dangerous," respectively, the patient is assessed as "Risk level C10: Autonomic nervous system is somewhat disturbed and the body is tired. Consultation with a medical institution is recommended." This is because it is estimated that the autonomic nervous system is in a state of disorder and the patient is in a state of high fatigue.

[0119] Furthermore, when the results of the CVR-R, LF / HF, and estimated score are "dangerous," "sympathetic dominant," and "dangerous," respectively, the patient is assessed as "Risk level C11: Autonomic nervous system is somewhat disturbed and the body is tired. Consultation with a medical institution is recommended." This is because it is estimated that the autonomic nervous system is in a state of disorder and the patient is in a state of high fatigue.

[0120] Next, the output unit 35 sets a judgment comment based on the judgment result of the judgment unit 34 as described above, and outputs judgment comment data including the judgment comment to the user terminal 2. Specifically, as the judgment comment, the following various comments are set corresponding to the risk levels C1 to C11, as shown in FIGS. 7A to 7B.

[0121] Risk level C1: No health problems. Risk Level C2: You are highly motivated. Be careful not to get too excited. Risk Level C3: Very relaxed. You may not feel motivated to work. Risk level C4: Your autonomic nervous system is somewhat disturbed. Please be mindful of your lifestyle habits to regulate your autonomic nervous system. Risk level C5: Your autonomic nervous system is somewhat disturbed. Be careful not to work too hard. Risk level C6: Your autonomic nervous system is somewhat disturbed. Please be mindful of your lifestyle habits to regulate your autonomic nervous system.

[0122] Risk Level C7: You appear to be fatigued. Please try to recover from fatigue. Risk level C8: You seem to be fatigued. Please try to recover from fatigue. Risk level C9: There is a possibility of a problem with your body. Please get checked at a hospital. Risk level C10: There is a possibility that you have a medical problem, so please get checked at a hospital. Risk level C11: You may have a medical problem, so please get checked at a hospital.

[0123] The judgment comment data including the judgment comment set as described above is output from the output unit 35 to the user terminal 2. The judgment comment data is then received by the user terminal 2 and displayed on the display 2a of the user terminal 2, allowing the user M to properly know his or her own health condition.

[0124] As described above, the server 3 serving as the health condition determination device of the third embodiment determines the health condition of user M based on the combination of CVR-R, LF / HF, and estimated score, and the time when the determination is performed. Therefore, the health condition of user M can be accurately determined with content appropriate for the time when the determination is performed, taking into account the current state and autonomic nervous balance of user M in addition to the state of fatigue accumulated in user M's mind and body up to the present time.

[0125] In the third embodiment, the Chalder fatigue scale is used as the estimated score, but instead, any one of OSI, isoprostane, 8OHdG, homovanillic acid, vanillylmandelic acid, dROMs, insomnia scale, and irritability level may be used as the estimated score.

[0126] Next, a health condition determination device according to a fourth embodiment of the present invention will be described. The health condition determination device according to this embodiment is configured with a server 3 in terms of hardware, similar to the health condition determination device according to the first embodiment. However, some of the functions of this server 3 are different from those of the server 3 according to the first embodiment, and therefore the following description will focus on the differences.

[0127] In this embodiment, three machine learning models that respectively output a fatigue score, an insomnia score, and an irritability level as estimated scores are stored in the storage unit 31. In this embodiment, the fatigue score is the Chalder fatigue scale, and the insomnia score is the PSQI, which is the insomnia scale.

[0128] The fatigue score may be OSI, isoprostane, 8OHdG, homovanillic acid, vanillylmandelic acid, or dROMs, or the average of these and the Chalder Fatigue Scale. The insomnia score may be the Athens Insomnia Scale instead of PSQI.

[0129] Furthermore, when a still image of user M's face is input from the user terminal 2, the estimated score acquisition unit 32 uses the three machine learning models in the memory unit 31 to acquire user M's fatigue score, insomnia score, and irritability level.

[0130] Meanwhile, the autonomic nervous state score acquisition unit 33 calculates CVR-R as the autonomic nervous state score of user M using the above-mentioned method. Note that the above-mentioned LF / HF may be used as the autonomic nervous state score of user M instead of CVR-R.

[0131] Furthermore, as described above, the judgment unit 34 judges whether user M's fatigue level and insomnia level are high and in a dangerous state based on the fatigue score and insomnia score, and judges whether user M's irritation level is high and in a dangerous state based on the irritation level, and as described above, judges whether user M's autonomic nerves are in a dangerous state based on the autonomic nerve state score.

[0132] In this case, if the frustration level is equal to or greater than the threshold value 0.5, it is determined that user M's frustration level is high and that he or she is in a dangerous state, and otherwise it is determined that user M's frustration level is low and that he or she is in a normal state.

[0133] Then, the judgment unit 34 judges the health state of the user M based on a combination of the judgment results of the autonomic nerve state score, fatigue score, insomnia score, and irritability level, as shown in Figures 8A to 8B. As shown in Figure 8A, when the judgment results of the autonomic nerve state score, fatigue score, insomnia score, and irritability level are all "normal," the health state of the user M is judged to be "risk level D1: no problem."

[0134] Furthermore, when the results of the autonomic nervous state score, fatigue score, insomnia score, and irritability level are "normal," "normal," "normal," and "dangerous," respectively, the system determines that the user M is experiencing "risk level D2: temporary emotional change, no problem." This is because it can be estimated that the user M is temporarily irritated.

[0135] Furthermore, when the results of the autonomic nervous system state score, fatigue score, insomnia score, and irritability level are "normal," "normal," "risk," and "normal," respectively, the patient is judged to be at "risk level D3: simply lack of sleep." This is because it can be estimated that although the patient is in a temporary state of insomnia, the autonomic nervous system is in a normal state and the fatigue and irritability levels are low.

[0136] On the other hand, when the results of the autonomic nervous system state score, fatigue score, insomnia score, and irritability level are "normal," "normal," "dangerous," and "dangerous," respectively, the patient is judged to be at "risk level D4: a state where the patient is easily irritated due to lack of sleep." This is because it is estimated that the autonomic nervous system is in a normal state, the fatigue level is low, but the irritability level is high due to insomnia.

[0137] Furthermore, when the results of the autonomic nervous system state score, fatigue score, insomnia score, and irritability level are "normal," "dangerous," "normal," and "normal," respectively, the patient is judged to be at "Risk level D5: temporary problem that can be resolved with adequate rest." This is because it can be estimated that there is no problem with the autonomic nervous system and sleep state, and the irritability level is low, but the fatigue level is high.

[0138] Furthermore, when the results of the autonomic nervous system state score, fatigue score, insomnia score, and irritability level are "normal," "dangerous," "normal," and "dangerous," respectively, the system determines "risk level D6: fatigue may be affecting emotions." This is because it is estimated that although there is no problem with the autonomic nervous system or sleep state, the fatigue level and irritability level are both high.

[0139] On the other hand, if the results of the autonomic nervous system state score, fatigue score, insomnia score, and irritability level are "dangerous," "normal," "normal," and "normal," respectively, the system will determine "Risk level D7: The autonomic nervous system is somewhat disturbed, but it will not affect the body. This can be addressed by adopting lifestyle habits that regulate the autonomic nervous system." This is because it is estimated that the autonomic nervous system is in a state of disorder, even though there is no problem with the sleep state and the fatigue and irritability levels are low.

[0140] Furthermore, when the results of the autonomic nervous system state score, fatigue score, insomnia score, and irritability level are "dangerous," "normal," "normal," and "dangerous," respectively, the system determines that "Risk level D8: The autonomic nervous system may be somewhat disturbed, affecting emotions. This can be addressed by adopting lifestyle habits that regulate the autonomic nervous system." This is because it is estimated that although the fatigue level is low and there are no problems with the sleep state, the autonomic nervous system is somewhat disturbed and the irritability level is high.

[0141] Furthermore, when the results of the autonomic nervous system state score, fatigue score, insomnia score, and irritability level are "normal," "dangerous," "dangerous," and "normal," respectively, the person is judged to be at "risk level D9: Possible overwork." This is because it is estimated that although there are no problems with the state of the autonomic nervous system or sleep, the fatigue level and irritability level are high.

[0142] On the other hand, as shown in Figure 8B, when the assessment results of the autonomic nervous system state score, fatigue score, insomnia score, and irritability level are "dangerous," "normal," "dangerous," and "normal," respectively, it is determined that "Risk level D10: The risk of overwork is low, but other factors are affecting sleep. A medical diagnosis is suggested." This is because it can be estimated that although the fatigue level and irritability level are low, the autonomic nervous system is in a state of disorder and the person is in a state of insomnia.

[0143] Furthermore, when the results of the autonomic nervous system state score, fatigue score, insomnia score, and irritability level are "dangerous," "dangerous," "normal," and "normal," respectively, the system determines that "Risk level D11: There is a risk of overwork. The autonomic nervous system is also being affected, and a medical examination is recommended." This is because it is estimated that although there is no problem with the sleep state and the irritability level is low, the autonomic nervous system is in a state of disorder and fatigue is high.

[0144] Furthermore, when the results of the autonomic nervous system state score, fatigue score, insomnia score, and irritability level are "dangerous," "normal," "dangerous," and "dangerous," respectively, the system will determine that "Risk level D12: Autonomic nervous system is disturbed, and recovery through sleep is difficult. You may not realize it, but if this continues, it may have an impact on your mental health. Medical examination is recommended." This is because it is estimated that although the fatigue level is low, the autonomic nervous system is disturbed, you are in a state of insomnia, and your irritability level is high.

[0145] On the other hand, if the results of the autonomic nervous system state score, fatigue score, insomnia score, and irritability level are "dangerous," "dangerous," "normal," and "dangerous," respectively, the system will determine that the patient is at risk level D13: "Although the patient is getting enough sleep, both the body and the autonomic nervous system are being affected. Medical examination is recommended." This is because it is estimated that although there is no problem with the sleep state, the autonomic nervous system is in a disturbed state, and the patient is in a state of high fatigue and irritability.

[0146] Furthermore, when the results of the autonomic nervous system state score, fatigue score, insomnia score, and irritability level are "normal," "dangerous," "dangerous," and "dangerous," respectively, the system determines that "Risk level D14: Overwork may be affecting emotions. Medical examination is recommended." This is because it is estimated that although there is no problem with the autonomic nervous system, the person is in a state of high fatigue, insomnia, and high irritability.

[0147] On the other hand, if the assessment results for the autonomic nervous system state score, fatigue score, insomnia score, and irritability level are "dangerous," "dangerous," "dangerous," and "normal," respectively, the system will assess the patient as "Risk level D15: Risk of overwork. Not getting enough sleep and not being able to recover. There is a high possibility of a serious illness, and we recommend immediate medical examination." This is because, although the irritability level is low, it is estimated that there are problems with the state of the autonomic nervous system and sleep, and that the fatigue level is high.

[0148] Furthermore, if the autonomic nervous system state score, fatigue score, insomnia score, and irritability score are all judged to be "dangerous," the system will judge the patient as "Risk level D16: Very dangerous condition. We suggest that you consult a medical institution immediately." This is because it is estimated that the autonomic nervous system, fatigue level, sleep level, and irritability level are all in a dangerous condition.

[0149] Next, the output unit 35 sets a judgment comment based on the judgment result of the judgment unit 34 as described above, and outputs judgment comment data including the judgment comment to the user terminal 2. Specifically, as the judgment comment, the following various comments are set corresponding to the risk levels D1 to D8, as shown in FIGS. 8A to 8B.

[0150] Risk level D1: No health problems. Risk level D2: No health problems. Risk level D3: You may be sleep deprived, so get enough sleep. Risk level D4: You may be sleep deprived, so get enough sleep. Risk Level D5: Adequate rest is required to recover. Risk Level D6: Adequate rest is required to recover. Risk level D7: Your autonomic nervous system is somewhat out of whack. Please be mindful of your lifestyle habits to regulate your autonomic nervous system.

[0151] Risk level D8: Your autonomic nervous system is somewhat out of whack. Please be mindful of your lifestyle habits to regulate your autonomic nervous system. Risk Level D9: You may be tired due to a heavy workload, so we recommend a moderate workload. Risk level D10: You may have a medical problem and should be examined at a hospital. Risk level D11: You may have a medical problem and should be examined at a hospital. Risk level D12: You may have a medical problem and should be examined at a hospital. Risk level D13: You may have a medical problem and should be examined at a hospital. Risk level D14: You may have a medical problem and should be examined at a hospital. Risk level D15: There is a high possibility of illness, so please seek medical attention immediately. Risk level D16: There is a high possibility of illness, so please seek medical attention immediately.

[0152] The judgment comment data is then received by the user terminal 2 and displayed on the display 2a of the user terminal 2, allowing the user M to properly know his or her own health condition.

[0153] As described above, the server 3 serving as the health condition determination device of the fourth embodiment determines the health condition of user M based on a combination of the autonomic nerve state score, fatigue score, insomnia score, and irritability level. Therefore, the health condition of user M can be accurately determined by taking into account the current state of user M's autonomic nerves, the state of fatigue accumulated in user M's mind and body up to the present time, and user M's current insomnia state, as well as user M's current irritability level. [Explanation of symbols]

[0154] 1. Health status assessment system 2. User terminal 3 Server (health status determination device) 31 Storage section 32 Estimated score acquisition section 33 Autonomic nervous system status score acquisition unit 34 Judgment section 35 Output section

Claims

1. a storage unit that stores a plurality of fatigue state estimation models that receive a still image of a user's face as input and output a plurality of different estimation scores that represent estimation results of at least one of the user's mental and physical fatigue states; an estimated score acquisition unit that acquires at least one estimated score of the user from a still image of the user's face using at least one of the plurality of fatigue state estimation models; an autonomic nervous state score acquisition unit that acquires an autonomic nervous state score representing a state of the autonomic nervous of the user from the moving image of the user's face using a predetermined acquisition method; a determination unit that determines a health state of the user based on a combination of the at least one estimated score and the autonomic nervous state score; an output unit that outputs determination data representing the determination result by the determination unit; A health condition determination device comprising:

2. 2. The health condition determination device according to claim 1, The health condition determination device is characterized in that the at least one estimated score is a score representing at least one of the Chalfant fatigue scale, OSI (Oxidative stress index), isoprostane, 8OHdG (8-hydroxy-2'-deoxyguanosine), homovanillic acid, vanillylmandelic acid, dROMs (Diacron Reactive Oxygen Metabolites), insomnia scale, and stress level.

3. 2. The health condition determination device according to claim 1, The health condition determination device is characterized in that the autonomic nervous state score is a score representing at least one of CVR-R (Coefficient of Variation of RR intervals) and LF / HF.

4. 2. The health condition determination device according to claim 1, the at least one estimated score is composed of a fatigue score representing an estimated result of a fatigue state of the user and an insomnia score representing an estimated result of an insomnia state of the user, The health condition determination device is characterized in that the determination unit determines the health condition of the user based on a combination of the fatigue score, the insomnia score, and the autonomic nervous state score.

5. 2. The health condition determination device according to claim 1, the at least one estimated score is composed of an estimated score representing one of the following: Chalfant fatigue scale, OSI (Oxidative stress index), isoprostane, 8OHdG (8-hydroxy-2'-deoxyguanosine), homovanillic acid, vanillylmandelic acid, dROMs (Diacron Reactive Oxygen Metabolites), insomnia scale, and stress level; The autonomic nervous state score is composed of CVR-R (Coefficient of Variation of RR intervals) and LF / HF, The health condition determination device is characterized in that the determination unit determines the health condition of the user based on a combination of the estimated score, the CVR-R, and the LF / HF.

6. The health condition determination device according to claim 3 or 5, A health condition determination device characterized in that the autonomic nervous system state score is composed of the LF / HF, which indicates whether the user's autonomic nervous system balance is in a sympathetic nervous system dominant state, a parasympathetic nervous system dominant state, or a normal state.

7. 2. The health condition determination device according to claim 1, the at least one estimated score is composed of a fatigue score representing an estimated result of a fatigue state of the user, an insomnia score representing an estimated result of an insomnia state of the user, and a frustration level representing an estimated result of a frustration state of the user, The health condition determination device is characterized in that the determination unit determines the health condition of the user based on a combination of the fatigue score, the insomnia score, the irritability level, and the autonomic nervous state score.

8. a storage step of storing a plurality of fatigue state estimation models that receive a still image of a user's face as input and output a plurality of different estimation scores that represent estimation results of at least one of the user's mental and physical fatigue states; an estimation score acquisition step of acquiring at least one of the plurality of fatigue state estimation models and a still image of the face of the user, the estimation score being acquired; a score acquisition step of acquiring an autonomic nervous state score representing a state of the autonomic nervous system of the user from the moving image of the user's face using a predetermined acquisition method; determining a health state of the user based on a combination of the at least one estimated score and the autonomic nervous state score; an output step of outputting a determination result in the determination step; A health condition determination method characterized by being executed by a processing device.

Images