Biological information analysis device, biological information analysis system, and biological information analysis program
The biological information analysis device and system accurately assess mental states by combining autonomic nerve data and behavior records to provide precise mental state analysis, enhancing counseling effectiveness.
Patent Information
- Application Number
- JP2021527494
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2019-06-17
- Filing Date
- 2020-05-22
- Publication Date
- 2024-11-14
- Estimated Expiration
- 2040-05-22
AI Technical Summary
Existing methods for understanding the mental state of individuals in psychological counseling are inaccurate due to the increasing number of counseling sessions and diversification of counseling candidates, necessitating a more precise method to grasp mental states.
A biological information analysis device and system that acquires autonomic nerve data and behavior record information, calculates mental state information, and outputs this information for display, utilizing a server device and display terminal to facilitate accurate mental state assessment.
Enables accurate and objective quantification of mental states, allowing for better identification of stress-causing actions and providing tailored advice to individuals.
Smart Images

Figure 0007583373000001 
Figure 0007583373000002 
Figure 0007583373000003
Abstract
Description
[Technical field]
[0001] An embodiment of the present invention relates to a biological information analysis device, a biological information analysis system, a biological information analysis program, and a biological information analysis method. [Background technology]
[0002] In psychological counseling, it is necessary to understand the mental state of the subject. For this reason, for example, every time a counselor has a session (interview) with a subject, the subject answers a questionnaire about his or her mental state, and the answers are analyzed to infer the subject's mental state. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2018-153244 A Summary of the Invention [Problem to be solved by the invention]
[0004] Here, in recent years, in the above-mentioned workplaces, there is a demand for a more accurate understanding of mental states due to an increase in the number of counseling sessions and a diversification of the attributes of those receiving counseling.
[0005] The present invention provides a biometric information analysis device, a biometric information analysis system, a biometric information analysis program, and a biometric information analysis method that are capable of accurately grasping the mental state of a subject. [Means for solving the problem]
[0006] In order to solve the above-mentioned problems and achieve the objective, the present invention provides a bioinformation analysis device comprising an acquisition unit that acquires bioinformation including autonomic nerve data of a subject and behavioral record information in which multiple behaviors of the subject are recorded, and a calculation unit that calculates mental state information for each behavior of the subject based on the bioinformation and the behavioral record information.
[0007] The present invention also provides a bioinformation analysis system including a server device and a display terminal, wherein the server device includes an acquisition unit that acquires bioinformation including autonomic nerve data of a subject and behavioral record information in which multiple behaviors of the subject are recorded, a calculation unit that calculates mental state information for each behavior of the subject based on the bioinformation and the behavioral record information, and an output control unit that transmits the mental state information for each behavior of the subject to the display terminal, and the display terminal includes a display control unit that receives the mental state information for each behavior of the subject transmitted from the server device and displays the received mental state information for each behavior of the subject.
[0008] The present invention also relates to a bioinformation analysis program that acquires bioinformation including autonomic nerve data of a subject and behavioral record information in which multiple behaviors of the subject are recorded, and causes a computer to execute various processes to calculate mental state information for each behavior of the subject based on the bioinformation and the behavioral record information.
[0009] The present invention also provides a bioinformation analysis method that includes acquiring bioinformation including autonomic nerve data of a subject and behavioral record information in which multiple behaviors of the subject are recorded, and calculating mental state information for each behavior of the subject based on the bioinformation and the behavioral record information. Effect of the Invention
[0010] According to the present invention, it is possible to provide a biometric information analysis device, a biometric information analysis system, a biometric information analysis program, and a biometric information analysis method that can accurately grasp the mental state of a subject. [Brief description of the drawings]
[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of a schematic configuration of an information processing device according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of the information processing device according to the embodiment. [Diagram 3] FIG. 3 is a flowchart illustrating an example of the operation of the information processing device according to the embodiment. [Figure 4] FIG. 4 is a diagram for explaining the process of the acquisition unit. [Diagram 5] FIG. 5 is a diagram for explaining the process of the acquisition unit. [Figure 6] FIG. 6 is a diagram for explaining the process of the acquisition unit. [Figure 7] FIG. 7 is a diagram for explaining the process of the acquisition unit. [Figure 8] FIG. 8 is a diagram for explaining the process of the acquisition unit. [Figure 9] FIG. 9 is a diagram for explaining the process of the acquisition unit. [Figure 10] FIG. 10 is a diagram for explaining the process of the calculation unit. [Figure 11] FIG. 11 is a diagram for explaining the process of the generation unit. [Figure 12] FIG. 12 is a diagram for explaining the process of the generation unit. [Figure 13] FIG. 13 is a diagram for explaining the process of the generation unit. [Figure 14] FIG. 14 is a diagram for explaining the process of the output control unit. [Figure 15] FIG. 15 is a diagram for explaining the process of the output control unit. [Figure 16] FIG. 16 is a diagram for explaining the process of the output control unit. [Figure 17] FIG. 17 is a diagram for explaining the process of the output control unit. [Figure 18] FIG. 18 is a diagram for explaining the process of the output control unit. [Figure 19]FIG. 19 is a diagram for explaining the process of the output control unit. [Figure 20] FIG. 20 is a diagram illustrating an example of a schematic configuration of an information processing device according to the second embodiment. [Figure 21] FIG. 21 is a diagram showing processes during learning and operation performed by the information processing device according to the second embodiment. [Figure 22] FIG. 22 is a flowchart illustrating an example of the operation of the information processing device according to the second embodiment. [Figure 23] FIG. 23 is a diagram illustrating an example of a schematic configuration of a system according to an embodiment. [Figure 24] FIG. 24 is a diagram for explaining trained models according to each of the examples and comparative examples. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] Hereinafter, a biometric analysis device, a biometric analysis system, a biometric analysis program, and a biometric analysis method according to embodiments will be described with reference to the drawings. Note that the following embodiments are not limited to the following description. Also, each embodiment can be combined with other embodiments or conventional techniques as long as there is no contradiction in the processing content.
[0013] In the following, a case where this embodiment is applied to psychological counseling will be described. That is, a case where an information processing device according to this embodiment is used by a person who provides psychological counseling (e.g., a counselor, a doctor, etc.) to understand the mental state of a subject of psychological counseling will be described. This embodiment is not limited to psychological counseling, and can be applied in various situations to understand the mental state of a subject.
[0014] (First embodiment) Fig. 1 is a diagram showing an example of a schematic configuration of an information processing device according to an embodiment. As shown in Fig. 1, an information processing device 20 according to an embodiment is capable of directly or indirectly communicating with a sensor 11 and a mobile terminal 12 via a network such as a LAN (Local Area Network) or a WAN (Wide Area Network). The information processing device 20 is an example of a bioinformation analysis device.
[0015] The sensor 11 and the mobile terminal 12 do not need to be constantly connected to the information processing device 20. For example, it is sufficient that they are connected to exchange various information when a session with a subject is performed. Furthermore, when exchanging information via an external recording medium or the like, the sensor 11 and the mobile terminal 12 do not need to be connected to the information processing device 20.
[0016] The sensor 11 is a sensor device attached to a subject. For example, the sensor 11 is a multi-sensor device equipped with multiple sensor functions. Specifically, the sensor 11 has the functions of an electrocardiograph, an optical pulse wave meter, a body motion meter, and a thermistor. The electrocardiograph measures time-series electrocardiogram waveform data. The optical pulse wave meter measures time-series pulse wave data. The body motion meter measures time-series acceleration data. The thermistor measures time-series temperature data. The sensor 11 records the various measured measurement data in a memory inside the device in association with the measurement start time. It should be noted that the electrocardiograph, the optical pulse wave meter, the body motion meter, and thermistor can be appropriately selected and applied from known technologies.
[0017] Here, the sensor 11 is basically always attached to the subject. When a session is held, the subject removes the sensor 11 and hands it over to an operator of the information processing device 20 (typically, a counselor or a doctor). The operator of the information processing device 20 transfers the measurement data recorded after the previous session from the sensor 11 received from the subject to the information processing device 20. Details of the process of acquiring bioinformation from the measurement data will be described later.
[0018] The sensor 11 is not necessarily limited to a multi-sensor device, but may have at least the function of an electrocardiograph. The sensor 11 is not necessarily limited to being worn at all times, but may be removed as long as it does not affect the biological information analysis process described below. The timing of transfer of the measurement data recorded in the sensor 11 is not necessarily limited to during a session, but may be periodically (e.g., once a day) via a network, for example.
[0019] The mobile terminal 12 is, for example, a smartphone carried by the subject, and a recording application for collecting behavior record information in which the subject's behavior is recorded is introduced (installed) into the mobile terminal 12. The behavior record information is recorded in the internal memory of the mobile terminal 12 automatically or manually by the subject. The behavior record information recorded in the mobile terminal 12 is transferred to the information processing device 20 at any timing. The process of acquiring the behavior record information will be described later in detail.
[0020] The mobile terminal 12 is not necessarily limited to a smartphone, and may be any information processing device into which an application for collecting behavioral record information can be installed, such as a tablet or a personal computer. However, due to the nature of collecting behavioral record information, it is preferable that the mobile terminal 12 is a device (mobile terminal) that the subject can carry around with him or her on a daily basis.
[0021] The information processing device 20 is, for example, a computer such as a personal computer or a workstation. The information processing device 20 calculates mental state information indicating the mental state of the subject based on information acquired by the sensor 11 and the mobile terminal 12. The information processing device 20 includes an acquisition unit 201, a calculation unit 202, a generation unit 203, and an output control unit 204. Note that the functions of the information processing device 20 are not limited to the acquisition unit 201, the calculation unit 202, the generation unit 203, and the output control unit 204. The acquisition unit 201, the calculation unit 202, the generation unit 203, and the output control unit 204 will be described later.
[0022] Here, a hardware configuration of the information processing device 20 will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of a hardware configuration of the information processing device 20 according to an embodiment. As shown in Fig. 2, the information processing device 20 includes a CPU (Central Processing Unit) 21, a ROM (Read Only Memory) 22, a RAM (Random Access Memory) 23, an auxiliary storage device 24, an input device 25, a display device 26, and an external I / F (Interface) 27.
[0023] The CPU 21 is a processor (processing circuit) that executes a program to comprehensively control the operation of the information processing device 20 and realize various functions of the information processing device 20. The various functions of the information processing device 20 will be described later.
[0024] The ROM 22 is a non-volatile memory and stores various data (information written during the manufacturing stage of the information processing device 20) including a program for starting up the information processing device 20. The RAM 23 is a volatile memory having a working area for the CPU 21. The auxiliary storage device 24 stores various data such as the program executed by the CPU 21. The auxiliary storage device 24 is configured, for example, by a hard disk drive (HDD), a solid state drive (SSD), or the like.
[0025] The input device 25 is a device for performing various operations by an operator using the information processing device 20. The input device 25 is configured with, for example, a mouse, a keyboard, a touch panel, or hardware keys. The operator corresponds to, for example, a medical professional such as a doctor.
[0026] The display device 26 displays various information. For example, the display device 26 displays image data, model data, a GUI (Graphical User Interface) for receiving various operations from an operator, medical images, and the like. The display device 26 is configured, for example, as a liquid crystal display, an organic EL (Electro Luminescence) display, or a cathode ray tube display. Note that the input device 25 and the display device 26 may be configured integrally, for example, in the form of a touch panel.
[0027] The external I / F 27 is an interface for connecting (communicating) with external devices such as the sensor 11 and the mobile terminal 12.
[0028] The process (biometric information analysis process) of calculating mental state information of a subject by the information processing device 20 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an example of the operation of the information processing device 20 according to the embodiment. Fig. 3 will be described with reference to Figs. 4 to 19. Note that the biometric information analysis process shown in Fig. 3 can be executed at any timing, but is preferably executed every time a session is held for the subject.
[0029] 3, the acquisition unit 201 acquires biological information of a subject (step S101). For example, the acquisition unit 201 acquires the biological information of the subject by generating the biological information of the subject based on various measurement data collected by the sensor 11.
[0030] Here, the biological information is information including, for example, autonomic nerve data, total power data, body movement data, body temperature data, pulse data, etc. The autonomic nerve data is time series information of activity indexes of the autonomic nerves, and includes sympathetic nerve data, parasympathetic nerve data, and total power data. The sympathetic nerve data is time series information of activity indexes of the sympathetic nerves (Sympathetic Nervous System: SNS level). The parasympathetic nerve data is time series information of activity indexes of the parasympathetic nerves (ParaSympathetic Nervous System: PSNS level). The total power data is data in which the sympathetic nerve data and the parasympathetic nerve data are integrated, and corresponds to, for example, the sum of the sympathetic nerve data and the parasympathetic nerve data. The body movement data is time series information indicating the intensity of body movement. The body temperature data is time series information of the body temperature on the body surface (the part where the sensor 11 is attached). The pulse data is time series information of the pulse.
[0031] An example of a process for acquiring biological information of a subject will be described with reference to Fig. 4 and Fig. 5. Fig. 4 and Fig. 5 are diagrams for explaining the process of the acquiring unit 201. Fig. 4 shows an example of electrocardiogram waveform data (electrocardiogram) measured by the sensor 11. Fig. 5 shows an example of autonomic nerve data generated by the acquiring unit 201.
[0032] For example, the acquiring unit 201 acquires the autonomic nerve data by generating the autonomic nerve data from electrocardiogram waveform data showing a time-series electrocardiogram waveform. Specifically, the acquiring unit 201 detects the position of an R wave from the electrocardiogram waveform data shown in FIG. 4, and generates a heartbeat interval variation time series (RRI time series) by plotting the interval (RR interval: RRI) of the detected R wave over time. Then, the acquiring unit 201 calculates a power spectrum from the RRI time series, and integrates the power of a predetermined frequency range. As an example, the acquiring unit 201 calculates the integrated value of the power in the low frequency band (0.05 Hz to 0.15 Hz) as the SNS level (upper part of FIG. 5), and calculates the integrated value of the power in the high frequency band (0.15 Hz to 0.40 Hz) as the PSNS level (lower part of FIG. 5). It is generally known that the SNS level is dominant in a stressed state, and the PSNS level is dominant in a relaxed state. In this manner, the acquiring unit 201 generates the sympathetic nerve data and the parasympathetic nerve data.
[0033] The acquiring unit 201 also acquires total power data based on the sympathetic nerve data and the parasympathetic nerve data. For example, the acquiring unit 201 acquires the total power data by adding up the SNS level and the PSNS level corresponding to each time.
[0034] In this way, the acquisition unit 201 acquires biological information of the subject based on various measurement data collected by the sensor 11. Note that the above-mentioned methods of acquiring sympathetic nerve data, parasympathetic nerve data, and total power data are merely examples, and are not limited to the above description. For example, known techniques can be appropriately selected and applied as methods of acquiring sympathetic nerve data, parasympathetic nerve data, and total power data.
[0035] In addition to the above data, the acquisition unit 201 can acquire data based on various measurement data collected by the sensor 11. For example, the acquisition unit 201 can acquire time series data of heart rate and respiration rate from electrocardiogram waveform data. The acquisition unit 201 can acquire time series data of percutaneous arterial oxygen saturation (SpO2) and pulse from pulse wave data. The acquisition unit 201 can acquire body movement data from acceleration data. The acquisition unit 201 can acquire body temperature data from temperature data. As a method for acquiring these data, known techniques can be appropriately selected and applied.
[0036] Next, the acquiring unit 201 acquires the subject's behavior record information (step S102). For example, the acquiring unit 201 acquires the behavior record information from the subject's mobile terminal 12. Here, the behavior record information includes behavior data indicating the content of each behavior, time data indicating the time when each behavior was performed, location data indicating the location where each behavior was performed, medication history data indicating the medication history, and mood history data indicating the mood history at the time when each behavior was performed. In addition, the behavior data is represented by hierarchical classification information.
[0037] An example of a process for acquiring behavior record information of a subject will be described with reference to Fig. 6 and Fig. 7. Fig. 6 and Fig. 7 are diagrams for explaining the process of the acquiring unit 201. Fig. 6 shows an example of behavior record information recorded in the mobile terminal 12. Fig. 7 shows an example of classification of behavior data.
[0038] As shown in FIG. 6, the mobile terminal 12 collects behavior record information including behavior data, time data, place data, medication history data, and mood history data. For example, in the behavior record information shown in FIG. 6, the second record is associated with behavior data "breakfast meal", time data "1 hour 15 minutes 7:00", place data "home", medication history data "drug name: AAA medication time 7:40", and mood history data "expression mark (smile)". In other words, the second record indicates that the subject had breakfast at home for 1 hour 15 minutes from 7:00, and the subject felt good at that time. The mobile terminal 12 collects behavior record information including behavior data, time data, place data, medication history data, and mood history data for the other records in the same manner.
[0039] It is preferable that the behavioral data is represented by hierarchical classification information as shown in FIG. 7. For example, the behavioral data is divided into a major classification, a medium classification, and a minor classification. As an example, the major classification "work" includes the medium classifications "meeting" and "desk work". Furthermore, the medium classification "desk work" includes the minor classifications "report preparation" and "document organization". It is preferable that the classification information of the behavioral data is predefined, but the subject can add, delete, and edit the classification information.
[0040] Here, the behavior data, time data, and location data are automatically input by the mobile terminal 12 based on a schedule. For example, the mobile terminal 12 acquires schedule information of the subject from a schedule application installed in the mobile terminal 12. This schedule information includes information such as the subject's planned behavior, date and time, and location. Then, the mobile terminal 12 extracts the behavior data, time data, and location data from the schedule information, and records the extracted information as behavior record information. The behavior data, time data, and location data automatically recorded by the mobile terminal 12 can be manually corrected.
[0041] The location data may be automatically corrected based on coordinate information acquired by a GPS (Global Positioning System) function of the mobile terminal 12. In this case, each location data is associated in advance with the coordinate information of the location on the GPS function and registered in the recording application. If the location data corresponding to the coordinate information actually acquired by the GPS function differs from the location data based on the schedule information, the mobile terminal 12 discards the location data based on the schedule information. If the coordinate information indicates movement, "movement" is input as the location data and the behavior data.
[0042] The medication history data and mood history data are input manually. For example, the subject launches a recording application every time he or she takes a medication, and inputs the name of the medication taken and the time of taking the medication. After completing each action, the subject selects the mood history data corresponding to the mood at the time the action was performed. The medication history data and mood history data can also be manually corrected. The mood history data is not limited to facial expression marks, and may be represented by text information expressing the mood, such as "sleepy" or "relaxed," and numerical information expressing the degree of the mood.
[0043] In this way, the mobile terminal 12 collects the behavior record information of the subject. Then, the mobile terminal 12 transfers the collected behavior record information to the information processing device 20. Note that the timing of transferring the behavior record information can be set arbitrarily. For example, the mobile terminal 12 may transfer the behavior record information periodically, or may transfer the behavior record information in response to a request from the subject or the operator of the information processing device 20. In this way, the acquisition unit 201 acquires the behavior record information of the subject from the mobile terminal 12.
[0044] The subject's behavior record information can be corrected based on the body movement data acquired from the sensor 11. For example, the acquisition unit 201 acquires the start time and end time of the subject's sleep based on the intensity of the body movement in the body movement data. If the start time and end time of sleep based on the body movement data differ from the start time and end time of sleep in the behavior record information, the start time and end time of sleep in the behavior record information can be discarded and overwritten with the start time and end time of sleep based on the body movement data.
[0045] The acquiring unit 201 also acquires question answer information of the subject (step S103). For example, the acquiring unit 201 acquires, as the question answer information, mental state answer data related to the mental state of the subject and session evaluation answer data related to the evaluation of the session conducted for the subject.
[0046] An example of the mental state answer data and the session evaluation answer data will be described with reference to Fig. 8 and Fig. 9. Fig. 8 and Fig. 9 are diagrams for explaining the processing of the acquisition unit 201. Fig. 8 shows an example of the mental state answer data. Fig. 9 shows an example of the session evaluation answer data.
[0047] For example, each time a session is held, the operator of the information processing device 20 distributes to the subject a questionnaire corresponding to each item of the mental state answer data (FIG. 8) and the session evaluation answer data (FIG. 9). Then, when the operator receives the questionnaire on which the subject has written his / her answers, the operator inputs the answers (question answer information) written on the received questionnaire into the information processing device 20. The question answer information may be input by image recognition of data scanned by a camera or the like, or may be input manually by the operator. As a result, the acquisition unit 201 acquires question answer information including the mental state answer data and the session evaluation answer data.
[0048] Then, the calculation unit 202 calculates the mental state information based on the biological information, the behavior record information, and the question answer information (step S104). For example, the calculation unit 202 identifies partial data corresponding to a period during which each behavior was performed from the time-series autonomic nerve data, and calculates the statistical value of the identified partial data as the mental state information.
[0049] An example of the mental state information will be described with reference to Fig. 10. Fig. 10 is a diagram for explaining the processing of the calculation unit 202. Note that, in "XXX" in Fig. 10, numerical values indicating the mental state information calculated by the calculation unit 202 are input.
[0050] 10, the calculation unit 202 calculates mental state information for each behavior based on various information acquired by the acquisition unit 201. Here, the mental state information is, for example, representative values of the sympathetic nerve data, the parasympathetic nerve data, and the total power data for each behavior.
[0051] For example, the calculation unit 202 matches the measurement start time of the sensor 11 with the time data included in the behavior record information to associate the time series of the biological information with the time series of the behavior record information. Then, the calculation unit 202 specifies partial data corresponding to the period during which each behavior was performed from the time series of the autonomic nerve data. Specifically, the calculation unit 202 specifies sympathetic nerve data of the time series of the sympathetic nerve data corresponding to the period "24:00 to 7:00" during which the behavior data "sleep" was performed. Then, the calculation unit 202 calculates the average value of the sympathetic nerve data of the specified period as the mental state information of FIG. 10. Note that the calculation unit 202 also calculates the parasympathetic nerve data and the total power data in the same manner as the sympathetic nerve data.
[0052] In this way, the calculation unit 202 calculates the mental state information for each behavior based on the biological information and the behavior record information. In the above description, the "average value" is calculated as the representative value of the sympathetic nerve data for each behavior, but the present invention is not limited to this, and any statistical value such as a peak value, a median value, or a standard deviation can be calculated.
[0053] Furthermore, the calculation unit 202 calculates the mental state score of the subject based on the mental state answer data. For example, the calculation unit 202 calculates the mental state score related to the mental state of the subject by inputting the score for each item included in the mental state answer data into an arbitrary function.
[0054] Furthermore, the calculation unit 202 calculates a session evaluation score of the session based on the session evaluation answer data. For example, the calculation unit 202 calculates a session evaluation score related to the evaluation of the session by inputting the score for each item included in the session evaluation answer data into an arbitrary function.
[0055] The method of calculating the mental state score and the session evaluation score can be changed as appropriate depending on the question items included in each piece of answer-to-question information.
[0056] Then, the generating unit 203 generates a comment based on the mental state information (step S105). For example, the generating unit 203 generates at least one of a comment and an image that evaluates the mental state in each behavior based on the mental state information in each behavior.
[0057] An example of the process of generating a comment will be described with reference to Fig. 11, Fig. 12, and Fig. 13. Fig. 11, Fig. 12, and Fig. 13 are diagrams for explaining the process of the generation unit 203. Fig. 11 shows an example of a template for each comment. Fig. 12 shows an example of a comment generated by the generation unit 203. Fig. 13 shows an example of a comprehensive judgment generated by the generation unit 203.
[0058] As shown in Fig. 11, the auxiliary storage device 24 stores information in which a template of each comment is associated with a generation condition of each comment for each of a plurality of comments. For example, the auxiliary storage device 24 stores information in which a template "xx% of the time from xx:xx to xx:xx is good quality sleep" is associated with a generation condition of sympathetic nerve data "less than threshold A", a generation condition of parasympathetic nerve data "more than threshold B", and a generation condition of behavioral data "sleeping". This information indicates that a comment based on the template "xx% of the time from xx:xx to xx:xx is good quality sleep" is generated when there is a behavior in which the sympathetic nerve data is "less than threshold A", the parasympathetic nerve data is "more than threshold B", and the behavioral data is "sleeping".
[0059] For example, the generation unit 203 determines whether or not each behavior included in the behavior record information satisfies the generation condition of each comment. If a comment is determined to satisfy the generation condition, the generation unit 203 reads a template for that comment from the auxiliary storage device 24. Then, the generation unit 203 generates a comment for the behavior that satisfies the generation condition based on the read template.
[0060] For example, when there is an action in which the sympathetic nerve data is "less than threshold A", the parasympathetic nerve data is "above threshold B", and the action data is "sleeping", the generating unit 203 reads out a comment template based on the template "xx% of xx minutes to xx minutes was good quality sleep". The generating unit 203 then calculates the start time, end time, and percentage of good quality sleep for the "sleeping" that satisfies the generation condition, and inputs the calculated information into the template to generate a comment such as "29% of 23:05 to 6:00 was good quality sleep" (Fig. 12). The percentage of good quality sleep is calculated, for example, as the percentage of the period during which the PSNS level was equal to or higher than a predetermined value during the sleeping time.
[0061] The comment generated by the generating unit 203 is added to an action that satisfies the condition for generating the comment. For this reason, the comment "29% of the time from 11:05 p.m. to 6:00 a.m. was good quality sleep" is added to a sleep that ended at "6:00 a.m." on the date "2018 / 10 / 19."
[0062] Furthermore, the generating unit 203 generates an overall judgment of a common behavior based on mental state information for each of the multiple common behaviors performed multiple times within a certain period of time. For example, if the behavior data "sleeping" has been performed seven times between the previous session and the current session, the generating unit 203 performs an "overall judgment of sleep" based on mental state information for the seven sleeps. Specifically, the generating unit 203 inputs the sympathetic nerve data and parasympathetic nerve data for the seven sleeps into an arbitrary function, and performs an overall judgment based on a four-level score of "0" to "3" shown in FIG. 13. Then, the generating unit 203 generates an overall judgment comment corresponding to the score of the overall judgment. The overall judgment comment is stored in the auxiliary storage device 24 in advance.
[0063] Then, the output control unit 204 outputs the mental state information and the comment (step S106). For example, the output control unit 204 outputs information in which behavior data indicating each behavior, a period during which each behavior was performed, and mental state information for each behavior are associated with each other.
[0064] An example of displaying autonomic nerve data will be described with reference to Fig. 14, Fig. 15, and Fig. 16. Fig. 14, Fig. 15, and Fig. 16 are diagrams for explaining the processing of the output control unit 204. In Fig. 14, Fig. 15, and Fig. 16, the vertical axis corresponds to the SNS level, and the horizontal axis corresponds to the PSNS level. Also, in Fig. 14, Fig. 15, and Fig. 16, the size of the circle corresponds to the length of time (time) during which each action was performed.
[0065] As shown in Fig. 14, for example, the output control unit 204 displays the autonomic nerve data for each behavior. In the example shown in Fig. 14, the output control unit 204 plots the four behaviors of work, movement, eating, and sleeping. This allows the operator to easily grasp the difference in the SNS level and PSNS level for each behavior. For example, the operator can see that the SNS level for work is higher than that for movement, eating, and sleeping.
[0066] As shown in Fig. 15, the output control unit 204 simultaneously displays the latest autonomic nerve data and past autonomic nerve data for each action. In the example shown in Fig. 15, the output control unit 204 plots the autonomic nerve data for the current job, the autonomic nerve data for the previous job, and the autonomic nerve data for the job before last. This allows the operator to easily grasp the difference between the current mental state and the past mental state for each action. For example, the operator can see that the SNS level at work is higher than before.
[0067] As shown in FIG. 16, the output control unit 204 simultaneously displays standard autonomic nerve data for each behavior. In the example shown in FIG. 16, the output control unit 204 plots the autonomic nerve data for each behavior of the subject together with the autonomic nerve data of a standard person of the same generation as the subject. This allows the operator to easily grasp the difference between the mental state of the subject and that of a standard person. For example, the operator can see that the SNS level of the subject is high in all behaviors of work, movement, eating, and sleeping. In addition, the operator can see that the subject's sleeping time is shorter than that of a standard person.
[0068] Furthermore, the output control unit 204 displays the mental state score and the session evaluation score as the mental state information.
[0069] Display examples of the mental state score and the session evaluation score will be described with reference to Fig. 17 and Fig. 18. Fig. 17 and Fig. 18 are diagrams for explaining the processing of the output control unit 204. In Fig. 17 and Fig. 18, the vertical axis corresponds to the index of each data, and the horizontal axis corresponds to the number of sessions. The vertical axis is plotted with the index of each data at the first session set to 100.
[0070] As shown in Fig. 17, the output control unit 204 displays the mental state score and the total power data. This total power data is a representative value of the total power data for a period (e.g., one week) separated by each session. This allows the operator to simultaneously view the objective mental state information detected by the sensor 11 along with the mental state score. For example, the mental state score may contain inaccuracies or lies depending on the mental state of the subject at the time of answering, but the operator can simultaneously view mental state information that objectively indicates the mental state that causes inaccuracies or lies, which can be useful for giving advice for the next session.
[0071] 18, the output control unit 204 displays the session evaluation score and the total power data. This total power data is a representative value of the total power data for a period (e.g., one week) divided by each session. This allows the operator to simultaneously view the objective mental state information detected by the sensor 11 along with the session evaluation score. For example, the mental state score may contain inaccuracies or lies depending on the mental state of the subject at the time of answering, but the operator can simultaneously view mental state information that objectively indicates the mental state that causes inaccuracies or lies, which can be useful when deciding the content of the next session.
[0072] Furthermore, the output control unit 204 displays comments regarding each behavior in association with the behavior record information.
[0073] A display example of a comment will be described with reference to Fig. 19. Fig. 19 is a diagram for explaining the processing of the output control unit 204. Fig. 19 illustrates information in which biological information including arrhythmia, heart rate, amount of physical activity, sympathetic nerve, and parasympathetic nerve is associated with behavior record information including behavior history and mood history. In Fig. 19, the horizontal axis corresponds to time.
[0074] 19, the output control unit 204 compares the measurement start time of the sensor 11 with the time data included in the action record information to display the time series of the biometric information and the time series of the action record information in association with each other. Then, the output control unit 204 displays the comments on each action in association with the action record information.
[0075] Here, the comment generated by the generating unit 203 is attached to the behavior that satisfies the generation condition of the comment. Therefore, the output control unit 204 displays the comment "29% of the time from 11:05 p.m. to 6:00 a.m. was good quality sleep" in association with "sleep" that ended at time "6:00" on date "2018 / 10 / 19". In addition, the output control unit 204 displays the comment "Feeling a little excited while eating" in association with "dinner" that ended at time "20:00" on date "2018 / 10 / 18".
[0076] In this way, the output control unit 204 outputs the mental state information and the comment. Note that, although the case where the output control unit 204 displays the mental state information and the comment on the display device 26 has been described here, the present invention is not limited to this. For example, the output control unit 204 may transmit the mental state information and the comment to an external device, or may store them in a memory or a recording medium.
[0077] Moreover, Figs. 14 to 19 are merely examples, and it is not required that all display forms be displayed at the same time. For example, an operator can appropriately select and display the display forms exemplified in Figs. 14 to 19 as necessary. Furthermore, the display items shown in Figs. 14 to 19 can be changed at the operator's discretion. For example, the total power data shown in Figs. 17 and 18 may be replaced with other mental state information, or multiple pieces of mental state information may be displayed. Furthermore, any statistical value such as the average value, peak value, median value, or standard deviation can be used as a representative value of the total power data.
[0078] As described above, in the information processing device 20 according to the embodiment, the acquisition unit 201 acquires bioinformation including autonomic nerve data of the subject and behavior record information in which a plurality of behaviors of the subject are recorded. In addition, the calculation unit 202 calculates mental state information for each behavior of the subject based on the bioinformation and the behavior record information. This allows the operator to accurately grasp the mental state of the subject. For example, according to the information processing device 20 according to the embodiment, the mental state can be objectively quantified based on the electrocardiogram waveform data collected by the sensor 11, so that the mental state of the subject can be accurately grasped.
[0079] In addition, for example, the information processing device 20 quantifies the mental state of each behavior, making it easier to identify behavior that causes stress. This allows the operator to provide more appropriate advice to the subject.
[0080] (Modification) It is known that the magnitude of autonomic nerve data varies from person to person, so it is preferable to adjust the comment generation condition (threshold) according to the subject's total power data.
[0081] For example, the generating unit 203 adjusts the generation condition of each comment stored in the auxiliary storage device 24 according to the total power data of the subject. For example, when the total power data of the subject is larger than the standard data, the generating unit 203 increases the threshold included in the generation condition. Also, when the total power data of the subject is smaller than the standard data, the generating unit 203 decreases the threshold included in the generation condition. Then, the generating unit 203 uses the adjusted generation condition to determine whether or not the generation condition of each comment is satisfied. This allows the information processing device 20 to appropriately assign a comment according to the magnitude of the autonomic nerve data of the individual.
[0082] Second Embodiment In the first embodiment, a case has been described in which mental state information is calculated based on biometric information, behavioral record information, and question answer information, and a comment is generated based on the calculated mental state information, but the embodiment is not limited to this. For example, it is also possible to generate a mental state judgment result by inputting biometric information and behavioral record information into a trained model created by machine learning. Note that the mental state judgment result is data equivalent to the mental state answer data and the session evaluation answer data generated (judged) by the trained model.
[0083] That is, the information processing device 50 according to the second embodiment constructs a trained model for multiple subjects (second subjects) having a specific mental tendency, using biometric information including autonomic nerve data of each subject, behavioral record information in which multiple behaviors of each subject are recorded, and question answer information answered by each subject. Then, the information processing device 50 inputs the biometric information and behavioral record information of an arbitrary subject (first subject) into the constructed trained model, thereby generating a mental state determination result in which the mental state of the arbitrary subject is determined. Note that the first subject may be included in the second subject.
[0084] In the second embodiment, the "second subject" is, for example, a subject including a person having a specific mental tendency. The "specific mental tendency" includes mental disorders (mental diseases) such as schizophrenia, mood disorders (depression, manic depression), panic disorder, eating disorder, developmental disorder, dementia, epilepsy, addiction, and higher brain dysfunction. Furthermore, among such persons having a specific mental tendency, if the second subject is a person having a mood disorder (depression, manic depression) and an eating disorder, the accuracy of the trained model (the accuracy of generating the mental state judgment result) is likely to be improved, and particularly, if the second subject is a person having depression, the accuracy of the trained model is more likely to be improved. However, the second subject is not limited to a person having a mental disorder, and may be a person classified according to a mental tendency such as a preference. Furthermore, the second subject may be a person who is receiving psychological counseling.
[0085] Fig. 20 is a diagram showing an example of a schematic configuration of an information processing device according to a second embodiment. As shown in Fig. 20, an information processing device 50 according to the embodiment is capable of directly or indirectly communicating with a sensor 11 and a mobile terminal 12 via a network such as a LAN or a WAN. The information processing device 50 is an example of a biometric information analysis device. The sensor 11 and the mobile terminal 12 are similar to the sensor 11 and the mobile terminal 12 shown in Fig. 1, respectively, and therefore description thereof will be omitted.
[0086] The information processing device 50 includes an acquisition unit 501, a determination unit 502, and an output control unit 503. The acquisition unit 501 and the output control unit 503 are similar to the acquisition unit 201 and the output control unit 204 shown in Fig. 1, respectively, and therefore their description will be omitted. The hardware configuration of the information processing device 50 is similar to the hardware configuration of the information processing device 20 shown in Fig. 2, and therefore their description will be omitted. The determination unit 502 will be described later.
[0087] The process during learning and operation performed by the information processing device 50 according to the second embodiment will be described with reference to Fig. 21. Fig. 21 is a diagram showing the process during learning and operation performed by the information processing device 50 according to the second embodiment. Note that the subject X shown in Fig. 21 is an example of the first subject. Furthermore, the subjects S-1 to SN are examples of the second subjects.
[0088] As shown in the upper part of FIG. 21, during learning, the information processing device 50 performs machine learning using, for example, the biometric information, behavior record information, and question answer information (mental state answer data and session evaluation answer data) of multiple subjects S-1 to SN (for example, multiple subjects prone to depression and multiple healthy subjects) as training data. Of the training data, the biometric information and behavior record information are input data, and the question answer information is correct answer data. Through this machine learning, a trained model is constructed that inputs the biometric information and behavior record information of the subject to be diagnosed and outputs a mental state assessment result of the subject to be diagnosed. This trained model is stored in a storage device (for example, ROM 22, RAM 23, auxiliary storage device 24, etc.).
[0089] Then, as shown in the lower part of FIG. 21, during operation, the information processing device 50 inputs the biological information and behavior record information of the subject X, who is the diagnosis target, to the trained model constructed during learning, thereby causing the trained model to output a mental state assessment result for the subject X. Here, the subject X is, for example, a person suspected of having a depressive tendency. Then, the information processing device 50 presents the mental state assessment result for the subject X output from the trained model to the operator (or the subject X).
[0090] The biological information is the same as the biological information described in the first embodiment. For example, the biological information may include all of the autonomic nerve data, body temperature data, pulse data, and body movement data, or may include only optional data. The target period of the biological information can be set arbitrarily, such as several hours, several days, or several weeks.
[0091] The behavior record information is the same as the behavior record information described in the first embodiment. For example, the behavior record information may include all of the behavior data, time data, and mood history data, or may include only optional data. The target period of the behavior record information can be set arbitrarily, but is preferably the same as the target period of the biological information.
[0092] The question answer information is the same as the question answer information described in the first embodiment. For example, the question answer information may include both mental state answer data and session evaluation answer data, or may include only one of the data. The question answer information may be acquired once during any target period (preferably the same as the target period of the biological information) as a trend for the entire period, or may be acquired any number of times at any timing, but it is preferable to acquire the information once as a trend for the entire period.
[0093] Further, the process of constructing the trained model is executed by, for example, the determination unit 502. That is, the determination unit 502 acquires, for a plurality of second subjects having a specific mental tendency, biometric information including autonomic nerve data of the second subjects, behavior record information in which a plurality of behaviors of the second subjects are recorded, and question answer information answered by the second subjects. Here, the biometric information, behavior record information, and question answer information of the second subjects are collected in advance and stored in a storage device (for example, the ROM 22, the RAM 23, the auxiliary storage device 24, etc.). For example, the determination unit 502 acquires the biometric information, behavior record information, and question answer information of the second subjects by reading the information from the storage device. Then, the determination unit 502 constructs the trained model by performing machine learning using the acquired biometric information, behavior record information, and question answer information. Note that the process of constructing the trained model is not limited to the determination unit 502 and can be executed by any processing unit (processor).
[0094] A process (mental state determination process) in which the information processing device 50 outputs a mental state determination result of the subject X during operation will be described with reference to Fig. 22. Fig. 22 is a flowchart showing an example of the operation of the information processing device 50 according to the second embodiment. The mental state determination process shown in Fig. 22 is a process for determining whether or not the subject X suspected of having a depressive tendency, etc., has a depressive tendency, etc. Furthermore, the mental state determination process can be executed at any timing.
[0095] 22, the acquisition unit 501 acquires biometric information of the subject X (step S201). For example, the acquisition unit 501 acquires the biometric information of the subject X by generating the biometric information of the subject X based on various measurement data collected by the sensor 11.
[0096] Next, the acquisition unit 501 acquires the behavior record information of the subject X (step S202). For example, the acquisition unit 501 acquires the behavior record information from the mobile terminal 12 of the subject X.
[0097] Then, the determination unit 502 generates a mental state determination result of the subject X by inputting the biometric information and behavior record information of the subject X to the trained model. For example, the determination unit 502 reads out the trained model stored in a storage device. Then, the determination unit 502 inputs the biometric information and behavior record information acquired by the acquisition unit 501 to the read trained model, thereby causing the trained model to output the mental state determination result of the subject X. Thereafter, the determination unit 502 sends the mental state determination result output from the trained model to the output control unit 503.
[0098] The mental state assessment result output from the trained model includes the same type of information as the question answer information used to construct the trained model. That is, when the question answer information used in machine learning includes both mental state answer data and session evaluation answer data, the mental state assessment result output from the trained model includes both mental state answer data and session evaluation answer data. When the question answer information used in machine learning includes mental state answer data, the mental state assessment result output from the trained model includes mental state answer data. When the question answer information used in machine learning includes session evaluation answer data, the mental state assessment result output from the trained model includes session evaluation answer data.
[0099] The output control unit 503 outputs the mental state assessment result of the subject X. For example, the output control unit 503 displays the mental state assessment result on the display device 26. Note that the output destination of the mental state assessment result is not limited to the display device 26, and may be, for example, transmitted to an external device or stored in a memory or a recording medium.
[0100] As described above, in the information processing device 50 according to the second embodiment, the acquisition unit 501 acquires biometric information including autonomic nerve data of the subject and behavior record information in which a plurality of behaviors of the subject are recorded. In addition, the determination unit 502 generates a mental state determination result of the subject by inputting the biometric information and behavior record information of the subject to the trained model. This allows the operator to accurately grasp the mental state of the subject.
[0101] For example, according to the information processing device 50 of the second embodiment, the mental state assessment result corresponding to the mental state answer data and / or the session evaluation answer data can be automatically generated (determined) by the trained model, even if the subject does not enter it himself / herself. Therefore, the operator can easily understand whether or not the subject has a specific mental tendency by viewing the mental state assessment result output from the information processing device 50.
[0102] In the second embodiment, the case where the mental state judgment process is applied to a person suspected of having a tendency to depression or the like has been described, but the present invention is not limited to this. For example, the mental state judgment process can also be applied to a person who is not suspected of having a tendency to depression or the like. In other words, the subject X to be judged (also called the first subject or judgment subject) does not need to be particularly limited and may be any subject. For example, the first subject may be any one of a plurality of second subjects.
[0103] In the second embodiment, the case where the trained model is constructed by the information processing device 50 has been described, but the embodiment is not limited to this. For example, the trained model may be constructed by an external device different from the information processing device 50. In this case, the information processing device 50 acquires the trained model constructed by the external device and executes the above-mentioned mental state assessment process.
[0104] (Modification 1 of the second embodiment) In addition, the information processing device 50 described in the second embodiment is capable of not only presenting the mental state assessment result directly to the operator, but also calculating and presenting the “match rate” with the question answer information of persons prone to depression, etc.
[0105] For example, the determination unit 502 compares the mental state determination result of the subject X with representative question answer information of a person with a tendency towards depression, etc. Note that, as the representative question answer information of a person with a tendency towards depression, etc., any question answer information of a person with a tendency towards depression, etc. may be used as it is, or the question answer information of a plurality of people with a tendency towards depression, etc. may be integrated (averaged) and used.
[0106] Specifically, the determination unit 502 compares each item included in the mental state determination result with each item included in the question answer information, and calculates the number of matching items. The determination unit 502 then calculates the "matching rate" by dividing the number of matching items by the total number of items. Note that the calculation method for the "matching rate" described here is merely an example, and any calculation method can be applied.
[0107] Then, the output control unit 503 outputs the "match rate" calculated by the determination unit 502. This allows the information processing device 50 to support a diagnosis by an operator (a medical professional such as a doctor). For example, the operator can accurately and easily diagnose whether or not a subject has a specific mental tendency by viewing the match rate.
[0108] In addition, when the matching rate is output, the mental state assessment result does not necessarily have to be output.
[0109] (Modification 2 of the second embodiment) In addition, the information processing device 50 described in variant example 1 of the second embodiment is capable of not only presenting the matching rate directly to the operator, but also suggesting the possibility of whether or not the person is prone to depression, etc. (suggesting the possibility of a mental tendency).
[0110] For example, the suggestion of the possibility of a mental tendency may be information indicating the degree of possibility of a depressive tendency in stages. For example, the mental tendency judgment result may be information that classifies the possibility of a depressive tendency into three stages, "high," "medium," and "low," according to the degree of matching. When classifying into three stages, two thresholds are used.
[0111] Furthermore, when information indicating a possible mental tendency is output, the mental state assessment result and the matching rate do not necessarily need to be output.
[0112] In addition, the information indicating possible mental tendencies that is presented to the operator is intended to assist diagnosis and is not a diagnosis result itself. In other words, the processing of the information processing device 50 does not constitute a medical procedure.
[0113] (Modification 3 of the second embodiment) The trained model (mental state assessment process) described in the second embodiment may be applied to the information processing device 20 shown in the first embodiment. This allows the information processing device 20 to output mental state information and comments even if the subject himself / herself does not enter question answer information.
[0114] Specifically, the information processing device 20 executes the process of step S203 in Fig. 22 instead of the process of step S103 in Fig. 3. That is, the information processing device 20 generates a mental state assessment result of the subject by inputting the biological information and the behavior record information acquired in steps S101 and S102 in Fig. 3 to the trained model.
[0115] Then, the information processing device 20 uses the mental state assessment result instead of the question answer information in the process of step S104 in Fig. 3. That is, the information processing device 20 calculates the mental state information based on the biological information, the behavior record information, and the mental state assessment result. Note that the mental state assessment result is data equivalent to the mental state answer data and the session evaluation answer data, and therefore can be used instead of the question answer information.
[0116] Then, the information processing device 20 can output the mental state information and the comment by executing the same processes as those in steps S105 and S106 in FIG.
[0117] (Other embodiments) In addition to the above-described embodiment, the present invention may be embodied in various different forms.
[0118] (Use of sensors that can be attached to the body surface) In the above-described embodiment, it has been described that the sensor 11 can be applied by appropriately selecting known technology. However, in order to collect biological information for a long period of time (several days or several weeks), it is preferable to apply a sensor that can be attached to the body surface. Such a sensor that can be attached to the body surface is preferably a wearable sensor having a circuit that will not break when bent or the like inside a flexible device so that it can flexibly follow the movement of the body. In addition, such a sensor that can be attached to the body surface is preferably a sensor that can be attached to the body surface via an adhesive such as an acrylic resin that does not cause discomfort to the body and has high medical quality characteristics and is compatible with biological safety tests.
[0119] For example, the acquisition unit 201 acquires biometric information based on data collected by the sensor 11 that can be attached to the subject's body surface. As a result, the acquisition unit 201 can collect both biometric information during sleep and biometric information during wakefulness without omission, so that continuous biometric information can be collected over a long period of time. Note that the biometric information used in this embodiment does not necessarily have to be continuous data, but continuous data is preferable. Such long-term continuous data is preferably continuous data for 24 hours or more, more preferably continuous data for 72 hours or more, particularly preferably continuous data for 144 hours or more, and most preferably continuous data for 168 hours or more. In addition, such continuous data preferably includes continuous data on weekdays and continuous data on holidays. With such continuous data, a more accurate trained model can be constructed in the second embodiment.
[0120] (Biometric analysis system) The functions of the information processing device 20 according to the embodiment described above may be provided as a system.
[0121] Fig. 23 is a diagram showing an example of a schematic configuration of a system according to an embodiment. As shown in Fig. 23, a system 1 according to an embodiment includes a server device 30 and a display terminal 40. The server device 30 and the display terminal 40 are capable of communicating with each other directly or indirectly via a network such as a LAN (Local Area Network) or a WAN (Wide Area Network). The system 1 is an example of a bioinformation analysis system.
[0122] Furthermore, the sensor 11 and the mobile terminal 12 are capable of communicating with the server device 30 directly or indirectly via a network, but do not need to be constantly connected to the server device 30. The sensor 11 and the mobile terminal 12 basically have the same configuration as the sensor 11 and the mobile terminal 12 shown in Fig. 1, and therefore a description thereof will be omitted.
[0123] The server device 30 is provided in, for example, a service center that provides bioinformation analysis processing as a cloud service. The server device 30 includes an acquisition unit 301, a calculation unit 302, a generation unit 303, and an output control unit 304. The processing contents of the acquisition unit 301, the calculation unit 302, the generation unit 303, and the output control unit 304 are basically similar to the processing contents of the acquisition unit 201, the calculation unit 202, the generation unit 203, and the output control unit 204 shown in FIG. 1, and therefore description thereof will be omitted.
[0124] The display terminal 40 is, for example, an information processing terminal used by a user of the cloud service, and is, for example, a terminal equipped with a display device, such as a personal computer, a workstation, a smartphone, a tablet, etc. Here, the user of the cloud service corresponds to a person who provides psychological counseling (for example, a counselor, a doctor, etc.).
[0125] Here, the output control unit 304 of the server device 30 transmits the mental state information for each behavior of the subject to the display terminal. The display control unit 401 of the display terminal 40 receives the mental state information for each behavior of the subject transmitted from the server device 30, and displays the received mental state information for each behavior of the subject. This allows the user to accurately grasp the mental state of the subject.
[0126] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or a part of it can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc. Furthermore, each processing function performed by each device can be realized in whole or in any part by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.
[0127] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified.
[0128] The bioinformation analysis method described in the above embodiment can be realized by executing a prepared bioinformation analysis program on a computer such as a personal computer or a workstation. This bioinformation analysis program can be distributed via a network such as the Internet. This bioinformation analysis program can also be recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, an MO, or a DVD, and executed by being read from the recording medium by a computer.
[0129] According to at least one of the embodiments described above, the mental state of a subject can be accurately grasped.
[0130] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents described in the claims, as well as in the scope and spirit of the invention.
[0131] [Example] The present invention will be described in more detail below based on examples, but the present invention is not limited to these examples.
[0132] (Examples 1 to 5, Comparative Example 1) A trained model was created that outputs mental state answer data and session evaluation answer data in response to input of biometric information and behavioral record information by the method described in the second embodiment. By varying the type of data used for machine learning, trained models were created for each of Examples 1 to 5 and Comparative Example 1 shown in Fig. 24. Fig. 24 is a diagram for explaining trained models relating to each Example and Comparative Example.
[0133] First, the input information (training data) used to create the trained model will be described. In this embodiment, the input information used was "biometric information," "behavioral record information," "mental state answer data," and "session evaluation answer data." In addition, 15 people prone to depression and 15 healthy people were used as subjects, and input information for each was collected for one week (168 hours including weekdays and holidays).
[0134] The "biometric information" used included autonomic nerve data, body temperature data, pulse data, and physical movement data. The autonomic nerve data included sympathetic nerve data, parasympathetic nerve data, and total power data, each of which was generated from electrocardiogram waveform data. The body temperature data (temperature data) is time-series information on the body temperature on the subject's body surface. The pulse data is time-series information on the pulse. The physical movement data indicates the amount of physical activity, and was generated from acceleration data. The electrocardiogram waveform data, body temperature data, pulse wave data, and acceleration data were collected over time using a sensor that can be attached to the subject's body surface (for example, a wearable device in the manner described in JP2017-510390).
[0135] As the "behavioral record information", the behavioral data, time data, and mood history data shown in Fig. 6 were used. The behavioral record information was collected by having each subject record their behavior over the target period (the above-mentioned 7 days).
[0136] The "mental state answer data" and "session evaluation answer data" used were the same data as those shown in Figures 8 and 9, respectively. The mental state answer data and session evaluation answer data were collected by having each subject record the data once at the end of the target period as a trend during the period for each Example and Comparative Example.
[0137] Next, the creation of the trained models according to each embodiment and comparative example will be described. The trained models according to each embodiment and comparative example were created using various data corresponding to the "circles" in Fig. 24 as training data. The trained models according to each embodiment and comparative example were created by machine learning using biometric information and behavioral record information as "input data" and mental state answer data and session evaluation answer data as "correct answer data", and random forest was used as the machine learning method in this embodiment.
[0138] The trained model according to the first embodiment was created using training data (input data and correct answer data) for 24 hours (one day) corresponding to weekdays (Monday to Friday). In the first embodiment, autonomic nerve data was used as biological information, and machine learning was performed without using body temperature data, pulse data, or body movement data.
[0139] The trained model according to the second embodiment was created using training data for 24 hours (one day) corresponding to holidays (Saturday and Sunday). In the second embodiment, autonomic nerve data was used as biological information, and machine learning was performed without using body temperature data, pulse data, or body movement data.
[0140] The trained model according to Example 3 was created using training data for 24 hours (one day) corresponding to a weekday.
[0141] The trained model according to the fourth embodiment was created using training data for 72 hours (three days) corresponding to weekdays. The three days of training data are, for example, a combination of training data for Monday, Wednesday, and Friday.
[0142] The trained model according to Example 5 was created using training data for 168 hours (one week) corresponding to weekdays and holidays.
[0143] The trained model in the comparative example was created using training data for 24 hours (one day) corresponding to a weekday. In the comparative example, machine learning was performed using pulse data as biological information, without using autonomic nerve data, body temperature data, or physical movement data.
[0144] Next, a method for evaluating the trained models according to each of the examples and comparative examples will be described. For one subject randomly selected from the subjects from whom the above training data was collected, input information for one week (168 hours including weekdays and holidays) was collected and used as input data for each trained model. The collected information was cut out for a period corresponding to the "acquisition period" of the trained model according to each of the examples and comparative examples and used as input data.
[0145] In addition, the mental state answer data and session evaluation answer data for evaluation were collected by having the subjects record them once at the end of the target period. The reason why the mental state answer data and session evaluation answer data were collected once at the end of each target period was that it was thought that trends throughout each target period could be obtained. In addition, when the training data for Monday, Wednesday, and Friday were integrated, the mental state answer data and session evaluation answer data collected at the end of the data collection for Friday were used.
[0146] The collected mental state answer data and session evaluation answer data for evaluation were then compared with the output data (mental state answer data and session evaluation answer data) output from each trained model to determine the match rate. The match rate was determined according to the proportion of items that matched among the multiple items included in the mental state answer data and the session evaluation answer data. Specifically, a match of 70% or more was determined as a "match." The match rate was also evaluated according to the number of matches (number of matches) determined out of 10 times. Specifically, "AAA" indicates that the proportion of items that matched was 90% or more. "AA" indicates that the proportion of items that matched was 85% or more and less than 90%. "A" indicates that the proportion of items that matched was 80% or more and less than 85%. "B" indicates that the proportion of items that matched was 75% or more and less than 80%. "C" indicates that the proportion of items that matched was less than 75%.
[0147] As shown in FIG. 24, the matching degree of the trained model according to Example 1 was "A". The matching degree of the trained model according to Example 2 was "B". The matching degree of the trained model according to Example 3 was "AA". The matching degree of the trained model according to Example 4 was "AA". The matching degree of the trained model according to Example 5 was "AAA". The matching degree of the trained model according to the comparative example was "C".
[0148] The degree of agreement of the trained model according to Example 5 was higher than the degrees of agreement of other trained models. This result suggests that it is preferable that the input data during training and operation is for 7 days or more (168 hours). In other words, it is preferable that the acquisition unit 201 acquires biometric information and behavioral record information for 7 days or more, and the calculation unit 202 calculates mental state information based on the biometric information and behavioral record information for 7 days or more.
[0149] This result also suggests that it is preferable that input data during learning and operation includes weekdays and holidays. In other words, it is preferable that the acquisition unit 201 acquires biometric information and behavior record information for several days or more including weekdays and holidays, and the calculation unit 202 calculates mental state information based on the biometric information and behavior record information for several days or more including weekdays and holidays.
[0150] Moreover, the degree of agreement of the trained models according to Examples 3 and 4 was higher than that of the trained models according to Examples 1 and 2. This result suggests that it is preferable for input data during training and operation to include body temperature data, pulse data, and body movement data as biological information.
[0151] Moreover, the degree of agreement of the trained model according to Example 1 was higher than the degree of agreement of the trained model according to Example 2. This result suggests that weekday data is more suitable than holiday data for input data during training and operation.
[0152] The comparative example is an example in which only pulse data that can be collected by a relatively large number of wearable devices is used. The degree of agreement of the trained model in the comparative example was lower than the degrees of agreement of the trained models in Examples 1 to 5. This result suggests that it is preferable for the input data during learning and operation to include autonomic nerve data as biological information.
Claims
1. an acquisition unit that acquires biometric information including autonomic nerve data including sympathetic nerve data, parasympathetic nerve data, and total power data obtained by integrating the sympathetic nerve data and the parasympathetic nerve data of the first subject based on data collected by a wearable sensor that can be attached to a body surface of the first subject, and physical movement data indicating the intensity of the first subject's physical movements, and also acquires behavior record information including behavior data indicating each behavior of the first subject and time data indicating the time when each behavior was performed, mental state answer data regarding the first subject's mental state, and a start time and end time of sleep based on the physical movement data, and corrects the behavior record information based on the start time and end time of sleep; a calculation unit that identifies partial data corresponding to a period during which each behavior was performed among the time-series autonomic nerve data based on the biological information and the corrected behavior record information, calculates a statistical value of the identified partial data as mental state information for each behavior of the first subject, and calculates a mental state score of the first subject based on the mental state answer data; a generation unit that generates at least one of a comment and an image that evaluates the mental state in each behavior based on the mental state information in each behavior; a determination unit that generates a mental state determination result in which the mental state of a plurality of second subjects having a specific mental tendency is determined by inputting the biometric information and the behavior record information of the first subject into a trained model trained using the biometric information of the second subjects, the behavior record information of the second subjects, and question answer information answered by the second subjects; and an output control unit that outputs, for the first subject, behavioral data indicating each behavior, a period during which each behavior was performed, the mental state information for each behavior, and information associating at least one of a comment and an image evaluating the mental state for each behavior, the mental state score, and the mental state assessment result; A biological information analysis device comprising:
2. The trained model is a trained model trained using 72 hours or more of the biometric information of the second subject and 72 hours or more of the behavioral record information of the second subject, The biological information analysis device according to claim 1 .
3. The biometric information of the second subject and the behavior record information of the second subject include data on weekdays and holidays. The biological information analysis device according to claim 2 .
4. The biometric information of the second subject and the behavior record information of the second subject include continuous data for 24 hours or more. The biological information analysis device according to claim 2 .
5. The biometric information of the second subject and the behavioral record information of the second subject include continuous data for 144 hours or more. The biological information analysis device according to claim 2 .
6. The biometric information of the second subject and the behavioral record information of the second subject include continuous data for 168 hours or more. The biological information analysis device according to claim 2 .
7. The generation unit is adjusting a generation condition of at least one of the comment and the image according to the total power data of the first subject; generating at least one of the comment and the image using the adjusted generation conditions; The biological information analyzing device according to any one of claims 1 to 6.
8. The acquisition unit acquires session evaluation response data regarding an evaluation of a session conducted for the first subject, the calculation unit calculates a session evaluation score of the session based on the session evaluation answer data. The biological information analyzing device according to any one of claims 1 to 7.
9. The acquisition unit acquires the biological information and the behavior record information for the first subject for 144 hours or more, The calculation unit calculates the mental state information based on the biological information and the behavior record information of the first subject for 144 hours or more. The biological information analyzing device according to any one of claims 1 to 8.
10. A biometric information analysis system including a server device and a display terminal, The server device includes: an acquisition unit that acquires biometric information including autonomic nerve data including sympathetic nerve data, parasympathetic nerve data, and total power data obtained by integrating the sympathetic nerve data and the parasympathetic nerve data of the first subject based on data collected by a wearable sensor that can be attached to a body surface of the first subject, and physical movement data indicating the intensity of the first subject's physical movements, and also acquires behavior record information including behavior data indicating each behavior of the first subject and time data indicating the time when each behavior was performed, mental state answer data regarding the first subject's mental state, and a start time and end time of sleep based on the physical movement data, and corrects the behavior record information based on the start time and end time of sleep; a calculation unit that identifies partial data corresponding to a period during which each behavior was performed among the time-series autonomic nerve data based on the biological information and the corrected behavior record information, calculates a statistical value of the identified partial data as mental state information for each behavior of the first subject, and calculates a mental state score of the first subject based on the mental state answer data; a generation unit that generates at least one of a comment and an image that evaluates the mental state in each behavior based on the mental state information in each behavior; a determination unit that generates a mental state determination result in which the mental state of a plurality of second subjects having a specific mental tendency is determined by inputting the biometric information and the behavior record information of the first subject into a trained model trained using the biometric information of the second subjects, the behavior record information of the second subjects, and question answer information answered by the second subjects; and an output control unit that transmits, to the display terminal, behavioral data indicating each behavior of the first subject, a period during which each behavior was performed, the mental state information for each behavior, and information associating at least one of a comment and an image evaluating the mental state for each behavior, the mental state score, and the mental state assessment result, Equipped with The display terminal includes: a display control unit that receives and displays behavior data indicating each behavior of the first subject transmitted from the server device, a period during which each behavior was performed, the mental state information for each behavior, and information associating at least one of a comment and an image evaluating the mental state for each behavior, the mental state score, and the mental state assessment result. A bioinformation analysis system comprising:
11. Based on data collected by a wearable sensor that can be attached to the body surface of a first subject, obtain biometric information including autonomic nerve data including sympathetic nerve data, parasympathetic nerve data, and total power data obtained by integrating the sympathetic nerve data and the parasympathetic nerve data of the first subject, and physical movement data indicating the intensity of the first subject's physical movements, and obtain behavioral record information including behavioral data indicating each behavior of the first subject and time data indicating the time when each behavior was performed, mental state answer data regarding the mental state of the first subject, and sleep start time and end time based on the physical movement data, and correct the behavioral record information based on the sleep start time and end time; Based on the biological information and the corrected behavior record information, partial data corresponding to a period during which each behavior was performed is identified from the time-series autonomic nerve data, and a statistical value of the identified partial data is calculated as mental state information for each behavior of the first subject, and a mental state score of the first subject is calculated based on the mental state answer data; generating at least one of a comment and an image evaluating the mental state in each behavior based on the mental state information in each behavior; generating a mental state determination result in which the mental state of a plurality of second subjects having a specific mental tendency is determined by inputting the biometric information and the behavioral record information of the first subject into a trained model trained using the biometric information of the second subjects, the behavioral record information of the second subjects, and question answer information answered by the second subjects; Outputting, for the first subject, information associating behavioral data indicating each behavior, a period during which each behavior was performed, the mental state information for each behavior, and at least one of a comment and an image evaluating the mental state for each behavior, the mental state score, and the mental state assessment result. A biometric analysis program that causes a computer to carry out each process.
Citation Information
Patent Citations
Emotion estimating server, emotion estimating method, presenting device and emotion estimating system
JP2018139087A
Biological information recording system, biological information analyzer, biological information display method, and biological information display program
JP2018153244A
Autonomic state evaluation device, autonomic state evaluation system, autonomic state evaluation method and program
JP2019030389A
Presentation device, presentation method, emotion estimation server, emotion estimation method, and emotion estimation system
JP2019030557A
Emotional information management server device, emotional information management method, program, terminal device, and information communication system
JP2019175108A
