Method of operating a biomedical information analysis device, biomedical information analysis device, biomedical information analysis system, and biomedical information analysis program

Abstract

To correctly determine the mental condition of a subject.SOLUTION: A biological information analysis method acquires, for a plurality of subjects who have a specific mental tendency over a prescribed period, biological information including autonomic nerve data which includes sympathetic nerve data, parasympathetic nerve data, and total power data which is consolidation of the sympathetic nerve data and parasympathetic nerve data, behavior recording information in which multiple behaviors of the subjects are recorded, and inquiry answer information answered by the subjects, and performs machine learning using the biological information, the behavior recording information and the inquiry answer information, thus constructing a learned model.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] Embodiments of the present invention relate to a biological information analysis method, a biological information analysis apparatus, a biological information analysis system, and a biological information analysis program.

Background Art

[0002] In the field of psychological counseling, it is required to grasp the mental state of the subject. For this reason, for example, every time a session (interview) with the subject is conducted by a counselor, the subject is asked to answer a questionnaire regarding the mental state, and the mental state of the subject is inferred by analyzing the answer results.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Here, in the above-mentioned fields such as the site, in recent years, due to reasons such as an increase in the number of counseling sessions and diversification of the attributes of counseling subjects, it is required to grasp a more accurate mental state.

[0005] The present invention provides a biological information analysis method, a biological information analysis apparatus, a biological information analysis system, and a biological information analysis program that can accurately grasp the mental state of a subject.

Means for Solving the Problems

[0006] To solve the above-mentioned problems and achieve the objective, the present invention is a bio-information analysis method that includes acquiring bio-information including sympathetic nerve data, parasympathetic nerve data, and autonomic nerve data including total power data obtained by integrating the sympathetic nerve data and the parasympathetic nerve data, behavioral record information recording multiple actions of the subjects, and question answer information answered by the subjects, for a predetermined period of time of multiple subjects having specific mental tendencies, and constructing a trained model by performing machine learning using the bio-information, behavioral record information, and question answer information.

[0007] Furthermore, the present invention is a biological information analysis device comprising: an acquisition unit that acquires biological information including sympathetic nerve data, parasympathetic nerve data, and autonomic nerve data including total power data obtained by integrating the sympathetic nerve data and the parasympathetic nerve data, for a predetermined period of time of multiple subjects having a specific mental tendency; behavioral record information recording multiple actions of the subjects; and question answer information answered by the subjects; and a learning unit that constructs a trained model by performing machine learning using the biological information, the behavioral record information, and the question answer information.

[0008] Furthermore, the present invention is a bio-information analysis system comprising: an acquisition unit that acquires bio-information including sympathetic nerve data, parasympathetic nerve data, and autonomic nerve data including total power data obtained by integrating the sympathetic nerve data and the parasympathetic nerve data, for a predetermined period of time of multiple subjects having a specific mental tendency; behavioral record information recording multiple actions of the subjects; and question answer information answered by the subjects; and a learning unit that constructs a trained model by performing machine learning using the bio-information, the behavioral record information, and the question answer information.

[0009] Furthermore, the present invention is a bio-information analysis program that, for a predetermined period of time, acquires bio-information including sympathetic nerve data, parasympathetic nerve data, and autonomic nerve data including total power data in which the sympathetic nerve data and the parasympathetic nerve data are integrated, behavioral record information recording multiple actions of the subjects, and question answer information answered by the subjects, and constructs a trained model by performing machine learning using the bio-information, behavioral record information, and question answer information, and causes a computer to execute each of these processes. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide a method for analyzing biological information, a biological information analysis device, a biological information analysis system, and a biological information analysis program that can accurately grasp the mental state of a subject. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 shows an example of a schematic configuration of an information processing device according to the embodiment. [Figure 2] Figure 2 shows an example of the hardware configuration of the information processing device according to the present invention. [Figure 3] Figure 3 is a flowchart showing an example of the operation of the information processing device according to the present invention. [Figure 4] Figure 4 is a diagram illustrating the processing of the acquisition unit. [Figure 5] Figure 5 is a diagram illustrating the processing of the acquisition unit. [Figure 6] Figure 6 is a diagram illustrating the processing of the acquisition unit. [Figure 7] Figure 7 is a diagram illustrating the processing of the acquisition unit. [Figure 8] Figure 8 is a diagram illustrating the processing of the acquisition unit. [Figure 9] Figure 9 is a diagram illustrating the processing of the acquisition unit. [Figure 10]FIG. 10 is a diagram for explaining the processing of the calculation unit. [Figure 11] FIG. 11 is a diagram for explaining the processing of the generation unit. [Figure 12] FIG. 12 is a diagram for explaining the processing of the generation unit. [Figure 13] FIG. 13 is a diagram for explaining the processing of the generation unit. [Figure 14] FIG. 14 is a diagram for explaining the processing of the output control unit. [Figure 15] FIG. 15 is a diagram for explaining the processing of the output control unit. [Figure 16] FIG. 16 is a diagram for explaining the processing of the output control unit. [Figure 17] FIG. 17 is a diagram for explaining the processing of the output control unit. [Figure 18] FIG. 18 is a diagram for explaining the processing of the output control unit. [Figure 19] FIG. 19 is a diagram for explaining the processing of the output control unit. [Figure 20] FIG. 20 is a diagram showing an example of the schematic configuration of the information processing apparatus according to the second embodiment. [Figure 21] FIG. 21 is a diagram showing the processing during learning and operation performed by the information processing apparatus according to the second embodiment. [Figure 22] FIG. 22 is a flowchart showing an operation example of the information processing apparatus according to the second embodiment. [Figure 23] FIG. 23 is a diagram showing an example of the schematic configuration of the system according to the embodiment. [Figure 24] FIG. 24 is a diagram for explaining the learned models according to each example and comparative example.

MODE FOR CARRYING OUT THE INVENTION

[0012] Hereinafter, a biological information analysis method, biological information analysis apparatus, biological information analysis system, and biological information analysis program according to the embodiments will be described with reference to the drawings. Note that the following embodiments are not limited to those described below. Furthermore, each embodiment can be combined with other embodiments or prior art to the extent that there is no inconsistency in the processing content.

[0013] The following section describes how this embodiment applies to psychological counseling. Specifically, it describes how the information processing device according to this embodiment is used by a person conducting psychological counseling (for example, a counselor, a doctor, etc.) to understand the mental state of the person receiving psychological counseling. It should be noted that this embodiment is not limited to psychological counseling and can be applied in various situations to understand the mental state of a person.

[0014] (First Embodiment) Figure 1 shows an example of a schematic configuration of an information processing device according to the embodiment. As shown in Figure 1, the information processing device 20 according to the embodiment is capable of communicating directly or indirectly with the sensor 11 and the mobile terminal 12 via a network such as a LAN (Local Area Network) or WAN (Wide Area Network). Note that the information processing device 20 is an example of a biological information analysis device.

[0015] Furthermore, the sensor 11 and the mobile terminal 12 do not need to be constantly connected to the information processing device 20. For example, it is sufficient for them to be connected only when a session is conducted with a target person in order to exchange various types of information. Also, when information is exchanged via an external recording medium or the like, the sensor 11 and the mobile terminal 12 do not need to be connected to the information processing device 20.

[0016] Sensor 11 is a sensor device attached to a subject. For example, sensor 11 is a multi-sensor device equipped with multiple sensor functions. Specifically, sensor 11 has the functions of an electrocardiograph, an optical pulse wave meter, a body movement meter, and a thermistor. The electrocardiograph measures time-series electrocardiogram waveform data. The optical pulse wave meter measures time-series pulse wave data. The body movement meter measures time-series acceleration data. The thermistor measures time-series temperature data. Sensor 11 records the various measured data in the device's internal memory, corresponding to the measurement start time. Note that known technologies can be appropriately selected and applied for the electrocardiograph, optical pulse wave meter, body movement meter, and thermistor.

[0017] Here, the sensor 11 is basically worn by the subject at all times. When a session is held, the subject removes the sensor 11 and hands the removed sensor 11 to the operator of the information processing device 20 (typically a counselor or doctor). The operator of the information processing device 20 transfers the measurement data recorded since the previous session from the sensor 11 received from the subject to the information processing device 20. Details of the process for acquiring biometric information from the measurement data will be described later.

[0018] Furthermore, the sensor 11 is not necessarily limited to a multi-sensor device; it only needs to have at least the functionality of an electrocardiograph. Also, the sensor 11 is not necessarily limited to being worn at all times; it may be removed as long as it does not affect the biometric information analysis processing described later. In addition, the timing of transferring the measurement data recorded by the sensor 11 is not necessarily limited to during a session; for example, it may be done periodically via the network (for example, once a day).

[0019] The mobile terminal 12 is, for example, a smartphone owned by the subject, and a recording application for collecting behavioral record information that records the subject's actions is installed on it. The behavioral record information is recorded automatically or manually by the subject in the internal memory of the mobile terminal 12. The behavioral record information recorded on the mobile terminal 12 is transferred to the information processing device 20 at any time. Details of the process for acquiring behavioral record information will be described later.

[0020] The mobile device 12 is not necessarily limited to a smartphone; for example, any information processing device capable of installing an application for collecting behavioral data, such as a tablet or personal computer, is acceptable. However, given the nature of collecting behavioral data, it is preferable that the device be one that the subject can carry with them on a daily basis (a mobile device).

[0021] The information processing device 20 is, for example, a computer such as a personal computer or a workstation. The information processing device 20 calculates mental state information indicating the mental state of the subject based on the information acquired by the sensor 11 and the mobile terminal 12. The information processing device 20 comprises an acquisition unit 201, a calculation unit 202, a generation unit 203, and an output control unit 204. However, the functions of the information processing device 20 are not limited to the acquisition unit 201, the calculation unit 202, the generation unit 203, and the output control unit 204. The acquisition unit 201, the calculation unit 202, the generation unit 203, and the output control unit 204 will be described later.

[0022] Here, the hardware configuration of the information processing device 20 will be described using Figure 2. Figure 2 is a diagram showing an example of the hardware configuration of the information processing device 20 according to the embodiment. As shown in Figure 2, the information processing device 20 includes a CPU (Central Processing Unit) 21, a ROM (Read Only Memory) 22, a RAM (Random Access Memory) 23, an auxiliary storage device 24, an input device 25, a display device 26, and an external I / F (Interface) 27.

[0023] The CPU 21 is a processor (processing circuit) that comprehensively controls the operation of the information processing device 20 by executing programs, and realizes the various functions of the information processing device 20. The various functions of the information processing device 20 will be described later.

[0024] ROM22 is a non-volatile memory that stores various data, including programs for starting the information processing device 20 (information written during the manufacturing stage of the information processing device 20). RAM23 is a volatile memory that contains the CPU 21's workspace. Auxiliary storage device 24 stores various data, such as programs executed by the CPU 21. Auxiliary storage device 24 is composed of, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc.

[0025] The input device 25 is a device used by an operator of the information processing device 20 to perform various operations. The input device 25 consists of, for example, a mouse, keyboard, touch panel, or hardware keys. The operator is, for example, a medical professional such as a doctor.

[0026] The display device 26 displays various types of information. For example, the display device 26 displays image data, model data, a GUI (Graphical User Interface) for receiving various operations from the operator, and medical images. The display device 26 is composed of, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, or a cathode ray tube display. Alternatively, the input device 25 and the display device 26 may be integrated into a single unit, such as a touch panel.

[0027] External I / F27 is an interface for connecting (communicating) with external devices such as sensor 11 and mobile terminal 12.

[0028] Using Figure 3, we will explain the process by which the information processing device 20 calculates the mental state information of the subject (biometric information analysis process). Figure 3 is a flowchart showing an example of the operation of the information processing device 20 according to the embodiment. In explaining Figure 3, we will refer to Figures 4 to 19. Note that the biometric information analysis process shown in Figure 3 can be executed at any time, but it is preferable to execute it each time a session is conducted with the subject.

[0029] As shown in Figure 3, the acquisition unit 201 acquires the subject's biological information (step S101). For example, the acquisition unit 201 acquires the subject's biological information by generating the subject's biological information based on various measurement data collected by the sensor 11.

[0030] Here, biological information includes, for example, autonomic nervous system data, total power data, physical movement data, body temperature data, pulse rate data, etc. Autonomic nervous system data is time-series information of autonomic nervous system activity indicators and includes sympathetic nervous system data, parasympathetic nervous system data, and total power data. Sympathetic nervous system data is time-series information of sympathetic nervous system activity indicators (Sympathetic Nervous System: SNS level). Parasympathetic nervous system data is time-series information of parasympathetic nervous system activity indicators (ParaSympathetic Nervous System: PSNS level). Total power data is integrated data of sympathetic nervous system data and parasympathetic nervous system data, for example, corresponding to the sum of sympathetic nervous system data and parasympathetic nervous system data. Physical movement data is time-series information indicating the intensity of physical movement. Body temperature data is time-series information of body temperature at the body surface (the site where sensor 11 is attached). Pulse rate data is time-series information of pulse rate.

[0031] Figures 4 and 5 illustrate an example of the process for acquiring a subject's biological information. Figures 4 and 5 are diagrams illustrating the processing of the acquisition unit 201. Figure 4 shows an example of electrocardiogram waveform data (ECG) measured by the sensor 11. Figure 5 shows an example of autonomic nervous system data generated by the acquisition unit 201.

[0032] For example, the acquisition unit 201 acquires autonomic nervous system data by generating autonomic nervous system data from electrocardiogram waveform data that shows the time-series electrocardiogram waveform. Specifically, the acquisition unit 201 detects the position of the R wave from the electrocardiogram waveform data shown in Figure 4 and generates a time series of heart rate interval variability (RRI time series) by plotting the detected R wave interval (RR interval: RRI) over time. Then, the acquisition unit 201 calculates the power spectrum from the RRI time series and integrates the power in a predetermined frequency range. As an example, the acquisition unit 201 calculates the integrated power value in the low frequency band (0.05Hz~0.15Hz) as the SNS level (upper part of Figure 5) and the integrated power value in the high frequency band (0.15Hz~0.40Hz) as the PSNS level (lower part of Figure 5). Generally, it is known that the SNS level is dominant in a stressed state, and the PSNS level is dominant in a relaxed state. In this way, the acquisition unit 201 generates sympathetic nerve data and parasympathetic nerve data, respectively.

[0033] Furthermore, the acquisition unit 201 acquires total power data based on sympathetic nerve data and parasympathetic nerve data. For example, the acquisition unit 201 acquires total power data by adding the SNS level and PSNS level corresponding to each time point.

[0034] In this way, the acquisition unit 201 acquires the subject's biological information based on various measurement data collected by the sensor 11. It should be noted that the methods for acquiring sympathetic nerve data, parasympathetic nerve data, and total power data described above are merely examples and are not limited to the above explanation. For example, known technologies can be appropriately selected and applied as methods for acquiring sympathetic nerve data, parasympathetic nerve data, and total power data.

[0035] Furthermore, the acquisition unit 201 can acquire data based on various measurement data collected by the sensor 11, in addition to the data mentioned above. For example, the acquisition unit 201 can acquire time-series data of heart rate and respiratory rate from electrocardiogram waveform data. The acquisition unit 201 can also acquire time-series data of transcutaneous arterial oxygen saturation (SpO2) and pulse rate from pulse wave data. The acquisition unit 201 can also acquire body movement data from acceleration data. The acquisition unit 201 can also acquire body temperature data from temperature data. Known techniques can be appropriately selected and applied as methods for acquiring this data.

[0036] Next, the acquisition unit 201 acquires the subject's behavioral record information (step S102). For example, the acquisition unit 201 acquires behavioral record information from the subject's mobile terminal 12. Here, the behavioral record information includes behavioral data indicating the content of each behavior, time data indicating the time each behavior occurred, location data indicating the place where each behavior occurred, medication history data indicating the medication history, and mood history data indicating the mood history at the time each behavior occurred. In addition, the behavioral data is represented by gradual classification information.

[0037] Figures 6 and 7 illustrate an example of the process for acquiring behavioral record information of a subject. Figures 6 and 7 are diagrams illustrating the process of the acquisition unit 201. Figure 6 shows an example of behavioral record information recorded by the mobile terminal 12. Figure 7 shows an example of the classification of behavioral data.

[0038] As shown in Figure 6, the mobile terminal 12 collects behavioral record information, including behavioral data, time data, location data, medication history data, and mood history data. For example, in the behavioral record information shown in Figure 6, the second record is associated with the behavioral data "Breakfast Meal", time data "1 hour 15 minutes 7:00", location data "Home", medication history data "Drug name: AAA Medication time 7:40", and mood history data "Facial expression mark (smiling)". In other words, the second record indicates that the subject ate breakfast at home for 1 hour and 15 minutes starting at 7:00, and that the subject was in a good mood at that time. The mobile terminal 12 similarly collects behavioral record information, including behavioral data, time data, location data, medication history data, and mood history data, for other records as well.

[0039] Furthermore, behavioral data is preferably represented by hierarchical classification information, as shown in Figure 7. For example, behavioral data can be divided into major, medium, and minor categories. For instance, the major category "Work" includes the medium categories "Meetings" and "Desk Work." The medium category "Desk Work" also includes the minor categories "Report Preparation" and "Document Organization." While the classification information for behavioral data is predefined, it is preferable that the subject can add, delete, and edit the classification information.

[0040] Here, activity data, time data, and location data are automatically entered by the mobile terminal 12 based on the schedule. For example, the mobile terminal 12 obtains the subject's schedule information from a scheduling application installed on the mobile terminal 12. This schedule information includes information such as the subject's planned activities, date and time, and location. The mobile terminal 12 then extracts activity data, time data, and location data from the schedule information and records the extracted information as activity record information. The activity data, time data, and location data automatically recorded by the mobile terminal 12 can be manually modified.

[0041] Furthermore, location data may be automatically corrected based on coordinate information acquired by the GPS (Global Positioning System) function of the mobile terminal 12. In this case, each location data and the coordinate information of that location from the GPS function are pre-associated and registered in the recording application. If the location data corresponding to the coordinate information actually acquired by the GPS function differs from the location data based on the schedule information, the mobile terminal 12 discards the location data based on the schedule information. If the coordinate information is moving, "movement" is entered as both location data and activity data.

[0042] Furthermore, medication history data and mood history data are entered manually. For example, each time a participant takes medication, they launch a recording application and enter the name of the medication taken and the time of administration. After completing each action, the participant selects the mood history data corresponding to how they felt at the time of that action. Medication history data and mood history data can also be manually modified. In addition to facial expression icons, mood history data may also be represented by text information indicating mood, such as "sleepy" or "relaxed," and numerical information indicating the degree of that mood.

[0043] In this way, the mobile terminal 12 collects behavioral record information of the subject. The mobile terminal 12 then transfers the collected behavioral record information to the information processing device 20. The timing of the transfer of behavioral record information can be set arbitrarily. For example, the mobile terminal 12 may transfer the behavioral record information periodically, or it may transfer it in response to a request from the subject or the operator of the information processing device 20. As a result, the acquisition unit 201 acquires the subject's behavioral record information from the mobile terminal 12.

[0044] Furthermore, the subject's behavioral record information can be corrected based on the physical movement data acquired from the sensor 11. For example, the acquisition unit 201 acquires the subject's sleep start and end times based on the intensity of the physical movements in the physical movement data. If the sleep start and end times based on the physical movement data differ from the sleep start and end times in the behavioral record information, the sleep start and end times in the behavioral record information can be discarded and overwritten with the sleep start and end times based on the physical movement data.

[0045] Furthermore, the acquisition unit 201 acquires the subject's question-answer information (step S103). For example, the acquisition unit 201 acquires mental state response data regarding the subject's mental state, and session evaluation response data regarding the evaluation of the session conducted for the subject, as question-answer information.

[0046] Figures 8 and 9 illustrate examples of mental state response data and session evaluation response data. Figures 8 and 9 are diagrams illustrating the processing of the acquisition unit 201. Figure 8 shows an example of mental state response data. Figure 9 shows an example of session evaluation response data.

[0047] For example, the operator of the information processing device 20 distributes questionnaires corresponding to each item of the mental state response data (Figure 8) and session evaluation response data (Figure 9) to the subject each time a session is held. When the operator receives the questionnaire with the subject's answers filled in, the operator inputs the answers (question and answer information) entered into the information processing device 20. The input of the question and answer information may be performed by image recognition of data scanned with a camera or the like, or it may be entered manually by the operator. As a result, the acquisition unit 201 acquires the question and answer information, including the mental state response data and session evaluation response data.

[0048] Then, the calculation unit 202 calculates mental state information based on biological information, behavioral record information, and question answer information (step S104). For example, the calculation unit 202 identifies partial data from the time-series autonomic nervous system data that corresponds to the period in which each behavior occurred, and calculates the statistical value of the identified partial data as mental state information.

[0049] An example of mental state information is illustrated using Figure 10. Figure 10 is a diagram illustrating the processing of the calculation unit 202. In Figure 10, "XXX" is replaced with numerical values ​​representing the mental state information calculated by the calculation unit 202.

[0050] As shown in Figure 10, the calculation unit 202 calculates mental state information for each action based on various information acquired by the acquisition unit 201. Here, mental state information is, for example, representative values ​​of sympathetic nerve data, parasympathetic nerve data, and total power data for each action.

[0051] For example, the calculation unit 202 matches the time series of biological information with the time series of behavioral record information by comparing the measurement start time by the sensor 11 with the time data included in the behavioral record information. Then, the calculation unit 202 identifies the partial data from the time series of autonomic nervous system data that corresponds to the period in which each behavior occurred. Specifically, the calculation unit 202 identifies the sympathetic nervous system data from the time series of sympathetic nervous system data for the period in which the behavioral data "sleep" occurred, "24:00~7:00". Then, the calculation unit 202 calculates the average value of the sympathetic nervous system data for the identified period as the mental state information in Figure 10. The calculation unit 202 also calculates parasympathetic nervous system data and total power data in the same way as sympathetic nervous system data.

[0052] In this way, the calculation unit 202 calculates mental state information for each behavior based on biological information and behavioral record information. In the above explanation, the case in which the "mean value" is calculated as a representative value of the sympathetic nerve data for each behavior was described, but it is not limited to this, and any arbitrary statistical value such as peak value, median, and standard deviation can be calculated.

[0053] Furthermore, the calculation unit 202 calculates the subject's mental state score based on the mental state response data. For example, the calculation unit 202 calculates the subject's mental state score by inputting the scores for each item included in the mental state response data into an arbitrary function.

[0054] Furthermore, the calculation unit 202 calculates a session evaluation score for a session based on the session evaluation response data. For example, the calculation unit 202 calculates a session evaluation score related to the evaluation of a session by inputting the score for each item included in the session evaluation response data into an arbitrary function.

[0055] Furthermore, the calculation methods for the mental state score and session evaluation score can be modified as appropriate depending on the question items included in the respective question response information.

[0056] Then, the generation unit 203 generates comments based on the mental state information (step S105). For example, the generation unit 203 generates at least one of a comment and an image that evaluates the mental state in each action, based on the mental state information in each action.

[0057] An example of the process for generating comments will be explained using Figures 11, 12, and 13. Figures 11, 12, and 13 are diagrams illustrating the process of the generation unit 203. Figure 11 shows an example of a template for each comment. Figure 12 shows an example of a comment generated by the generation unit 203. Figure 13 shows an example of an overall judgment generated by the generation unit 203.

[0058] As shown in Figure 11, the auxiliary memory device 24 stores information that associates each of the multiple comments with the template for each comment and the conditions for generating each comment. For example, the auxiliary memory device 24 stores information that associates the template "○○% of the time between ○:○○ and ○:○○ is quality sleep" with the conditions for generating sympathetic nerve data "below threshold A", the conditions for generating parasympathetic nerve data "above threshold B", and the conditions for generating behavioral data "sleep". This information indicates that a comment based on the template "○○% of the time between ○:○○ and ○:○○ is quality sleep" will be generated when there is a behavior where the sympathetic nerve data is "below threshold A", the parasympathetic nerve data is "above threshold B", and the behavioral data is "sleep".

[0059] For example, the generation unit 203 determines whether the conditions for generating each comment are met for each action included in the action record information. If there is a comment that is determined to meet the conditions for generation, the generation unit 203 reads the template for that comment from the auxiliary storage device 24. Then, based on the read template, the generation unit 203 generates a comment for the action that meets the conditions for generation.

[0060] For example, if the generation unit 203 finds an activity where the sympathetic nerve data is "less than threshold A", the parasympathetic nerve data is "greater than or equal to threshold B", and the activity data is "sleep", it reads a comment template based on the template "XX% of the time between XX:XX and XX:XX was quality sleep". The generation unit 203 then calculates the start time, end time, and percentage of quality sleep for the "sleep" that meets the conditions, and inputs this calculated information into the template to generate a comment such as "29% of the time between 23:05 and 6:00 was quality sleep" (Figure 12). The percentage of quality sleep is calculated, for example, as the percentage of the time during which the PSNS level was above a predetermined value during sleep.

[0061] The comments generated by the generation unit 203 are assigned to actions that meet the conditions for generating that comment. For example, the comment "29% of the time between 23:05 and 6:00 was quality sleep" is assigned to sleep that ended at 6:00 on the date "2018 / 10 / 19".

[0062] Furthermore, the generation unit 203 generates an overall judgment of a common action that has been performed multiple times within a certain period, based on the mental state information for each of the multiple actions. For example, if the action data "sleep" was performed 7 times between the previous session and the current session, the generation unit 203 performs an "overall judgment of sleep" based on the mental state information for the 7 sleep sessions. Specifically, the generation unit 203 inputs the sympathetic and parasympathetic nerve data for the 7 sleep sessions into an arbitrary function to perform an overall judgment based on a 4-level score from "0" to "3" as shown in Figure 13. The generation unit 203 then generates an overall judgment comment corresponding to the overall judgment score. The overall judgment comment is stored in advance in the auxiliary storage device 24.

[0063] Then, the output control unit 204 outputs mental state information and comments (step S106). For example, the output control unit 204 outputs information that associates action data indicating each action, the period during which each action was performed, and mental state information for each action.

[0064] Figures 14, 15, and 16 illustrate examples of autonomic nervous system data display. Figures 14, 15, and 16 are diagrams illustrating the processing of the output control unit 204. In Figures 14, 15, and 16, the vertical axis corresponds to the SNS level, and the horizontal axis corresponds to the PSNS level. Also, in Figures 14, 15, and 16, the size of the circles corresponds to the duration (time) during which each action was performed.

[0065] As shown in Figure 14, for example, the output control unit 204 displays autonomic nervous system data for each activity. In the example shown in Figure 14, the output control unit 204 plots data for four activities: work, travel, eating, and sleep. This allows the operator to easily understand the differences in SNS levels and PSNS levels for each activity. For example, the operator can see that the SNS level is higher for work compared to travel, eating, and sleeping.

[0066] Furthermore, as shown in Figure 15, the output control unit 204 simultaneously displays the latest autonomic nervous system data and past autonomic nervous system data for each action. In the example shown in Figure 15, the output control unit 204 plots the autonomic nervous system data for the current job, the autonomic nervous system data for the previous job, and the autonomic nervous system data for the job before that. This allows the operator to easily grasp the difference between their current mental state and their past mental state for each action. For example, the operator can see that their SNS level for the job is higher than before.

[0067] Furthermore, as shown in Figure 16, the output control unit 204 simultaneously displays standard autonomic nervous system data for each activity. In the example shown in Figure 16, the output control unit 204 plots the autonomic nervous system data for each activity of the subject together with the autonomic nervous system data of a typical person of the same age group as the subject. This allows the operator to easily grasp the differences between the mental state of the subject and that of a typical person. For example, the operator can see that the subject has a high SNS level in all activities, including work, travel, eating, and sleeping. The operator can also see that the subject sleeps less than that of a typical person.

[0068] Furthermore, the output control unit 204 displays mental state information, including a mental state score and a session evaluation score.

[0069] Figures 17 and 18 illustrate examples of displaying mental state scores and session evaluation scores. Figures 17 and 18 are diagrams illustrating the processing of the output control unit 204. In Figures 17 and 18, the vertical axis corresponds to the index of each data, and the horizontal axis corresponds to the number of sessions. The vertical axis plots each data index during the first session with the value set to 100.

[0070] As shown in Figure 17, the output control unit 204 displays the mental state score and the total power data. This total power data is a representative value of the total power data for a period defined in each session (for example, one week). This allows the operator to simultaneously view objective mental state information detected by the sensor 11 along with the mental state score. For example, the mental state score may include fluctuations or lies depending on the subject's mental state at the time of their response, but the operator can simultaneously view mental state information that objectively shows the mental state that causes fluctuations or lies, which can be used to provide advice for the next session.

[0071] Furthermore, as shown in Figure 18, the output control unit 204 displays the session evaluation score and the total power data. This total power data is a representative value of the total power data for the period defined by each session (for example, one week). This allows the operator to simultaneously view objective mental state information detected by the sensor 11 along with the session evaluation score. For example, the mental state score may include fluctuations or lies depending on the mental state of the subject at the time of their response, but the operator can simultaneously view mental state information that objectively shows the mental state that causes fluctuations or lies, which can be useful when deciding on the content of the next session.

[0072] Furthermore, the output control unit 204 displays comments related to each action, corresponding to the action record information.

[0073] An example of comment display will be explained using Figure 19. Figure 19 is a diagram illustrating the processing of the output control unit 204. Figure 19 illustrates information that associates biological information, including arrhythmia, heart rate, physical activity level, sympathetic nervous system activity, and parasympathetic nervous system activity, with behavioral record information, including behavioral history and mood history. In Figure 19, the horizontal axis corresponds to time.

[0074] As shown in Figure 19, the output control unit 204 compares the measurement start time from the sensor 11 with the time data included in the behavioral record information, thereby displaying the time series of the biological information and the time series of the behavioral record information in association. The output control unit 204 then displays comments related to each behavior, corresponding to the behavioral record information.

[0075] Here, the comments generated by the generation unit 203 are assigned to actions that satisfy the conditions for generating the comment. For this reason, the output control unit 204 displays the comment "29% of the time between 23:05 and 6:00 was quality sleep" in association with the "sleep" that ended at "6:00" on the date "2018 / 10 / 19". The output control unit 204 also displays the comment "Excited during meal" in association with the "dinner" that ended at "20:00" on the date "2018 / 10 / 18".

[0076] In this manner, the output control unit 204 outputs mental state information and comments. While this description explains the case where the output control unit 204 displays the information on the display device 26, it is not limited to this. For example, the output control unit 204 may transmit the mental state information and comments to an external device, or store them in memory or a recording medium.

[0077] Furthermore, Figures 14 to 19 are merely examples, and it is not required that all display formats be displayed simultaneously. For example, the operator can select and display the example display formats shown in Figures 14 to 19 as needed. Also, the display items shown in Figures 14 to 19 can be changed at the operator's discretion. For example, the total power data shown in Figures 17 and 18 may be replaced with other mental state information, or multiple mental state information may be displayed. In addition, any statistical value such as mean, peak value, median, and standard deviation can be used as representative values ​​for the total power data.

[0078] As described above, in the information processing device 20 according to the embodiment, the acquisition unit 201 acquires biological information including autonomic nervous system data of the subject and behavioral record information recording multiple actions of the subject. The calculation unit 202 calculates mental state information for each action of the subject based on the biological information and behavioral record information. This allows the operator to accurately grasp the mental state of the subject. For example, according to the information processing device 20 according to the embodiment, the mental state can be objectively quantified based on electrocardiogram waveform data collected by the sensor 11, so the mental state of the subject can be accurately grasped.

[0079] Furthermore, for example, the information processing device 20 quantifies the mental state during each action, making it easier to identify actions that cause stress. Therefore, the operator can provide more appropriate advice to the subject.

[0080] (modified version) It is known that there are individual differences in the magnitude of autonomic nervous system data. Therefore, it is preferable to adjust the conditions (thresholds) for generating comments according to the total power data of the subject.

[0081] For example, the generation unit 203 adjusts the generation conditions for each comment stored in the auxiliary storage device 24 according to the subject's total power data. For instance, if the subject's total power data is larger than the standard data, the generation unit 203 increases the threshold included in the generation conditions. Conversely, if the subject's total power data is smaller than the standard data, the generation unit 203 decreases the threshold included in the generation conditions. Then, the generation unit 203 uses the adjusted generation conditions to determine whether or not the generation conditions for each comment are met. This allows the information processing device 20 to appropriately assign comments according to the size of the individual's autonomic nervous system data.

[0082] (Second embodiment) In the first embodiment, a case was described in which mental state information is calculated based on biometric information, behavioral record information, and question answer information, and comments are generated based on the calculated mental state information. However, the embodiments are not limited to this. For example, it is also possible to generate a mental state judgment result by inputting biometric information and behavioral record information into a trained model created by machine learning. The mental state judgment result is data corresponding to mental state response data and session evaluation response data that is generated (judged) by the trained model.

[0083] In other words, the information processing device 50 according to the second embodiment constructs a trained model for multiple subjects (second subjects) having a specific mental tendency, using biometric information including autonomic nervous system data for each subject, behavioral record information recording multiple actions of each subject, and question answer information answered by each subject. The information processing device 50 then inputs the biometric information and behavioral record information of an arbitrary subject (first subject) into the constructed trained model to generate a mental state determination result in which the mental state of the arbitrary subject is determined. Note that the first subject may be included among the second subjects.

[0084] In the second embodiment, the "second subject" includes, for example, individuals with specific mental tendencies. "Specific mental tendencies" include mental disorders (mental illnesses) such as schizophrenia, mood disorders (depression, bipolar disorder), panic disorder, eating disorders, developmental disorders, dementia, epilepsy, addiction, and higher brain dysfunction. Furthermore, among individuals with such specific mental tendencies, if the second subject includes individuals with mood disorders (depression, bipolar disorder) and eating disorders, the accuracy of the trained model (accuracy of generating mental state judgment results) tends to improve, and in particular, if the second subject includes individuals with depression, the accuracy of the trained model tends to improve even more. However, the second subject is not limited to individuals with mental disorders; it may also include individuals classified by mental tendencies such as preferences. Additionally, the second subject may include individuals receiving psychological counseling.

[0085] Figure 20 shows an example of the schematic configuration of an information processing device according to the second embodiment. As shown in Figure 20, the information processing device 50 according to the embodiment is capable of communicating directly or indirectly with the sensor 11 and the mobile terminal 12 via a network such as a LAN or WAN. The information processing device 50 is an example of a biological information analysis device. The sensor 11 and the mobile terminal 12 are the same as those shown in Figure 1, so their description is omitted.

[0086] The information processing device 50 comprises an acquisition unit 501, a determination unit 502, and an output control unit 503. Note that the acquisition unit 501 and the output control unit 503 are the same as the acquisition unit 201 and the output control unit 204 shown in Figure 1, respectively, so their description is omitted. Furthermore, the hardware configuration of the information processing device 50 is the same as the hardware configuration of the information processing device 20 shown in Figure 2, so its description is omitted. The determination unit 502 will be described later.

[0087] Using Figure 21, the learning and operation processes performed by the information processing device 50 according to the second embodiment will be explained. Figure 21 is a diagram showing the learning and operation processes performed by the information processing device 50 according to the second embodiment. Subject X shown in Figure 21 is an example of the first subject. Subjects S-1 to SN are examples of the second subject.

[0088] As shown in the upper part of Figure 21, during learning, the information processing device 50 performs machine learning using, for example, biometric information, behavioral record information, and question response information (mental state response data and session evaluation response data) of multiple subjects S-1 to SN (e.g., multiple individuals with depressive tendencies and multiple healthy individuals) as training data. Of the training data, biometric information and behavioral record information are input data, and question response information is ground truth data. Through this machine learning, a trained model is constructed that outputs a mental state assessment result for a subject by inputting the biometric information and behavioral record information of the subject to be diagnosed. This trained model is stored in a memory device (e.g., ROM 22, RAM 23, auxiliary memory device 24, etc.).

[0089] Then, as shown in the lower part of Figure 21, during operation, the information processing device 50 inputs the biometric information and behavioral record information of subject X, who is the subject of diagnosis, into the trained model constructed during learning, causing the trained model to output a mental state assessment result for subject X. Here, subject X is, for example, a person suspected of having depressive tendencies. The information processing device 50 then presents the mental state assessment result for subject X output from the trained model to the operator (or subject X).

[0090] The biometric information is the same as the biometric information described in the first embodiment. For example, the biometric information may include all of the following: autonomic nervous system data, body temperature data, pulse rate data, and physical movement data, or it may include only arbitrary data. Furthermore, the period for which the biometric information is collected can be arbitrarily set to several hours, several days, several weeks, etc.

[0091] Furthermore, the behavioral record information is the same as the behavioral record information described in the first embodiment. For example, the behavioral record information may include all of the behavioral data, time data, and mood history data, or it may include only arbitrary data. In addition, the target period for the behavioral record information can be set arbitrarily, but it is preferable that it be the same as the target period for the biological information.

[0092] Furthermore, the question-answer information is the same as the question-answer information described in the first embodiment. For example, the question-answer information may include both mental state response data and session evaluation response data, or it may include only one of the data. Also, the question-answer information may be acquired once as a trend over an arbitrary target period (preferably the same as the target period for biological information), or it may be acquired any number of times at arbitrary timings, but it is preferable to acquire it once as a trend over an arbitrary period.

[0093] Furthermore, the process of constructing a trained model is performed, for example, by the determination unit 502. That is, the determination unit 502 acquires biometric information, including autonomic nervous system data, behavioral record information, which records multiple actions of the second subject, and question answer information, for multiple second subjects who have a specific mental tendency. Here, the biometric information, behavioral record information, and question answer information of the second subject are collected in advance and stored in a storage device (e.g., ROM 22, RAM 23, auxiliary storage device 24, etc.). For example, the determination unit 502 acquires this information by reading the biometric information, behavioral record information, and question answer information of the second subject from the storage device. Then, the determination unit 502 constructs a trained model by performing machine learning using the acquired biometric information, behavioral record information, and question answer information. Note that the process of constructing a trained model is not limited to the determination unit 502, but can be executed by any processing unit (processor).

[0094] Using Figure 22, we will explain the process (mental state determination process) by which the information processing device 50 outputs the mental state determination result of subject X during operation. Figure 22 is a flowchart showing an example of the operation of the information processing device 50 according to the second embodiment. The mental state determination process shown in Figure 22 is a process for determining whether or not subject X, who is suspected of having depressive tendencies, actually has depressive tendencies. Furthermore, the mental state determination process can be executed at any time.

[0095] As shown in Figure 22, the acquisition unit 501 acquires the subject X's biological information (step S201). For example, the acquisition unit 501 acquires the subject X's biological information by generating the subject X's biological information based on various measurement data collected by the sensor 11.

[0096] Next, the acquisition unit 501 acquires the behavioral record information of subject X (step S202). For example, the acquisition unit 501 acquires the behavioral record information from subject X's mobile terminal 12.

[0097] The determination unit 502 then inputs the subject X's biometric information and behavioral record information to the trained model to generate a mental state determination result for subject X. For example, the determination unit 502 reads the trained model stored in the memory device. The determination unit 502 then inputs the biometric information and behavioral record information acquired by the acquisition unit 501 to the read trained model to cause the trained model to output the mental state determination result for subject X. Subsequently, the determination unit 502 sends the mental state determination result output from the trained model to the output control unit 503.

[0098] Furthermore, the mental state assessment results output from the trained model include the same type of information as the question-answer information used to construct the trained model. In other words, if the question-answer information used in machine learning includes both mental state response data and session evaluation response data, the mental state assessment results output from the trained model will include both mental state response data and session evaluation response data.

[0099] The output control unit 503 outputs the result of the mental state assessment of subject X. For example, the output control unit 503 displays the mental state assessment result on the display device 26. Note that the output destination of the mental state assessment result is not limited to the display device 26; for example, it may be sent to an external device or stored in memory or a recording medium.

[0100] As described above, in the information processing device 50 according to the second embodiment, the acquisition unit 501 acquires biological information including the subject's autonomic nervous system data and behavioral record information recording multiple actions of the subject. The determination unit 502 generates a mental state determination result for the subject by inputting the subject's biological information and behavioral record information into a trained model. This allows the operator to accurately grasp the subject's mental state.

[0101] For example, according to the information processing device 50 of the second embodiment, mental state judgment results corresponding to mental state response data and / or session evaluation response data can be automatically generated (determined) by a trained model, even if the subject does not fill them out themselves. Therefore, by viewing the mental state judgment results output from the information processing device 50, the operator can easily understand whether or not the subject has a particular mental tendency.

[0102] In the second embodiment, we described the case where the mental state assessment process is applied to a person suspected of having depressive tendencies, but it is not limited to this. For example, the mental state assessment process can also be applied to a person who is not suspected of having depressive tendencies. In other words, the subject X (also called the first subject or the subject to assessment) does not need to be particularly limited and can be any subject. For example, the first subject may be any of the multiple second subjects.

[0103] Furthermore, although the second embodiment described a case in which the trained model is constructed by the information processing device 50, the embodiments are not limited to this. For example, the trained model may be constructed by an external device different from the information processing device 50. In this case, the information processing device 50 acquires the trained model constructed by the external device and executes the mental state determination process described above.

[0104] (Modification 1 of the second embodiment) Furthermore, the information processing device 50 described in the second embodiment can not only present the mental state assessment results to the operator as they are, but can also calculate and present the "match rate" with the question answer information of individuals with depressive tendencies, etc.

[0105] For example, the determination unit 502 compares the result of the mental state assessment of subject X with representative question answer information for individuals with depressive tendencies. The representative question answer information for individuals with depressive tendencies may be any question answer information for individuals with depressive tendencies used as is, or it may be an average of question answer information for multiple individuals with depressive tendencies.

[0106] Specifically, the judgment unit 502 compares each item included in the mental state judgment result with each item included in the question answer information and calculates the number of matching items. Then, the judgment unit 502 calculates the "match rate" by dividing the number of matching items by the total number of items. Note that the calculation method of the "match rate" described here is merely an example, and any calculation method can be applied.

[0107] The output control unit 503 then outputs the "match rate" calculated by the determination unit 502. This allows the information processing device 50 to support the diagnosis by the operator (medical professional such as a doctor). For example, by viewing the match rate, the operator can accurately and easily diagnose whether or not the subject has a particular mental tendency.

[0108] Furthermore, if the matching rate is output, the mental state assessment result does not necessarily need to be output.

[0109] (Modification 2 of the second embodiment) Furthermore, the information processing device 50 described in Modification 1 of the second embodiment can not only present the matching rate directly to the operator, but can also provide an indication of the possibility of whether or not the person has depressive tendencies (an indication of possible mental tendencies).

[0110] For example, the indication of a possible mental tendency may be information that shows the degree of likelihood of depressive tendencies in stages. For instance, the mental tendency assessment result may be information that classifies the likelihood of depressive tendencies into three stages: "high," "medium," and "low," depending on the degree of agreement. In the case of classifying into three stages, two thresholds are used.

[0111] Furthermore, if information suggesting a possible mental tendency is output, the mental state assessment result and the matching rate do not necessarily need to be output.

[0112] Furthermore, the information presented to the operator suggesting possible mental tendencies is intended solely to support the diagnosis and is not the diagnosis itself. In other words, the processing performed by the information processing device 50 does not constitute a medical act.

[0113] (Modification 3 of the second embodiment) Furthermore, the trained model (mental state determination process) described in the second embodiment may also be applied to the information processing device 20 shown in the first embodiment. This allows the information processing device 20 to output mental state information and comments without the subject having to enter question answer information themselves.

[0114] Specifically, the information processing device 20 executes the process in step S203 of Figure 22 instead of the process in step S103 of Figure 3. In other words, the information processing device 20 generates a result for determining the subject's mental state by inputting the biometric information and behavioral record information acquired in steps S101 and S102 of Figure 3 into a trained model.

[0115] Then, in step S104 of Figure 3, the information processing device 20 uses the mental state determination result instead of the question answer information. In other words, the information processing device 20 calculates mental state information based on biological information, behavioral record information, and the mental state determination result. The mental state determination result is data equivalent to the mental state response data and session evaluation response data, and can therefore be used in place of the question answer information.

[0116] The information processing device 20 can then output mental state information and comments by performing the same processing as in steps S105 and S106 of Figure 3.

[0117] (Other embodiments) In addition to the embodiments described above, the device may be implemented in various other forms.

[0118] (Use of sensors that can be attached to the body surface) In the embodiments described above, it was explained that the sensor 11 can be appropriately selected and applied from known technologies. However, in order to collect biological information over a long period of time (several days or several weeks), it is preferable to use a sensor that can be attached to the body surface. As such a sensor that can be attached to the body surface, it is preferable to use a wearable sensor that has a flexible device with a circuit inside that will not break when bent or otherwise broken, so that it can flexibly follow the body's movements. Furthermore, as such a sensor that can be attached to the body surface, it is preferable to use a sensor that can be attached to the body surface via an adhesive material such as an acrylic resin that does not cause discomfort to the body, has high medical quality characteristics, and is compliant with biological safety tests.

[0119] For example, the acquisition unit 201 acquires biological information based on data collected by a sensor 11 that can be attached to the surface of the subject's body. As a result, the acquisition unit 201 can collect biological information during both sleep and wakefulness without omission, thus enabling the collection of continuous biological information over a long period. The biological information used in this embodiment does not necessarily have to be continuous data, but it is preferable that it be continuous data. Such continuous data over a long period is preferably 24 hours or more, more preferably 72 hours or more, particularly preferably 144 hours or more, and most preferably 168 hours or more. Furthermore, it is preferable that such continuous data include continuous data for weekdays and continuous data for holidays. With such continuous data, a more accurate trained model can be constructed in the second embodiment.

[0120] (Biometric information analysis system) Each of the functions provided by the information processing device 20 according to the above embodiment may also be provided as a system.

[0121] Figure 23 is a diagram showing an example of the schematic configuration of a system according to the embodiment. As shown in Figure 23, System 1 according to the embodiment comprises a server device 30 and a display terminal 40. The server device 30 and the display terminal 40 are capable of communicating with each other directly or indirectly via a network such as a LAN (Local Area Network) or WAN (Wide Area Network). System 1 is an example of a biological information analysis system.

[0122] Furthermore, while the sensor 11 and the mobile terminal 12 are capable of communicating with the server device 30 directly or indirectly via a network, they do not need to be constantly connected to the server device 30. Note that the sensor 11 and mobile terminal 12 have essentially the same configuration as those shown in Figure 1, so their explanation will be omitted.

[0123] The server device 30 is installed, for example, in a service center that provides biometric information analysis processing as a cloud service. The server device 30 comprises an acquisition unit 301, a calculation unit 302, a generation unit 303, and an output control unit 304. The processing contents of the acquisition unit 301, calculation unit 302, generation unit 303, and output control unit 304 are basically the same as the processing contents of the acquisition unit 201, calculation unit 202, generation unit 203, and output control unit 204 shown in Figure 1, so a detailed explanation is omitted.

[0124] The display terminal 40 is, for example, an information processing terminal used by users of the cloud service, and is a terminal equipped with a display device, such as a personal computer, workstation, smartphone, or tablet. Here, users of the cloud service correspond to those who provide psychological counseling (for example, counselors, doctors, etc.).

[0125] Here, the output control unit 304 of the server device 30 transmits information about the subject's mental state during each of their actions to the display terminal. The display control unit 401 of the display terminal 40 receives the information about the subject's mental state during each of their actions transmitted from the server device 30 and displays the received information about the subject's mental state during each of their actions. This allows the user to accurately understand the subject's mental state.

[0126] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions. Moreover, each processing function performed by each device can be implemented, in whole or in any part, by a CPU and the program that is analyzed and executed by that CPU, or by hardware using wired logic.

[0127] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified.

[0128] Furthermore, the biometric information analysis method described in the above embodiments can be implemented by executing a pre-prepared biometric information analysis program on a computer such as a personal computer or workstation. This biometric information analysis program can be distributed via a network such as the Internet. Alternatively, this biometric information analysis program can be recorded on a computer-readable recording medium such as a hard disk, flexible disk (FD), CD-ROM, MO, or DVD, and executed by reading it from the recording medium by a computer.

[0129] According to at least one embodiment described above, the mental state of the subject can be accurately grasped.

[0130] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents.

[0131] [Examples] The present invention will be described in more detail below based on examples, but the present invention is not limited thereto.

[0132] (Examples 1-5, Comparative Example 1) Using the method described in the second embodiment, a trained model was created that outputs mental state response data and session evaluation response data by inputting biometric information and behavioral record information. By varying the type of data used for machine learning, trained models were created for each of the Examples 1 to 5 and Comparative Example 1 shown in Figure 24. Figure 24 is a diagram illustrating the trained models for each example and comparative example.

[0133] First, we will explain the input information (training data) used to create the trained model. In this example, "biometric information," "behavioral record information," "mental state response data," and "session evaluation response data" were used as input information. In addition, input information was collected for one week (168 hours including weekdays and holidays) from 15 individuals with depressive tendencies and 15 healthy individuals.

[0134] The "biometric information" used included autonomic nervous system data, body temperature data, pulse rate data, and physical movement data. Autonomic nervous system data included sympathetic nervous system data, parasympathetic nervous system data, and total power data, each generated from electrocardiogram waveform data. Body temperature data (temperature data) is time-series information of body temperature at the subject's body surface. Pulse rate data is time-series information of pulse rate. Physical movement data indicates the amount of physical activity and was generated from acceleration data. The electrocardiogram waveform data, body temperature data, pulse wave data, and acceleration data were collected over time using sensors that could be attached to the subject's body surface (for example, wearable devices as described in Japanese Patent Publication No. 2017-510390).

[0135] For "behavioral record information," we used behavioral data, time data, and mood history data as shown in Figure 6. Behavioral record information was collected by having each participant record their activities over the target period (the 7 days mentioned above).

[0136] The "mental state response data" and "session evaluation response data" used were the same data as those shown in Figures 8 and 9, respectively. The mental state response data and session evaluation response data were collected by having each participant record their response once at the end of the target period, representing the trend for each example and comparative example over the target period.

[0137] Next, the creation of the trained models for each example and comparative example will be described. The trained models for each example and comparative example were created using various data corresponding to the "○ marks" in Figure 24 as training data. Furthermore, the trained models for each example and comparative example were created using machine learning with biometric information and behavioral record information as "input data" and mental state response data and session evaluation response data as "ground truth data". In this example, Random Forest was used as the machine learning method.

[0138] The trained model for Example 1 was created using 24 hours (1 day) of training data (input data and ground truth data) corresponding to a weekday (Monday to Friday). In Example 1, autonomic nervous system data was used as biometric information, and machine learning was performed without using body temperature data, pulse rate data, or body movement data.

[0139] The trained model for Example 2 was created using 24 hours (1 day) of training data corresponding to a holiday (Saturday and Sunday). In Example 2, autonomic nervous system data was used as biometric information, and machine learning was performed without using body temperature data, pulse rate data, or body movement data.

[0140] The trained model for Example 3 was created using 24 hours (1 day) of training data corresponding to a weekday.

[0141] The trained model in Example 4 was created using 72 hours (3 days) of training data corresponding to weekdays. This 3-day training data was, for example, an integrated version of the training data from Monday, Wednesday, and Friday.

[0142] The trained model for Example 5 was created using 168 hours (one week's worth) of training data corresponding to weekdays and holidays.

[0143] The trained model for the comparative example was created using 24 hours (1 day) of training data corresponding to a weekday. In the comparative example, pulse rate data was used as biometric information, and machine learning was performed without using autonomic nervous system data, body temperature data, or physical movement data.

[0144] Next, the evaluation method for the trained models in each example and comparative example will be described. From the subjects from whom the above training data was collected, one subject was randomly selected, and input information for one week (168 hours including weekdays and holidays) was collected and used as input data for each trained model. The collected information was extracted into periods corresponding to the "acquisition period" of the trained model in each example and comparative example, and used as input data.

[0145] Furthermore, mental state response data and session evaluation response data for evaluation were collected by having participants record them once at the end of each target period. The reason for collecting mental state response data and session evaluation response data once at the end of each target period was that it was thought that trends throughout each target period could be obtained. In addition, when the training data from Monday, Wednesday, and Friday were integrated, the mental state response data and session evaluation response data collected at the end of Friday's data collection were used.

[0146] The collected mental state response data and session evaluation response data were then compared with the output data (mental state response data and session evaluation response data) from each trained model to determine the matching rate. The matching rate was determined according to the proportion of matching items among the multiple items included in the mental state response data and session evaluation response data. Specifically, a match was determined if there was a 70% or higher match. The matching rate was also evaluated by the number of times a match was determined out of 10 trials (match count). Specifically, "AAA" indicates that the proportion of matching items was 90% or higher. "AA" indicates that the proportion of matching items was 85% or higher but less than 90%. "A" indicates that the proportion of matching items was 80% or higher but less than 85%. "B" indicates that the proportion of matching items was 75% or higher but less than 80%. "C" indicates that the proportion of matching items was less than 75%.

[0147] As shown in Figure 24, the degree of match for the trained model in Example 1 was "A". The degree of match for the trained model in Example 2 was "B". The degree of match for the trained model in Example 3 was "AA". The degree of match for the trained model in Example 4 was "AA". The degree of match for the trained model in Example 5 was "AAA". The degree of match for the trained model in the comparative example was "C".

[0148] The degree of match of the trained model in Example 5 was higher than that of other trained models. This result suggests that it is preferable to have at least 7 days' worth (168 hours' worth) of input data during training and operation. In other words, it is preferable for the acquisition unit 201 to acquire at least 7 days' worth of biological information and behavioral record information, and for the calculation unit 202 to calculate mental state information based on at least 7 days' worth of biological information and behavioral record information.

[0149] Furthermore, these results suggest that it is preferable for the input data during learning and operation to include both weekdays and holidays. Specifically, it is suggested that it is preferable for the acquisition unit 201 to acquire biometric information and behavioral record information for several days or more, including weekdays and holidays, and for the calculation unit 202 to calculate mental state information based on the biometric information and behavioral record information for several days or more, including weekdays and holidays.

[0150] Furthermore, the degree of match of the trained models in Examples 3 and 4 was higher than that of the trained models in Examples 1 and 2. This result suggests that it is preferable for the input data during training and operation to include body temperature data, pulse rate data, and body movement data as biometric information.

[0151] Furthermore, the degree of match of the trained model in Example 1 was higher than that of the trained model in Example 2. This result suggests that weekday data is preferable to holiday data as input data during training and operation.

[0152] Furthermore, the comparative example is an example using only pulse rate data that can be collected by a relatively large number of wearable devices. The degree of fit of the trained model in the comparative example was lower than that of any of the trained models in Examples 1 to 5. From this result, it is suggested that it is preferable to include autonomic nervous system data as biometric information in the input data during training and operation.

Claims

1. A method for operating a biological information analysis device, The bio-information analysis device acquires, for a predetermined period of time, bio-information including sympathetic nerve data, parasympathetic nerve data, and autonomic nerve data including total power data obtained by integrating the sympathetic nerve data and the parasympathetic nerve data, behavioral record information recording multiple actions of the first subjects, and question answer information answered by the first subjects. A construction step involves constructing a trained model that generates a mental state determination result containing the same type of information as the question response information, in which the mental state of an arbitrary second subject is determined by inputting the biometric information and behavioral record information of the second subject, by performing machine learning using the biometric information, behavioral record information and question response information. Perform The biometric information and behavioral record information of the first subject include continuous data including weekdays and holidays. How to operate a biological information analysis device.

2. The aforementioned predetermined period corresponds to a period of 144 hours or more. A method for operating the biological information analysis device according to claim 1.

3. The aforementioned action record information includes action data indicating each action and time data indicating the time when each action was performed. A method for operating the biological information analysis device according to claim 1 or 2.

4. The biological information of the first subject further includes body movement data indicating the intensity of the body movements of the first subject, The acquisition step involves correcting the behavioral record information of the first subject based on the physical movement data of the first subject. A method for operating a biological information analysis device according to any one of claims 1 to 3.

5. The information of the first subject's answers to the questions includes session evaluation response data relating to the evaluation of the session conducted by the person providing psychological counseling to the first subject, A method for operating a biological information analysis device according to any one of claims 1 to 4.

6. The information of the first subject that answers the questions includes mental state response data relating to the mental state of the first subject, A method for operating a biological information analysis device according to any one of claims 1 to 5.

7. The acquisition step involves acquiring the autonomic nervous system data of the first subject using a wearable sensor that can be attached to the body surface of the first subject. A method for operating a biological information analysis device according to any one of claims 1 to 6.

8. An acquisition unit acquires, for a predetermined period of time, biological information including sympathetic nerve data, parasympathetic nerve data, and autonomic nerve data including total power data in which the sympathetic nerve data and the parasympathetic nerve data are integrated, behavioral record information recording multiple actions of the first subjects, and question answer information answered by the first subjects. A learning unit constructs a trained model that generates a mental state determination result containing the same type of information as the question answer information, in which the mental state of a second subject is determined by inputting the biometric information and behavioral record information of an arbitrary second subject, by performing machine learning using the biometric information, behavioral record information and question answer information. Equipped with, The biometric information and behavioral record information of the first subject include continuous data including weekdays and holidays. A biological information analysis device.

9. An acquisition unit acquires, for a predetermined period of time, biological information including sympathetic nerve data, parasympathetic nerve data, and autonomic nerve data including total power data in which the sympathetic nerve data and the parasympathetic nerve data are integrated, behavioral record information recording multiple actions of the first subjects, and question answer information answered by the first subjects. A learning unit constructs a trained model that generates a mental state determination result containing the same type of information as the question answer information, in which the mental state of a second subject is determined by inputting the biometric information and behavioral record information of an arbitrary second subject, by performing machine learning using the biometric information, behavioral record information and question answer information. Equipped with, The biometric information and behavioral record information of the first subject include continuous data including weekdays and holidays. A biometric information analysis system.

10. For a predetermined period of time, for multiple first subjects having a specific mental tendency, biological information including sympathetic nerve data, parasympathetic nerve data, and autonomic nerve data including total power data in which the sympathetic nerve data and the parasympathetic nerve data are integrated, behavioral record information recording multiple actions of the first subjects, and question response information answered by the first subjects are acquired. By performing machine learning using the aforementioned biometric information, behavioral record information, and question response information, a trained model is constructed that, upon inputting the biometric information and behavioral record information of an arbitrary second subject, generates a mental state determination result that includes the same type of information as the question response information, thereby determining the mental state of the second subject. Let the computer execute each process, The biometric information and behavioral record information of the first subject include continuous data including weekdays and holidays. A program for analyzing biological information.

Images