Information processing systems, information processing methods, and programs

Abstract

This invention provides an information processing system, information processing method, and program that enable further improvements in the accuracy of estimating health status and other conditions based on biometric information and machine learning models. [Solution] In an information processing system in which an information processing device, a user terminal, and a wearable device are configured to communicate with each other via a telecommunications line such as a network, the processor of the information processing device has an acquisition unit that continuously acquires the user's biometric information S001, and an update unit that estimates a state score that quantitatively indicates the user's health or psychological state based on the continuously acquired biometric information and a learning model S002, updates a cumulative score obtained by continuously accumulating the estimated state scores S003, and displays visual information related to the cumulative score S004.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] For example, Patent Document 1 discloses "an information processing device that appropriately estimates information on an ideal physical condition of a user according to desired health information indicating a desired health condition of the user." The information processing device includes an acquisition unit that acquires a machine learning model learned to estimate health information indicating a health condition from biological information, and maps the health information of the user to the latent space of the machine learning model, and based on first health feature information corresponding to the health information of the user mapped to the latent space, an estimation unit that estimates ideal biological information, which is biological information corresponding to the desired health information, from the desired health information indicating the desired health condition of the user.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, for the functions according to the above-known technology, particularly in the aspect of estimating a health condition or the like from biological information, a technology for further improving the estimation accuracy is desired.

[0005] In view of the above circumstances, the present invention aims to provide an information processing system or the like that can further improve the estimation accuracy in estimating a health condition or the like based on biological information and a machine learning model.

Means for Solving the Problems

[0006] According to one aspect of the present invention, an information processing system is provided, comprising at least one processor, the processor being configured to execute a program such that the following steps are performed: in the acquisition step, the system continuously acquires the user's biometric information; in the update step, it estimates a state score that quantitatively indicates the user's health or psychological state based on the continuously acquired biometric information and a learning model, and updates a cumulative score obtained by continuously accumulating the estimated state scores; and in the display control step, it displays visual information relating to the cumulative score, thereby enabling the user to continuously grasp their health or psychological state.

[0007] This configuration allows for even greater accuracy in estimating health status and other conditions based on biometric information and machine learning models. [Brief explanation of the drawing]

[0008] [Figure 1] This is a diagram showing the configuration of Information Processing System 1. [Figure 2] This is a block diagram showing the hardware configuration of the information processing device 2. [Figure 3] This is a block diagram showing the hardware configuration of user terminal 3. [Figure 4] This is a functional block diagram showing the functions of the information processing device 2. [Figure 5] This flowchart shows an overview of the processes performed by Information Processing System 1. [Figure 6] This is an activity diagram showing specific examples of processes performed by Information Processing System 1. [Figure 7] This figure shows an example of a results screen 5 displayed on the display unit 34 of the user terminal 3 in this embodiment. [Figure 8] This is a diagram to explain the movement of objects on a graph. [Figure 9] This figure shows another example of how the estimated results of user U's psychological state are displayed. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. The various features shown in the embodiments below can be combined with each other.

[0010] Incidentally, the program for implementing the software appearing in one embodiment may be provided as a non-transitory computer-readable medium, or it may be provided as a downloadable medium from an external server, or it may be provided so that the program is launched on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0011] Furthermore, in various information processing according to one embodiment, an input and an output corresponding to the input can be realized. Here, as long as an output is obtained as a result of the input, the form of the information referenced in such information processing (hereinafter referred to as "reference information") is not limited. The reference information may be, for example, rule-based information such as a database, a lookup table, or a predetermined function (including a decision formula such as a regression equation constructed by a statistical method), or a pre-trained model that has learned the correlation between input and output in advance, or a large-scale language model that can output a desired result by inputting a prompt.

[0012] Furthermore, in one embodiment, "part" may include, for example, hardware resources implemented by a circuit in a broad sense, and the information processing of software that can be specifically realized by these hardware resources. Also, in one embodiment, various types of information are handled, and this information can be represented, for example, by the physical values ​​of signal values ​​representing voltage and current, the high or low values ​​of signal values ​​as a set of binary bits composed of 0s or 1s, or by quantum superposition (so-called qubits), and communication and calculations can be performed on a circuit in a broad sense.

[0013] Furthermore, a circuit in a broad sense is a circuit realized by combining at least a suitable combination of circuits, circuits, processors, and memory. The processor may be a general-purpose processor or a dedicated circuit. In other words, it includes application-specific integrated circuits (ASICs), programmable logic devices (for example, simple programmable logic devices (SPLDs), complex programmable logic devices (CPLDs), and field programmable gate arrays (FPGAs)), etc.

[0014] 1. Hardware Configuration This section describes the hardware configuration of one embodiment.

[0015] 1.1 Information Processing System 1 Figure 1 is a configuration diagram representing information processing system 1. Information processing system 1 comprises an information processing device 2, a user terminal 3, and a wearable device 4. The information processing device 2, the user terminal 3, and the wearable device 4 are configured to communicate with each other via a telecommunications line such as a network 11. In one embodiment, information processing system 1 consists of one or more devices or components. For example, if it consists only of the information processing device 2, then information processing system 1 can be the information processing device 2. If it consists only of the wearable device 4, then information processing system 1 can be the wearable device 4. These components will be described below.

[0016] 1.2 Information Processing Device 2 FIG. 2 is a block diagram showing the hardware configuration of the information processing apparatus 2. The information processing apparatus 2 includes a communication bus 20, a communication unit 21, a storage unit 22, and a processor 23. The communication unit 21, the storage unit 22, and the processor 23 are electrically connected to each other inside the information processing apparatus 2 via the communication bus 20.

[0017] <Communication unit 21> Although wired communication means such as USB, IEEE1394, Thunderbolt (registered trademark), wired LAN network communication, etc. are preferable for the communication unit 21, wireless LAN network communication, mobile communication such as 3G / LTE / 5G, BLUETOOTH (registered trademark) communication, etc. may be included as necessary. That is, it is more preferable to implement it as a collection of these plural communication means. That is, the information processing apparatus 2 may communicate various information from the outside via the communication unit 21 and the network.

[0018] <Storage unit 22> The storage unit 22 stores various information defined by the foregoing description. This may be implemented as a storage device such as a solid state drive (SSD) that stores various programs related to the information processing apparatus 2 executed by the processor 23, or as a memory such as a random access memory (RAM) that stores information (arguments, arrays, etc.) temporarily necessary for program calculations. The storage unit 22 stores various programs, variables, etc. related to the information processing apparatus 2 executed by the processor 23.

[0019] <Processor 23> The processor 23 performs processing and control of the overall operation related to the information processing device 2. The processor 23 is, for example, a central processing unit (CPU) not shown. The processor 23 realizes various functions related to the information processing device 2 by reading predetermined programs stored in the memory unit 22. That is, information processing by software stored in the memory unit 22 can be concretely realized by the processor 23, which is an example of hardware, and executed as each functional unit included in the processor 23. These will be described in more detail in the next section. Note that the processor 23 is not limited to being a single unit, and may be implemented with multiple processors 23 for each function, or a combination thereof.

[0020] 1.3 User Terminal A user terminal is a device owned by the user. A user terminal is defined as a terminal 3 operated by the user to measure its own internal state. The user terminal can be a smartphone, tablet, computer, or any other device capable of accessing the information processing device 2 via a telecommunications line; its form is not restricted.

[0021] Figure 3 is a block diagram showing the hardware configuration of user terminal 3. The following explanation will use user terminal 3 as an example. User terminal 3 comprises a communication bus 30, a communication unit 31, a storage unit 32, a processor 33, a display unit 34, and an input unit 35. The communication unit 31, storage unit 32, processor 33, display unit 34, and input unit 35 are electrically connected within user terminal 3 via the communication bus 30. The explanation of the communication unit 31, storage unit 32, and processor 33 is the same as the explanation of each part in the information processing device 2 and will therefore be omitted.

[0022] <Display section 34> The display unit 34 displays a graphical user interface (GUI) screen that can be operated by the user. The display unit 34 may be included in the casing of the user terminal 3 or it may be an external component. Specifically, the display unit 34 may be implemented as a display device such as a CRT display, liquid crystal display, organic EL display, or plasma display. It is preferable that these display devices be used according to the type of user terminal 3.

[0023] <Input section 35> The input unit 35 receives operation input from the user. The operation input is transmitted to the processor 33 via the communication bus 30 as a command signal. The processor 33 can perform predetermined controls or calculations based on the transmitted command signal as needed. The input unit 35 may be included in the casing of the user terminal 3 or it may be external. For example, the input unit 35 may be implemented as a touch panel integrated with the display unit 34. When the input unit 35 is implemented as a touch panel, the user can input tap operations, swipe operations, etc. to the input unit 35. Instead of a touch panel, the input unit 35 can be a switch button, a mouse, a QWERTY keyboard, etc.

[0024] 1.4 Wearable Devices 4 The wearable device 4 is worn by the user and is configured to continuously measure the user's biometric information IF10 via sensors (not shown). These sensors may include temperature sensors, infrared sensors, light sensors, pressure sensors, potential sensors, chemical sensors, acceleration sensors, etc. The wearable device 4 may also include, for example, a smartwatch, smart ring, smart glasses, smart earphones, etc. The wearable device 4 can transmit and receive information via wireless communication with the communication unit 21 of the information processing device 2. In one embodiment, the user terminal 3 and the wearable device 4 are described separately, but the wearable device 4 may also function as the user terminal 3.

[0025] 2. Functional Configuration This section describes the functional configuration. Figure 4 is a functional block diagram showing the functions of the information processing device 2. As mentioned above, the information processing performed by the software stored in the memory unit 22 is concretely realized by the hardware (specifically, the processor 23), and can be executed as each functional unit included in the processor 23. In other words, the information processing system 1 comprises each functional unit in the program.

[0026] Specifically, the processor 23 includes, as functional units, a reception unit 231, an acquisition unit 232, an estimation unit 233, an update unit 234, a determination unit 235, and a display control unit 236.

[0027] The reception unit 231 is configured to receive various types of information as a reception step. Specifically, the reception unit 231 is configured to receive information via the communication unit 21 or the storage unit 22 and to read it into the working memory. Preferably, the reception unit 231 receives input from the user via the input unit 35 of the user terminal 3 to initiate the estimation of the user's health or psychological state.

[0028] The acquisition unit 232 is configured to acquire various types of information as an acquisition step. Specifically, the acquisition unit 232 acquires the user's biometric information IF10 via the wearable device 4.

[0029] The estimation unit 233 is configured to estimate various types of information as an estimation step. Specifically, the estimation unit 233 is configured to estimate a state score SC10 that quantitatively indicates the user's health or psychological state based on the biometric information IF10 acquired by the acquisition unit 232. Further details will be described later.

[0030] The update unit 234 is configured to update various information stored in the memory unit 22 as an update step. Specifically, the update unit 234 updates the cumulative score SC20, which is obtained by accumulating the state scores SC10 estimated by the estimation unit 233. Further details will be described later.

[0031] The determination unit 235 is configured to make a determination regarding the execution of a process based on various information as a determination step. Specifically, the determination unit 235 determines whether the cumulative score SC20, which is the cumulative value of the state score SC10, has reached a predetermined threshold. Details will be described later.

[0032] As a display control step, the display control unit 236 causes various information stored in the storage unit 22, or screens containing such information, to be displayed in a manner that can be viewed on a terminal such as the user terminal 3. When the phrase "cause to display" is used, it is not particularly important whether the display medium to be displayed is in the local environment or whether processing is performed to display it via the network 11. In this way, various information is presented to the user operating the user terminal 3. The display control unit 236 controls the display of the terminal to show visual information such as screens, images, icons, and messages. The display control unit 236 may only generate rendering information for displaying the visual information on the terminal.

[0033] 3. Information Processing The information processing of Embodiment 1 will be described below.

[0034] 3.1 Overview of the process Information processing system 1 comprises at least one processor 23, which is configured to execute a program so that the following parts are performed. In other words, such information processing method comprises the steps of information processing system 1 described below. From another perspective, such program causes a computer to execute each step of information processing system 1. Figure 5 is a flowchart outlining the processing performed by information processing system 1. The steps shown in Figure 5 will be described below.

[0035] First, the acquisition unit 232 continuously acquires the user's biometric information IF10 (step S001). Next, the update unit 234 estimates a state score SC10 that quantitatively indicates the user's health or psychological state based on the continuously acquired biometric information IF10 and the learning model LM, and updates the cumulative score which is the sum of the estimated state scores (steps S002, S003). Furthermore, the display control unit 236 displays visual information IF30 related to the cumulative score SC20, thereby enabling the user to continuously understand their health or psychological state (step S004).

[0036] In this configuration, users can understand their own health or psychological state based on the cumulative score SC20, which is the cumulative value of the state score SC10 estimated based on the learning model LM. By using the cumulative score SC20 for health or psychological state, users can accurately understand information indicating their own health state, etc., without being significantly affected by temporary changes in measured or estimated values ​​during the measurement of biometric information IF10.

[0037] 3.2 Specific Examples of Information Processing Figure 6 is an activity diagram showing a specific example of processing performed by the information processing system 1. The specific example may be included within the scope defined in the overview described above. In the specific example, we will describe an example in which user U understands their own psychological state based on at least biometric information IF10. User U is wearing a wearable device 4, and the measurement unit of the wearable device 4 measures user U's biometric information IF10. In this embodiment, we will describe an example in which user U's psychological state is estimated based on biometric information IF10, but this is not limited to this example.

[0038] First, the measurement unit (not shown) of the wearable device 4 continuously measures the user U's biometric information IF10. The measured biometric information IF10 is transmitted from the wearable device 4 to the information processing device 2 (Activity A101). Here, the biometric information IF10 may include, for example, pulse wave, skin temperature, arm acceleration, absolute or relative value of sweat volume or an equivalent amount, or a biometric score obtained from at least one of these pieces of information. It may also include a sleep score that quantifies the quality and quantity of sleep, blood pressure, heart rate, blood oxygen saturation, respiratory rate, body temperature, electrocardiogram, electroencephalogram, blood glucose, alcohol, lactic acid, level of consciousness, number of steps, walking distance, exercise volume, or a biometric score obtained from at least one of these pieces of information.

[0039] Furthermore, the biometric information IF10 is acquired continuously. The biometric information IF10 may be acquired at different intervals. In this embodiment, it is explained that the measurement unit of the wearable device 4 measures at least the electroencephalogram, skin temperature, arm acceleration, and sweat volume. The measurement intervals are preferably 0.0025 to 0.005 seconds (200 to 400 Hz) for the electroencephalogram, 0.1 to 0.5 seconds (2 to 10 Hz) for the skin temperature, 0.025 to 0.050 seconds (20 to 40 Hz) for the arm acceleration, and 0.1 to 0.5 seconds (2 to 10 Hz) for the sweat volume (e.g., skin electrical activity).

[0040] Next, via the communication unit 21 of the information processing device 2, the acquisition unit 232 acquires the user U's biometric information IF10 as an acquisition step (activity A102). Thereafter, as time passes, the wearable device 4 continues to acquire the user U's biometric information IF10 and transmit it to the information processing device 2. The acquired biometric information IF10 is stored in the storage unit 22.

[0041] Independent of the processing related to Activity A102, the receiving unit 231 of the information processing device 2 receives input from user U to begin estimating the psychological state of user U (Activity A103).

[0042] Next, the determination unit 235, as a determination step, determines whether the biological information IF10 stored in the memory unit 22 satisfies predetermined conditions (activity A104). For example, the predetermined conditions may be that the state in which the acceleration of the arm is less than or equal to a predetermined value continues for a predetermined time or longer. The predetermined time is, for example, 10 to 60 seconds, preferably 20 to 40 seconds, and specifically, for example, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60 seconds, and may be within the range of any two of the values ​​exemplified here. The predetermined conditions are not limited to these. For example, the determination unit 235 may determine that a predetermined condition is met when a predetermined amount of time has elapsed since receiving input from user U in activity A103, that is, when data acquisition has continued for a predetermined period of time. Alternatively, for example, the determination unit 235 may determine that a predetermined condition is met when the number of data acquired by the acquisition unit 232 is greater than or equal to a predetermined amount. The predetermined condition may be, for example, a condition for determining that the measurement environment is stable enough to accurately acquire the biological information IF10.

[0043] If the biological information IF10 in activity A104 meets the predetermined conditions, the process proceeds to activity A105. On the other hand, if the biological information IF10 does not meet the conditions, the determination unit 235 makes a determination again after a predetermined time has elapsed.

[0044] Next, the estimation unit 233 estimates a state score SC10 based on at least a portion of the acquired biometric information IF10 and environmental information IF20 (Activity A105). The state score SC10 is a value that quantitatively represents the psychological state of user U. Here, the state score SC10 may be estimated by inputting at least the biometric information IF10 to the learning model LM.

[0045] The learning model LM is generated and updated, for example, by performing machine learning using training data. Training data refers to learning data and consists of pairs of input data and output data (correct answer data) for learning. In this embodiment, the input data is at least biometric information IF10, and the output data may be information indicating the user's psychological state. Information indicating the user's psychological state may include, for example, information received from the user based on the user's own subjective opinion. Preferably, the learning model LM may be generated for each user by performing machine learning individually. Alternatively, the learning model LM may be generated for each index indicating the psychological state. In this embodiment, it is explained that a learning model LMa for estimating the user's emotional valence and a learning model LMb for estimating the user's arousal level are generated, updated, and stored.

[0046] Furthermore, even after the learning model LM has been set in the information processing device 2 where it is used and has been used to perform user state estimation, additional learning may be performed. The learning model LM may be updated through this additional learning.

[0047] In this embodiment, the learning model LM is described as being stored in the memory unit 22 of the information processing device 2, but this is not limited to this. For example, the processor 23 of the information processing device 2 may be configured to receive at least the user U's biometric information IF10 as input to the learning model LM stored outside the information processing device 2, and to receive the output user U's state score SC10 from the learning model LM.

[0048] The state score SC10 is a value that quantitatively represents the internal state of user U. The state score SC10 is estimated by the learning model LM. Furthermore, the state score SC10 is estimated for each indicator. In this embodiment, at least emotional valence and arousal level are included as indicators, and the state score SC10 is estimated for emotional valence and arousal level, respectively. Preferably, the learning model LM includes a learning model LMa for estimating emotional valence and a learning model LMb for estimating arousal level, and learning models LMa and LMb may have different configurations. More specifically, the types of biometric information IF10 input to learning model LMa and learning model LMb may be the same or different. Details of emotional valence and arousal level will be described later.

[0049] Furthermore, in this embodiment, logistic regression is used as the learning algorithm for the learning model LM. For example, the state of user U estimated by the estimation unit 233 in response to the input of biometric information IF10 to the learning model LM is quantified and output in the range of 0 to 1. In this embodiment, if the output value is 0.5 or greater, the state score SC10 is determined to be +1 (positive), and if it is less than 0.5, it is determined to be -1 (incorrect). The learning algorithm for the learning model LM is not limited to this.

[0050] Furthermore, in this embodiment, in activity A105, the state score SC10 is estimated based on environmental information IF20. That is, the acquisition unit 232 further acquires environmental information IF20. Environmental information IF20 is information about the environment surrounding user U. The state score SC10 is estimated based on at least environmental information IF20, biometric information IF10, and the learning model LM. Environmental information IF20 may include the room temperature, humidity, brightness, noise level of the space where user U is present when biometric information IF10 is measured, or an environmental score obtained from at least one of these pieces of information. Environmental information IF20 may be measured by the measurement unit of the measurement terminal 8 (not shown) and transmitted from the measurement terminal 8 to the information processing device 2. That is, the acquisition unit 232 of the information processing device 2 may acquire environmental information IF20 from the measurement terminal 8 via the communication unit 21. Preferably, the acquisition unit 232 may continuously acquire environmental information IF20, but is not limited to this. For example, the environmental information IF20 may be configured to be acquired only once by the acquisition unit 232. In this configuration, the estimated psychological state of the user can also reflect information about the user's surrounding environment. Therefore, the user's health condition and other factors can be estimated more accurately.

[0051] Next, the processor 23 calculates the cumulative value of the state score SC10. Then, the determination unit 235 determines whether or not this cumulative value has reached a predetermined threshold (activity A106). More specifically, the cumulative value of the state score SC10 is calculated by adding the state score SC10 of user U, newly calculated in activity A105, to the cumulative score SC20 accumulated for user U. Then, the determination unit 235 determines whether or not this cumulative value has reached a threshold. This cumulative value includes the value obtained by accumulating the state score SC10 that has been continuously estimated since the estimation started in activity A103. In this embodiment, the case where the upper limit of the cumulative score SC20 is set to +5 and the lower limit to -5 is set as the threshold is described. Also, although the thresholds for emotional valence and arousal are described as being the same, different thresholds may be set depending on the indicator.

[0052] Furthermore, the threshold can be changed according to the user's characteristics. For example, different thresholds may be set in advance for each user according to their characteristics. In addition, the threshold may be changed based on the accumulated user biometric information IF10 or estimation results. According to such a configuration, the user can more appropriately understand the changes in their health or psychological state over time.

[0053] If the cumulative value of the status score SC10 has not reached the threshold, the process proceeds to activity A107. The update unit 234 updates the cumulative score SC20 of user U. Specifically, the update unit 234 updates the cumulative score SC20 based on the cumulative value calculated in activity A106.

[0054] On the other hand, if the cumulative score SC20 reaches a threshold, the process proceeds to activity A108. In this case, the update unit 234 retains the value of the cumulative score SC20 without updating it. In other words, if the cumulative score SC20 exceeds a predetermined threshold when the state score SC10 is added, the update unit 234 restricts the updating of the cumulative score SC20. More specifically, the state score SC10 newly calculated in activity A105 is not added to the cumulative score SC20 accumulated for user U. Through this process, the cumulative score SC20 does not deviate from the preset threshold. With this configuration, the user can appropriately grasp the changes in their health or psychological state over time. More specifically, if the user's psychological state is stable, continuing to accumulate the state score SC10 and update the cumulative score SC20 will cause the cumulative score SC20 to become biased to one side in the axial direction. When the cumulative score SC20 is skewed towards one axis, even if a change occurs in the user's psychological state, such a change is unlikely to be reflected in the change in the cumulative score SC20. By setting a threshold, it is possible to avoid this situation and resolve the issues that arise from the accumulation of scores.

[0055] Next, the cumulative score SC20 determined in activity A107 or A108 is stored in the storage unit 22 (activity A109). Here, it is preferable that the data structure in which the cumulative score SC20 is stored has a queue.

[0056] Next, the display control unit 236, as a display control step, causes visual information IF30 related to the cumulative score SC20 to be displayed on the display unit 34 of the user terminal 3 (Activity A110). Details will be described later.

[0057] Next, the determination unit 235 determines whether the acquisition status of user U's biometric information IF10 satisfies predetermined conditions (activity A111). Specifically, the determination unit 235 determines whether user U's biometric information IF10 is continuously measured at predetermined intervals and acquired by the acquisition unit 232. If the acquisition status of biometric information IF10 satisfies predetermined conditions, the processing of activities A105 to A110 is repeated. The estimation period for the state score SC10 is, for example, 0.5 to 10 seconds, preferably 1 to 3 seconds. In this embodiment, the estimation period for the state score SC10 is assumed to be every second. Furthermore, the estimation of the state score SC10 may be performed by inputting the biometric information IF10 acquired within a predetermined time starting from the estimation time into the learning model LM. Alternatively, the estimation period for the state score SC10 may be determined based on the number of biometric information IF10 acquired. In this embodiment, the state score SC10 is estimated by inputting biometric information IF10, acquired within the past 30 seconds starting from the estimation time, into the learning model LM. The estimation period for the state score SC10 and the acquisition period for biometric information IF10 are not limited to this.

[0058] On the other hand, if it is determined that the acquisition status of user U's biometric information IF10 does not meet the predetermined conditions, the process is terminated. However, the predetermined conditions are not limited to these. For example, the predetermined condition may be that user U has given input to terminate the estimation.

[0059] The above is the processing flow for the specific example. 4. Related technical matters

[0060] (Result screen 5) Figure 7 shows an example of the results screen 5 displayed on the display unit 34 of the user terminal 3 in this embodiment. The results screen 5 is an example of the screen displayed on the display unit 34 in activity A110 of Figure 6. The results screen 5 includes a graph 50 which is visual information IF30.

[0061] This section describes graph 50 (visual information IF30) on which the cumulative score SC20 is plotted. The cumulative score SC20 has at least two indicators. Visual information IF30 has a configuration in which the cumulative score SC20 is plotted on a coordinate space having axes corresponding to the indicators. More specifically, visual information IF30 shows the estimated result of user U's psychological state as an object on a plane defined by emotional valence (horizontal axis) and arousal level (vertical axis). With this configuration, the user can easily grasp where their own psychological state is positioned.

[0062] Graph 50 includes an X-axis 50x and a Y-axis 50y. The X-axis 50x is the horizontal axis of the graph, and the position of the object on the X-axis 50x represents the emotional valence of user U. In this embodiment, the further the object moves on the X-axis 50x in the direction of a negative value, the longer user U is in an unpleasant state (label 520d). Conversely, the further the object moves on the X-axis 50x in the direction of a positive value, the longer user U is in a pleasant state (label 520c). Furthermore, a threshold is set for the emotional valence and is displayed on the X-axis 50x. In this embodiment, the threshold includes an upper limit and a lower limit. The upper limit of the emotional valence is "+5" and is indicated by scale 510c. On the other hand, the lower limit of the emotional valence is "-5" and is indicated by scale 510d.

[0063] The Y-axis 50y is the vertical axis of the graph, and the position of the object on the Y-axis 50y represents the user U's level of alertness. In this embodiment, the further the object moves along the Y-axis 50y in the direction of a negative value, the longer the user U remains in an alert state (label 520b). Conversely, the further the object moves along the Y-axis 50y in the direction of a positive value, the longer the user U remains in an alert state (label 520a). Furthermore, a threshold value is set for the level of alertness and is displayed on the Y-axis 50y. In this embodiment, the threshold value includes an upper limit and a lower limit. The upper limit of the level of alertness is "+5" and is displayed by scale 510a. On the other hand, the lower limit of the level of alertness is "-5" and is displayed by scale 510b.

[0064] Furthermore, Graph 50 includes four quadrants 50a to 50d, separated by the X-axis 50x and the Y-axis 50y. The first quadrant 50a is the region where both emotional valence and arousal are positive. That is, if object 500 is drawn in the first quadrant 50a, it indicates that user U's psychological state is "Happy". The second quadrant 50b is the region where emotional valence is negative and arousal is positive. That is, if object 500 is drawn in the second quadrant 50b, it indicates that user U's psychological state is "Nervous". The third quadrant 50c is the region where both emotional valence and arousal are negative. That is, if object 500 is drawn in the third quadrant 50c, it indicates that user U's psychological state is "Sad / Tired". The fourth quadrant 50d is the region where emotional valence is positive and arousal is negative. In other words, if object 500 is drawn in the fourth quadrant 50d, it indicates that user U's psychological state is "Relaxed". Note that in this embodiment, the four quadrants 50a to 50d of graph 50 are described as including the cases where the labels "Joy," "Anger," "Sadness," and "Happiness" are assigned to them respectively, but this is not limited to this. The labels assigned to the quadrants of the graph may be determined based on the estimation target.

[0065] The cumulative score SC20 identified in activity A107 or A108 in Figure 6 is plotted on graph 50. The cumulative score SC20 includes the cumulative score SC20a for emotional valence and the cumulative score SC20b for arousal. More specifically, the cumulative score SC20 is plotted on graph 50 with the cumulative score SC20a for emotional valence as the X coordinate and the cumulative score SC20b for arousal as the Y coordinate. In Figure 7, object 500 is plotted at the coordinates (emotional valence, arousal) = (+3, +3). The cumulative score SC20 is continuously estimated, and each time the cumulative score SC20 is estimated, object 500 is plotted at the coordinates corresponding to the cumulative score SC20. Therefore, the results screen 5 is displayed, for example, in a manner in which object 500 is moving on graph 50.

[0066] Furthermore, if the position where object 500 is drawn in graph 50 moves from one quadrant to another, it is determined that the user U's psychological state has changed. In other words, user U can visually grasp this change in psychological state through the results screen 5. With this configuration, the user can easily grasp changes in their own psychological state. Furthermore, the user can grasp how their own psychological state changes over time. Details of the movement of object 500 on graph 50 are explained in Figure 8.

[0067] (Movement of objects on the graph) Figure 8 is a diagram illustrating the movement of objects on a graph. The explanation assumes that the display mode changes in the order of graphs 60a to 60c. That is, the display control unit 236 displays the object in a way that is visible to the user, so that the object moves from the coordinates of object 610a to the coordinates of object 610c on the graph. Note that graphs 60a to 60c have the same configuration as graph 50 in Figure 7, and a detailed description and explanation of Figure 8 are omitted.

[0068] In Figure 6, at the start of estimation in Activity A103, an object is drawn on the graph at the coordinate (emotional valence, arousal) = (0,0) (not shown). Furthermore, for the emotional valence index, if the estimation result in Activity A105 is "pleasant", the state score SC10 is set to "+1". On the other hand, if the estimation result is "unpleasant", the state score SC10 is set to "-1". Similarly, for arousal, if the estimation result is "aroused", the state score SC10 is set to "+1". On the other hand, if the estimation result is "not aroused", the state score SC10 is set to "-1".

[0069] Graph 60a shows object 610a drawn on the coordinates (emotional valence, arousal) = (+5, +5). That is, from the start of estimation in activity A103 in Figure 6, it is assumed that there have been at least 5 estimations of the state score SC10 where (emotional valence, arousal) = (pleasant, aroused). Furthermore, graph 60a shows that both emotional valence and arousal have reached the predetermined threshold (upper limit). Therefore, even if subsequent estimation results are (emotional valence, arousal) = (pleasant, aroused), the cumulative score SC20 is not updated, and the score of (emotional valence, arousal) = (+5, +5) is retained.

[0070] Graph 60b has a configuration in which object 610b is plotted at coordinates (emotional valence, arousal) = (+5, +3). This configuration indicates that the state score SC10 was estimated twice after the cumulative score SC20 was plotted on graph 60a. More specifically, it indicates that the state score SC10 was estimated to be (emotional valence, arousal) = (pleasant, unaroused) in both cases. Therefore, the cumulative score SC20a, which represents emotional valence, is not updated because it has reached the threshold. Only the cumulative score SC20b, which represents arousal, is updated.

[0071] Similarly, graph 60c has a configuration in which object 610c is plotted at coordinates (emotional valence, arousal) = (+5, 0). This configuration indicates that the state score SC10 was estimated three times after the cumulative score SC20 was plotted on graph 60b. More specifically, it indicates that the state score SC10 was estimated to be (emotional valence, arousal) = (pleasant, unaroused) each time. Therefore, the cumulative score SC20a, which represents emotional valence, is not updated because it has reached the threshold. Only the cumulative score SC20b, which represents arousal, is updated.

[0072] This section explains the change in arousal level from Graph 60a to Graph 60c. Starting from a stable arousal level of (+5) in Graph 60a, the quadrant in which the object is located changes only when the estimated result of (arousal level = non-arousal) is obtained at least five times consecutively, and this is considered to indicate a change in user U's psychological state.

[0073] In this configuration, even if transient changes occur in the estimated state due to variability in measurement results during the measurement of biological information IF10, the user can accurately grasp the changes in their own state without being significantly affected by these factors.

[0074] More specifically, even if a temporary change occurs in the stable cumulative score SC20, the object's position will not change to another quadrant. Therefore, it is possible to reduce the possibility of measurement errors or estimation noise being mistakenly attributed to changes in the user's state.

[0075] On the other hand, let's consider the case where no threshold is set for the cumulative score SC20. In such a case, if the user's state is stable, that is, if the same estimation results continue, the object's position will gradually move away from the origin. In this embodiment, even if there is a change in the user's psychological state, the object has moved away from the origin of the graph, and it will take multiple estimations to move to another quadrant, which may make it difficult to properly determine the change in the user's psychological state.

[0076] Considering the above, in this embodiment, by performing state estimation using the cumulative score SC20, the psychological state can be estimated without being significantly affected by transient changes in the state score SC10 due to noise during measurement or estimation. Furthermore, by setting a threshold for the cumulative score SC20, changes in the user's psychological state can be appropriately judged.

[0077] In this configuration, even when the estimation accuracy of the model (learning model LM) for estimating the user's state is insufficient, it is possible to avoid misclassification and provide stable estimation results.

[0078] (Other display examples) Figure 9 shows another example of how the estimated results of user U's psychological state are displayed. Both graphs 70a and 70b depict the trajectory of the cumulative score SC20 over time. In other words, the visual information IF30 (graph 70b) further includes a configuration in which multiple cumulative scores SC20 are plotted continuously according to the time series. With this configuration, the user can more appropriately grasp the changes in their psychological state over time.

[0079] The objects drawn in graphs 70a and 70b are explained below. Coordinates corresponding to objects with low brightness indicate that the object has passed over or stayed over those coordinates many times. For example, in graph 70a, the object has passed over or stayed over the coordinates corresponding to object 750a the most times, followed by the coordinates corresponding to objects 730a and 770a. Similarly, in graph 70b, the object has passed over or stayed over the coordinates corresponding to objects 710b and 720b the most times. Specific examples of "passing over" and "staying over" objects will be discussed later.

[0080] Furthermore, Figure 9 illustrates the difference in object trajectories between plotting on a graph using a conventional method and plotting on a graph using the cumulative score SC20 in this embodiment. Graph 70a shows the object trajectory plotted using a conventional method. On the other hand, graph 70b shows the object trajectory plotted in this embodiment. Note that both graphs 70a and 70b are graphs in which objects are plotted for the same estimation result. The estimation result is shown in estimation result table T1.

[0081] (Estimated Result Table T1) Estimated Results Table T1 contains the estimated results of user U's psychological state. Estimated Results Table T1 is an example of a database where the cumulative score SC20 is managed in activity A109 in Figure 6. Estimated Results Table T1 contains 14 sets of estimation results. In addition, Estimated Results Table T1 includes the estimated results of emotional valence T11 and arousal level T12.

[0082] In the emotional valence T11, if the estimated result is "pleasant," it is indicated as "P" in the table, and the state score SC10 is "+1." On the other hand, if the estimated result is "unpleasant," it is indicated as "N" in the table, and the state score SC10 is "-1."

[0083] Similarly, for Awakening Level T12, if the estimated result is "Awakened," it is indicated as "P" in the table, and the state score SC10 is "+1." On the other hand, if the estimated result is "Not Awakened," it is indicated as "N" in the table, and the state score SC10 is "-1."

[0084] First, let's explain graph 70a, in which objects are drawn on the graph using the conventional method. In the conventional method, objects are drawn based on the sum of the estimation results for a predetermined number of times, in the order in which they are newly estimated. As an example, let's explain the case where objects are drawn based on the sum of the estimation results for the five most recently estimated times. First, as the sum of No. 1 to No. 5, object 710a is drawn at the coordinates (emotional valence, arousal level) = (5, 5). Next, as the sum of No. 2 to No. 6, object 720a is drawn at the coordinates (emotional valence, arousal level) = (4, 4). Similarly, the calculation and drawing are repeated, and objects are drawn in the order of object 720a, (object 730a), (object 740a), object 750a, object 750a, ...

[0085] Here, objects 720a and 750a are coordinates calculated as the sum of the estimation results. These coordinates can be said to be the coordinates where the objects "stayed." On the other hand, the coordinates corresponding to objects 730a and 740a are not coordinates calculated as the sum of the estimation results. These coordinates can be said to be the coordinates where the drawn objects "passed through" when they moved to the next coordinate. In other words, objects 730a and 740a are drawn on the graph as the trajectories that the objects passed through.

[0086] According to conventional methods, the trajectory of the estimation result based on the estimation result table T1 has a configuration in which objects move along the coordinates of objects 750a, 760a, and 770a, which are placed around the origin. Thus, when the estimation result temporarily fluctuates after it has stabilized, conventional methods draw the object on coordinates close to the origin of the graph. In other words, the object moves along the graph in a manner that traverses the vicinity of the origin.

[0087] On the other hand, in this embodiment, graph 70b, on which objects are drawn, will be described. In this embodiment, objects are drawn based on the cumulative score SC20. In this embodiment, "drawing" an object on a predetermined coordinate is considered synonymous with the object "staying" at that coordinate.

[0088] First, based on the score of No. 1, object 750b is drawn at the coordinates (emotional valence, arousal level) = (1,1). Next, based on the cumulative scores SC20 of No. 1 and No. 2, object 740b is drawn at the coordinates (emotional valence, arousal level) = (2,2). Similarly, the calculation and drawing of the cumulative scores SC20 are repeated, and objects 730b, 720b, and 710b are drawn in that order. The cumulative scores SC20 from No. 6 onwards converge to around (emotional valence, arousal level) = (4,4) or (emotional valence, arousal level) = (5,5), and objects are drawn on these coordinates. In other words, objects move on the graph in a manner that alternates between the positions of objects 710b and 720b on graph 70b. Thus, in this embodiment, if the estimation results temporarily fluctuate after they have stabilized, objects are drawn on coordinates near the threshold. In other words, the object moves along the graph in a manner that oscillates around a threshold.

[0089] Thus, by using the cumulative score SC20 for object rendering in estimating user U's psychological state, once the estimation results stabilize, even if a temporary instability in the estimation results occurs afterward, it does not significantly affect the estimation of the psychological state. Furthermore, by setting thresholds for each indicator, changes in the psychological state can be appropriately determined.

[0090] 5. Others With respect to the information processing system 1 according to the above embodiment, the following configurations may be adopted.

[0091] (Method for calculating state score SC10) In the above embodiment, an example was described in which the state score SC10 is +1 or -1, but this is not limited to this. For example, the psychological state of user U estimated by the estimation unit 233 in response to the input of biometric information IF10 to the learning model LM may be quantified and output in the range of 0 to 1, and this numerical value may be used as the state score SC10. With this embodiment, the degree of user U's psychological state can be estimated in more detail.

[0092] (Estimation of User U's internal state) In the above embodiment, the psychological state of user U is estimated, and specifically, emotions such as joy, anger, sadness, and happiness are determined from two indicators: emotional valence and arousal level. However, this is not limited to this. For example, the information processing system 1 may estimate at least one of fatigue, drowsiness, stress, or mental health as the internal state of user U. The information processing system 1 may, for example, perform a process to present recommendations to user U regarding driving a vehicle or operating machinery based on the estimated fatigue and drowsiness levels of user U. Alternatively, the information processing system 1 may, for example, perform a process to present recommendations to the user regarding work based on the estimated stress and mental health levels. The estimation of user U's internal state and the method of utilizing the estimation results are not limited to these.

[0093] (Learning algorithm for the learning model LM) In the above embodiment, logistic regression was used as the specific algorithm for training the learning model LM, but this is not limited to this. For example, linear regression, random forest, boosting, support vector machines, neural networks, etc., may be used as algorithms.

[0094] In the above embodiment, the cumulative score SC20 is plotted on a coordinate space having two axes, i.e., the graph is two-dimensional; however, this is not limited to this case. There may be three axes corresponding to the index.

[0095] The information processing device 2 may be in an on-premise configuration or a cloud configuration. In the case of a cloud-based information processing device 2, for example, the above functions and processing may be provided in the form of SaaS (Software as a Service) or cloud computing.

[0096] In one embodiment, the reception unit 231, acquisition unit 232, estimation unit 233, update unit 234, determination unit 235, and display control unit 236 are described as functional units realized by the processor 23 of the information processing device 2. However, at least a part of these may be implemented as functional units realized by an external server (not shown), or as functional units realized by the processor 33 of the user terminal 3. Alternatively, they may be implemented as functional units realized by a processor (not shown) of the wearable device 4.

[0097] In the above embodiment, the information processing device 2 performed various storage and control functions, but instead of the information processing device 2, multiple external devices may be used. That is, various information and programs may be stored in a distributed manner across multiple external devices using blockchain technology or the like.

[0098] Furthermore, the following embodiments may also be provided.

[0099] (1) An information processing system comprising at least one processor, wherein the processor is configured to execute a program such that the following steps are performed: an acquisition step, which continuously acquires the user's biometric information; an update step, which estimates a state score that quantitatively indicates the user's health or psychological state based on the continuously acquired biometric information and a learning model, and updates a cumulative score obtained by continuously accumulating the estimated state scores; and a display control step, which displays visual information relating to the cumulative score, thereby enabling the user to continuously grasp the health or psychological state.

[0100] In this configuration, users can accurately grasp information indicating their health status, etc., without being significantly affected by temporary changes.

[0101] (2) In the system described in (1) above, in the update step, if the cumulative score exceeds a predetermined threshold when the state scores are accumulated, the system restricts the updating of the cumulative score.

[0102] In this configuration, users can appropriately grasp changes in their health or psychological state over time. More specifically, it is possible to avoid a situation where cumulative scores are biased towards one axis, making it difficult to reflect changes in other directions, and thus resolve the issues that arise from the accumulation of scores.

[0103] (3) The system described in (2) above, wherein the threshold can be changed according to the characteristics of the user.

[0104] In this configuration, users can more appropriately understand how their health or psychological state changes over time.

[0105] (4) The system described in (1) above, wherein the cumulative score has at least two indicators, and the visual information has a configuration in which the cumulative score is plotted on a coordinate space having axes corresponding to the indicators.

[0106] In this configuration, users can easily understand where their own health or psychological state falls within.

[0107] (5) The system described in (4) above, wherein the visual information further comprises a configuration in which a plurality of cumulative scores are plotted sequentially according to the time series.

[0108] In this configuration, users can more appropriately understand how their own health or psychological state changes over time.

[0109] (6) The system described in (1) above, wherein in the acquisition step, environmental information is further acquired, the environmental information is information relating to the environment surrounding the user, and the state score is estimated based on at least the environmental information, the biometric information, and the learning model.

[0110] In this configuration, information about the user's environment can also be reflected in the estimated user's health or psychological state. Therefore, the user's health and other related conditions can be estimated more accurately.

[0111] (7) An information processing method comprising each step of the information processing system described in any one of (1) to (6) above.

[0112] (8) A program that causes a computer to perform each step of the information processing system described in any one of (1) to (6) above. Of course, this is not always the case.

[0113] Finally, various embodiments of the present invention have been described, but these are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of symbols]

[0114] 1: Information Processing System 2: Information Processing Device 20: Communications bus 21: Communications Department 22: Storage section 23: Processor 231: Reception Department 232: Acquisition Department 233: Estimation section 234: Update section 235: Judgment section 236: Display Control Unit 3: User terminal 30: Communications bus 31: Communications Department 32: Storage section 33: Processor 34: Display section 35: Input section 4: Wearable devices 5:Result screen 50: Graph 50a: First quadrant 50b: Second quadrant 50c: Third quadrant 50d: Fourth quadrant 50x :X axis 50y:Y axis 500: Object 510a: Scale 510b: Scale 510c: scale 510d: Scale 520a: Label 520b: Label 520c: Label 520d: Label 60a: Graph 60b: Graph 60c: Graph 610a: Object 610b: Object 610c: Object 70a: Graph 70b: Graph 710a: Object 710b: Object 720a: Object 720b: Object 730: Object 730a: Object 730b: Object 740a: Object 740b: Object 750a: Object 750b: Object 760a: Object 8: Measurement terminal IF10: Biometric Information IF20: Environmental Information IF30: Visual Information LM: Learning Model LMa: Learning Model LMb: Learning Model SC10: State Score SC20: Cumulative Score SC20a: Cumulative score SC20b: Cumulative score T1: Estimated result table T11: Emotional Valence T12: Awakening Level U: User

Claims

1. An information processing system, Equipped with at least one processor, The processor is configured to execute a program such that the following steps are performed: In the acquisition step, the user's biometric information is continuously acquired. In the update step, Based on the continuously acquired biometric information and the learning model, a state score that quantitatively represents the user's health or psychological state is estimated, The cumulative score obtained by continuously accumulating the estimated state scores is updated, In the display control step, the system displays visual information relating to the cumulative score, thereby enabling the user to continuously understand their health or psychological state.

2. In the system described in claim 1, In the update step, if the cumulative score exceeds a predetermined threshold when the status scores are accumulated, the system restricts the updating of the cumulative score.

3. In the system described in claim 2, The threshold can be changed according to the user's characteristics in the system.

4. In the system described in claim 1, The cumulative score has at least two indicators, The system comprises a visual information in which the cumulative score is plotted on a coordinate space having axes corresponding to the indicators.

5. In the system described in claim 4, The system further comprises a configuration in which the visual information is plotted sequentially with a plurality of cumulative scores according to a time series.

6. In the system described in claim 1, In the aforementioned acquisition step, environmental information is further acquired. The aforementioned environmental information is information about the environment surrounding the user, The state score is estimated based on at least the environmental information, the biological information, and the learning model of the system.

7. Information processing method, A method comprising each step of the information processing system described in any one of claims 1 to 6.

8. It is a program, A program that causes a computer to perform each step of the information processing system described in any one of claims 1 to 6.

Images