Information processing method, information processing device, and program

Abstract

Provided are an information processing method, an information processing device, and a program that make it possible to improve estimation accuracy of the psychological state of a user who are executing a task. An information processing method according to the present disclosure involves: acquiring first biological information items indicating biological information of a user in a first period that indicates a period during which the user is executing a task; storing difficulty level information items indicating difficulty levels of the task in association with the respective first biological information items; extracting a first biological information item associated with a difficulty level information item having a low difficulty level as reference biological information indicating biological information serving as a reference; acquiring a second biological information item indicating biological information of the user in a second period that is after the first period and that indicates a period during which the user is executing the task; and determining the psychological state of the user on the basis of the reference biological information and the second biological information item.

Description

Information processing method, information processing apparatus and program

[0001] The present disclosure relates to an information processing method, an information processing apparatus, and a program.

[0002] For example, there is known a technique for estimating a user's psychological state based on the biological signal of the user (a driver in this example) while the user is performing a task such as driving a vehicle.

[0003] Japanese Unexamined Patent Application Publication No. 2009-213768

[0004] For example, while driving a vehicle (during an operation of running the vehicle), the user is in an awake state, which is different from a normal state or a state where the vehicle is stopped. Further, the degree of arousal during driving varies depending on the person, and fluctuations in the physiological state can be considered depending on the driving time zone. For this reason, it has been difficult to accurately estimate the psychological state of a user who is performing a task such as driving.

[0005] An object of the present disclosure is to provide an information processing method, an information processing apparatus, and a program that can improve the estimation accuracy of the psychological state of a user who is performing a task such as driving a vehicle, for example.

[0006] In order to achieve the above object, an information processing method of the present disclosure, which is an information processing method executed by an information processing apparatus, includes: acquiring first biological information indicating biological information of the user in a first period indicating a period during which the user is performing a task; associating difficulty information indicating the difficulty of the task with each piece of the first biological information and storing the same; extracting the first biological information associated with the difficulty information having low difficulty as reference biological information indicating reference biological information; acquiring second biological information indicating the biological information of the user in a second period, which is a period after the first period and indicates a period during which the user is performing the task; and determining the psychological state of the user based on the reference biological information and the second biological information.

[0007] According to the present disclosure, it is possible to improve the estimation accuracy of the psychological state of a user who is performing a task. Note that the effects described herein are not necessarily limited, and may be any effect described in the present specification.

[0008] Figure 1 shows an example of the schematic configuration of the information processing system of the first embodiment. Figure 2 shows an example of the functions of the information processing device of the first embodiment. Figure 3 shows an example of linked registration information of the first embodiment. Figure 4 shows an example of an operating status report. Figure 5 shows an example of an operating status report. Figure 6 is a flowchart showing an example of the operation of the information processing device of the first embodiment. Figure 7 shows an example of the schematic configuration of the information processing system of the second embodiment. Figure 8 shows an example of the functions of the information processing device of the second embodiment. Figure 9 shows an example of a psychological state report. Figure 10 is a flowchart showing an example of the operation of the information processing device of the second embodiment. Figure 11 shows an example of the schematic configuration of the information processing system of the third embodiment. Figure 12 shows an example of the functions of the information processing device of the third embodiment. Figure 13 shows an example of score correspondence information of the third embodiment. Figure 14 is a flowchart showing an example of the operation of the information processing device of the third embodiment. Figure 15 is a flowchart showing an example of the operation of a modified information processing device.

[0009] The information processing method, information processing apparatus, and program according to the embodiments of this disclosure will be described in detail below with reference to the attached drawings.

[0010] (First Embodiment) Figure 1 is a diagram showing an example of the schematic configuration of the information processing system 1 according to this embodiment. The information processing system 1 of this embodiment is configured for the vehicle interior space and is a system that estimates and outputs the psychological state of the user based on the user's biometric information while the user is driving the vehicle (an example of a task). Note that "driving" in this disclosure refers to operations when the vehicle is moving and does not refer to operations when the vehicle is stopped. The specific details will be explained below.

[0011] As shown in Figure 1, the information processing system 1 includes a vehicle system 10 representing the vehicle-side system, a server 20, a communication module 30, an output device 40, and an output device 50. The vehicle system 10 is connected to the server 20 and the output device 40 via the communication module 30. The server 20 is also connected to the output device 50. Each of the output devices 40 and 50 may be a terminal device such as a smartphone, tablet, or PC (Personal Computer).

[0012] First, the configuration of the vehicle system 10 will be described. As shown in Figure 1, the vehicle system 10 includes a biosensor 11, a vehicle sensor 12, a GPS 13, a storage device 14, an information processing device 15, and an output device 16. Note that the hardware configuration of the vehicle system 10 is not limited to the example in Figure 1, and may include hardware elements other than those illustrated in Figure 1.

[0013] The biosensor 11 is a sensor that detects the user's biometric information. For example, the biometric information may be at least one of the following: information related to heart rate (e.g., pulse rate or heart rate), the user's degree of sweating, respiratory rate, pupil dilation, blinking frequency, blood pressure, skin temperature, and information related to brain waves. For example, the biosensor 11 may generate and output biometric information based on the detection results of the device itself, or the information processing device 15 may generate biometric information based on an electrical signal indicating the detection results of the biosensor 11.

[0014] For example, the biosensor 11 may be a photodetector that detects light from the user. It may also be equipped with a light source, and the photodetector may detect reflected light caused by light irradiated from the light source onto the user (the measurement wavelength may be visible light or near-infrared light). For example, the biosensor 11 may be a camera or smartwatch that includes an image sensor. Alternatively, for example, the biosensor 11 may be an electrocardiogram measuring device equipped with electrodes placed on the skin surface of the user's chest. The biosensor 11 may be composed of sensors depending on the object to be measured.

[0015] The vehicle sensor 12 is a sensor that detects various types of information related to the vehicle. For example, the vehicle sensor 12 may include a camera that takes images of the vehicle's direction of travel (forward or backward), a gyro sensor that detects the vehicle's movement (rotation), or an acceleration sensor that detects the vehicle's acceleration.

[0016] GPS13 is a device for obtaining the vehicle's current position. GPS13 receives GPS signals, which are radio waves transmitted from each of multiple GPS satellites, and can determine the vehicle's current position through three-dimensional positioning based on the received GPS signals.

[0017] The storage device 14 is a device for storing various types of information. For example, the storage device 14 can store programs executed by the information processing device 15, information indicating the results of processing by the information processing device 15, and so on.

[0018] The information processing device 15 is a device that determines the psychological state of a user based on the user's biometric information detected by the biosensor 11, information detected by the vehicle sensor 12, information acquired by GPS 13, etc. For example, the information processing device 15 may be a vehicle control unit (ECU) or a processing device including a processor.

[0019] Figure 2 shows an example of the functions of the information processing device 15. As shown in Figure 2, the information processing device 15 includes a biometric information acquisition unit 101, a state determination unit 102, a driving environment information acquisition unit 103, a difficulty determination unit 104, a linking unit 105, an extraction unit 106, an update unit 107, and an output control unit 108. In the example of Figure 5, only the functions necessary for explaining the main parts of this embodiment are illustrated, but the functions of the information processing device 15 are not limited to these. In this embodiment, the information processing device 15 realizes the functions of each of the above parts by executing a program stored in the storage device 14. However, it is not limited to this, and some or all of these functions may be realized by dedicated hardware circuits (semiconductor integrated circuits, etc.).

[0020] The biometric information acquisition unit 101 acquires the user's biometric information detected by the biosensor 11.

[0021] The state determination unit 102 determines the user's psychological state based on the biometric information acquired by the biometric information acquisition unit 101 and a reference value (threshold). For example, if the value of the biometric information acquired by the biometric information acquisition unit 101 is less than the reference value, the user's psychological state is determined to be "normal (neutral)," and if it is above the reference value, it can be determined to be in an "anxious" or "stressed (tense)" state. The method for determining the reference value will be described later.

[0022] The driving environment information acquisition unit 103 acquires driving environment information that indicates information about the vehicle's driving environment. More specifically, the driving environment information acquisition unit 103 acquires information detected by the vehicle sensor 12 and information acquired by the GPS 13, etc.

[0023] The difficulty level determination unit 104 determines the difficulty level of driving the vehicle by the user. For example, the difficulty level of driving the vehicle is determined based on information including the vehicle's speed and the direction in which the vehicle is traveling, or on images captured by a camera installed in the vehicle that captures the direction in which the vehicle is traveling. For example, the difficulty level determination unit 104 can determine the difficulty level of driving the vehicle based on driving environment information acquired by the driving environment information acquisition unit 103.

[0024] For example, the difficulty determination unit 104 can determine the vehicle's speed and direction from the time-dependent changes in information indicating the vehicle's current position based on GPS signals (longitude, latitude, etc.) included in the driving environment information. If the speed and direction are approximately constant, it can determine that the vehicle is driving at a constant speed on a straight road, and thus determine that the driving difficulty is low. There may be gradual acceleration and deceleration along the way for waiting at traffic lights or adjusting speed. However, if the vehicle is driving slowly, it may be driving on a narrow road or in an area where driving is difficult. Therefore, for example, a speed of 30 km / h or higher can be set as a condition for low-difficulty driving.

[0025] Furthermore, the difficulty determination unit 104 can also determine the difficulty of driving using a method such as that described in Japanese Patent No. 6593712. For example, the difficulty determination unit 104 can obtain evaluation information by identifying information about the road structure outside the vehicle (such as lane width), the presence, number, and type of traffic participants from images captured by a camera that images the direction of travel of the vehicle, which are included in the driving environment information. Based on the obtained evaluation information and correspondence information (for example, information in a table format) to which the difficulty level is pre-associated for each piece of evaluation information, the unit can determine the difficulty level corresponding to the obtained evaluation information.

[0026] The linking unit 105 links each piece of biometric information to difficulty information indicating the difficulty level determined by the difficulty level determination unit 104 and stores it in the storage device 14. In this embodiment, the linking unit 105 links biometric information acquired by the biometric information acquisition unit 101, difficulty information indicating the difficulty level determined by the difficulty level determination unit 104, and psychological state information indicating the user's psychological state determined by the state determination unit 102, and stores them in the storage device 14. In this embodiment, each time biometric information is acquired by the biometric information acquisition unit 101 while the user is driving the vehicle, the linking unit 105 links the biometric information to difficulty information indicating the difficulty level of driving the vehicle at that time, and psychological state information indicating the user's psychological state determined based on the biometric information, and stores them in the storage device 14. In the following description, the group of information linked to the linking unit 105 and stored in the storage device 14 may be referred to as "linked registration information".

[0027] Figure 3 shows an example of linked registration information. In the example in Figure 3, one combination of a driver ID that identifies the user (driver), time, biometric information (heart rate is assumed in the example in Figure 3), vehicle speed, location information (latitude and longitude) indicating the vehicle's position, difficulty information, reference value, and psychological state information is considered as one unit of information, and linked registration information can be thought of as being composed of multiple units of information. As will be described later, the reference value is updated by the update unit 107. Details of the update will be described later.

[0028] Returning to Figure 2, the explanation continues. The extraction unit 106 refers to the linked registration information and extracts the biological information linked to the difficulty information indicating that the difficulty of operation is low, as reference biological information that represents the standard biological information.

[0029] The update unit 107 updates the reference value when the number of extracted reference biological information items exceeds a predetermined number. For example, the update unit 107 may set the average value of the predetermined number of extracted reference biological information items as the new reference value, or it may set the value obtained by adding a buffer (for example, 10% to 20% of the average value) to the average value as the new reference value. Alternatively, the update unit 107 may set the average value of the reference biological information items over a recent period (not limited to a period corresponding to the predetermined number mentioned above), or the value obtained by adding the buffer to the average value, as the new reference value.

[0030] Furthermore, the system may be configured to include multiple reference values. For example, it may include a first reference value for determining an abnormal state of the user and a second reference value (< first reference value) for determining a relaxed state. For example, the update unit 107 can set a new first reference value by adding a buffer (for example, 10% to 20% of the average value) to the average value of a predetermined number of extracted reference biological information, and set a new second reference value by subtracting the buffer from the average value.

[0031] If the reference value is updated by the update unit 107, the state determination unit 102 uses the updated reference value to determine the user's psychological state. In the initial stage, when the reference value has not been updated even once, the state determination unit 102 uses a predetermined default reference value (hereinafter sometimes referred to as the "standard reference value") to determine the user's psychological state.

[0032] The output control unit 108 controls the output of various types of information. For example, the output control unit 108 controls the output of information based on psychological state information indicating the user's psychological state as determined by the state determination unit 102. In this embodiment, the output control unit 108 transmits the above-mentioned linked registration information to the server 20 via the communication module 30 (an example of output) each time the vehicle operation is completed or after a certain period of time has elapsed. The output control unit 108 also generates information indicating the psychological state of the user while driving the vehicle (hereinafter referred to as the "driving state report") based on the above-mentioned linked registration information each time the vehicle operation is completed or after a certain period of time has elapsed, and outputs the generated driving state report from the output device 16. The output device 16 is composed of a display device such as a liquid crystal display. Furthermore, the output control unit 108 transmits the generated driving state report to the server 20 or the output device 40 via the communication module 30. The server 20, upon receiving the driving state report, transmits the received driving state report to the output device 50, which can then output it. Furthermore, the output device 40, which receives the operating status report, can output the received operating status report. Here, the operating status report can be considered as an example of information based on psychological state information that indicates the user's psychological state as determined by the status determination unit 102.

[0033] For example, the driving status report may be an image in which route information showing the vehicle's traveled is superimposed with information showing the changes in the user's psychological state. For example, if the user's psychological state information has three states, such as "relaxed," "normal," and "stressed," these may be displayed in a color-coded format. Alternatively, as shown in Figure 4, for example, the route information may be superimposed with information showing the changes in difficulty level. Also, for example, if the difficulty level information has three states, such as "Easy" indicating low difficulty, "Normal" indicating average difficulty, and "Hard" indicating high difficulty, these may be displayed in a color-coded format, as shown in Figure 4 (A). The driving status report in this example is an image in which route information is superimposed with information showing the changes in the user's psychological state and information showing the changes in difficulty level, but it is not limited to this. For example, the driving status report may be an image in which only information showing the changes in the user's psychological state is superimposed with route information. Alternatively, for example, the driving status report may be an image in which only information showing the changes in difficulty level is superimposed with route information. Alternatively, as shown in Figure 4(B), for example, the changes in biological information may be displayed in conjunction with the changes in difficulty information, showing what psychological states correspond to the levels of biological information.

[0034] Furthermore, as shown in Figure 5, for example, the driving status report may be an image in which a pie chart representing the proportion of each of several types of psychological state information set as user psychological state information and a pie chart representing the proportion of each of several levels of difficulty information set as difficulty level information coexist. In the above case, the "output" of the driving status report is in the form of displaying an image of the driving status report on a display unit (display) such as the output device 16. The above is a description of the functions of the information processing device 15.

[0035] Based on the above description, the information processing device 15 of this embodiment acquires first biometric information indicating the user's biometric information during a first period indicating the period during which the user is performing driving (an example of a task), stores difficulty information indicating the difficulty of driving associated with each piece of first biometric information, and extracts the first biometric information associated with the difficulty information with the lowest difficulty as reference biometric information indicating the reference biometric information.Then, the information processing device 15 acquires second biometric information indicating the user's biometric information during a second period indicating a period after the first period during which the user is performing driving (a task in the same phase as the task in the first period, and in this case, operations when the vehicle is stopped are not included), and can be considered to be configured to determine the user's psychological state based on the reference biometric information and the second biometric information.

[0036] Furthermore, the information processing device 15 can be considered to be configured to determine the user's psychological state based on a reference value that is updated according to the extraction status of reference biological information and the second set of biological information. The second period corresponds to the period during which the biological information used to determine the psychological state was acquired, and the first period corresponds to the period prior to that (including the period during which the previous psychological state determination was made).

[0037] Next, the functions of the server 20 will be described. The server 20 can receive and store the above-mentioned linked registration information from the information processing device 15 in response to a request from the information processing device 15, and can also send previously stored linked registration information to the information processing device 15. In addition, the server 20 can receive the above-mentioned operating status report from the information processing device 15 and send the received operating status report to the output device 50 (to output it from the output device 50). For example, the server 20 may generate the above-mentioned operating status report based on the linked registration information received from the information processing device 15, and control the output of the generated operating status report from each unit (output device 16, output device 40, output device 50, etc.).

[0038] Next, an example of the operation of the information processing device 15 in this embodiment will be described. Figure 6 is a flowchart showing an example of the operation of the information processing device 15 while the user is driving the vehicle. Note that the order of processing in each step described below can be changed arbitrarily and is not limited to the example in Figure 6. In addition, in the specific details of each process described below, explanations will be omitted as appropriate where they overlap with the explanations of the parts described above.

[0039] As shown in Figure 6, first, the biometric information acquisition unit 101 acquires the user's biometric information (step S1). Next, the state determination unit 102 determines the user's psychological state based on the biometric information acquired in step S1 and the reference value (step S2). In the initial stage, when the reference value has not been updated even once, the state determination unit 102 determines the user's psychological state using the standard reference value described above. If an update has been performed, the user's psychological state is determined using the latest updated reference value.

[0040] Next, the driving environment information acquisition unit 103 acquires driving environment information (step S3). Then, the difficulty level determination unit 104 determines the difficulty level of driving the vehicle by the user based on the driving environment information acquired in step S3 (step S4).

[0041] Next, the linking unit 105 links and registers the biometric information acquired in step S1, the psychological state information indicating the user's psychological state determined in step S2, and the difficulty level information indicating the difficulty level determined in step S4 (step S5). More specifically, the linking unit 105 generates the aforementioned unit information by linking the biometric information acquired in step S1, the psychological state information indicating the user's psychological state determined in step S2, and the difficulty level information indicating the difficulty level determined in step S4, and adds the generated unit information to the aforementioned linked registration information. The information processing device 15 can also acquire past linked registration information from the server 20 in advance, such as at the timing immediately before the start of vehicle operation. In this case, the latest updated reference value may be used as the reference value immediately after the start of operation.

[0042] Next, the extraction unit 106 refers to the association registration information after step S5, and extracts the above reference biological information in the most recent predetermined period. The updating unit 107 determines whether or not the number of pieces of reference biological information extracted by the extraction unit 106 is equal to or greater than a predetermined number (step S6). When the result of step S6 is affirmative (step S6: Yes), the updating unit 107 updates the reference value (step S7). On the other hand, when the result of step S6 is negative (step S6: No), the updating unit 107 does not perform the update, and the current reference value is maintained (step S8).

[0043] After step S7 or step S8, the information processing apparatus 15 determines whether or not the user's driving of the vehicle has ended (step S9). When the result of step S9 is negative (step S9: No), the information processing apparatus 15 repeats the processing from step S1 onward. When the result of step S9 is affirmative (step S9: Yes), the processing ends.

[0044] After the above processing is completed, as described above, the information processing apparatus 15 transmits the above association registration information to the server 20 via the communication module 30. The information processing apparatus 15 also generates the above driving state report based on the association registration information, and outputs the generated driving state report from the output device 16. Further, an output control unit 108 transmits the generated driving state report to the server 20 or the output device 40 via the communication module 30.

[0045] (Operation and effect) As described above, the information processing apparatus 15 of the present embodiment determines the psychological state of the user during driving using a reference value based on biological information (reference biological information) of the user who is performing driving with low difficulty. While characteristics including the degree of arousal during driving differ depending on the user, in the present embodiment, the estimation accuracy of the user's psychological state during driving can be improved by using the reference value that reflects the user's characteristics.

[0046] Also, as described above, the information processing apparatus 15 of the present embodiment sequentially updates the reference value in accordance with the extraction status of the reference biological information, so that as the accumulation of biological information progresses, the reference value that more reflects the user's characteristics can be set.

[0047] Hereinafter, a comparison between the conventional technology and the present embodiment will be described. In conventional systems where a computer determines a user's state using biological information, there has been a technical problem that the determination reference value becomes inaccurate due to individual differences and / or situational changes (e.g., proficiency in driving tasks, changes in physical condition), and the reliability of the system decreases over time. If a reference value with low reliability is used, erroneous determinations of the user's state frequently occur, resulting in unnecessary recalculation processes and alerts. This has led to a problem that the efficiency of data processing deteriorates, and computer resources (memory, CPU, etc.) that are otherwise unnecessary are excessively consumed.

[0048] In contrast, according to the present embodiment, a processor dynamically extracts highly reliable biological information acquired in "a situation where the task difficulty is low", and uses this as "reference biological information" adapted to an individual or a situation. This generates a reference value that serves as a foundation for highly reliable data processing and can adapt to the user's state that fluctuates depending on the situation, thereby suppressing frequent occurrences of erroneous determinations of the user's state. Therefore, compared to conventional systems that use fixed and uniform reference values, the present disclosure can reduce recalculation processes caused by erroneous determinations, and reduce CPU load and memory usage. Note that the above-described effects can also be similarly achieved in each of the embodiments described later.

[0049] (Modification 1 of the First Embodiment) For example, all or part of the functions included in the information processing apparatus 15 described above may be implemented by being mounted on the server 20. That is, all of the functions shown in Fig. 2 may be provided in the server 20 instead of the information processing apparatus 15, or the functions shown in Fig. 2 may be distributed and mounted between the information processing apparatus 15 and the server 20.

[0050] (Modification 2 of the First Embodiment) In the first embodiment described above, the information processing device 15 registers information obtained at the corresponding time for each of the reference values ​​and psychological state information included in the linked registration information described above. However, it is not limited to this, and for example, the information processing device 15 may update each of the reference values ​​and psychological state information included in the linked registration information described above based on the latest reference values ​​each time the vehicle operation is completed or after a certain period of time has elapsed. For example, all reference values ​​included in the linked registration information described above may be updated to the latest reference values, and all psychological state information may be updated to the judgment result using the latest reference values. Then, the driving state report described above may be generated and output based on the updated linked registration information.

[0051] Furthermore, for example, the information processing device 15 may store pairs of biometric information and difficulty information without determining the user's psychological state while the user is driving or until a certain period of time has elapsed. After the vehicle has been driven or a certain period of time has elapsed, it may extract one or more biometric pieces of information linked to difficulty information indicating a low driving difficulty from the accumulated information set, and determine a reference value based on the one or more reference biometric pieces of information extracted. The method for determining the reference value may be the same as the method for updating the reference value described above. For example, the average value of the extracted reference biometric pieces of information may be set as the reference value, or a value obtained by adding a buffer (for example, 10% to 20% of the average value) to the average value may be set as the reference value. The information processing device 15 may then use the determined reference value to determine the user's psychological state at the time each piece of biometric information was acquired (determined retrospectively).

[0052] In other words, the information processing device 15 may acquire first biometric information indicating the user's biometric information during a first period indicating the period during which the user is performing an operation (an example of a task), associate and store difficulty information indicating the difficulty of the operation with each piece of first biometric information, extract first biometric information associated with low difficulty information as reference biometric information indicating a standard biometric information, and determine the user's psychological state during the first period based on the reference biometric information and the first biometric information. Combining with the first embodiment described above, the information processing device 15 can be configured to perform at least one of the following: acquire first biometric information indicating the user's biometric information during a first period, associate and store difficulty information with each piece of first biometric information, extract first biometric information associated with low difficulty information as reference biometric information indicating a standard biometric information, determine the user's psychological state during the first period based on the reference biometric information and the first biometric information; and acquire second biometric information indicating the user's biometric information during a second period, which is a period after the first period during which the user is performing a task, and determine the user's psychological state during the second period based on the reference biometric information and the second biometric information.

[0053] (Second Embodiment) Next, a second embodiment will be described. Parts common to the first embodiment described above will be omitted as appropriate. Figure 7 shows an example of the schematic configuration of the information processing system 2 according to this embodiment. The information processing system 2 of this embodiment is configured for office spaces and is a system that estimates and outputs the psychological state of a user based on the user's biometric information while the user is performing work (an example of a task) that includes at least one of business and learning using a PC (terminal). The specific details will be described below.

[0054] As shown in Figure 7, the information processing system 2 includes a biosensor 11, a PC 200, a server 20, a communication module 30, an output device 40, and an output device 50. The PC 200 is connected to the server 20 and the output device 40 via the communication module 30. The configuration other than the PC 200 is the same as that of the first embodiment described above, so a detailed explanation is omitted.

[0055] The configuration of the PC 200, which corresponds to the "terminal," will now be described. As shown in Figure 7, the PC 200 includes a PC input device 210, a storage device 220, an information processing device 230, and an output device 240. Note that the hardware configuration of the PC 200 is not limited to the example in Figure 7, and may include hardware elements other than those illustrated in Figure 7.

[0056] The PC input device 210 is a device operated by the user, and it inputs information (operation information) corresponding to the user's actions to the information processing device 230. For example, the PC input device 210 may be a keyboard or a mouse.

[0057] The storage device 220 is a device for storing various types of information. For example, the storage device 220 can store programs executed by the information processing device 230, information indicating the results of processing by the information processing device 230, and so on.

[0058] The information processing device 230 is a device that determines the psychological state of a user based on the user's biometric information detected by the biosensor 11, operation information entered by the PC input device 210, etc. For example, the information processing device 230 may be a processing device that includes a processor.

[0059] Figure 8 shows an example of the functions of the information processing device 230. As shown in Figure 8, the information processing device 230 includes a biometric information acquisition unit 231, a state determination unit 232, an operation information acquisition unit 233, a difficulty determination unit 234, a linking unit 235, an extraction unit 236, an update unit 237, and an output control unit 238. In the example in Figure 8, only the functions necessary for explaining the main parts of this embodiment are illustrated, but the functions of the information processing device 230 are not limited to these. In this embodiment, the information processing device 230 realizes the functions of each of the above parts by executing a program stored in the storage device 220. However, it is not limited to this, and some or all of these functions may be realized by dedicated hardware circuits (semiconductor integrated circuits, etc.).

[0060] The biometric information acquisition unit 231 acquires the user's biometric information detected by the biosensor 11.

[0061] The state determination unit 232 determines the user's psychological state based on the biological information acquired by the biological information acquisition unit 231 and a reference value. This is as described in the first embodiment above.

[0062] The operation information acquisition unit 233 acquires operation information input from the PC input device 210.

[0063] The difficulty determination unit 234 determines the difficulty level of a user's work using the PC 200 based on the operation information acquired by the operation information acquisition unit 233. For example, the difficulty level of a user's work using the PC 200 is determined according to the amount of operation on the PC 200, with the difficulty level increasing as the amount of operation increases. For example, a table-format information set may be prepared in which operation content, which is determined according to the combination of operation type and amount of operation, and difficulty information indicating the difficulty level are pre-associated. The operation content may be identified from the operation information acquired by the operation information acquisition unit 233, and difficulty information corresponding to the identified operation content may be selected. For example, for operation content where the amount of typing over a certain period is less than a threshold, difficulty information indicating a low difficulty level may be associated. Similarly, for operation content where the amount of mouse movement over a certain period is less than a threshold, difficulty information indicating a low difficulty level may be associated. Furthermore, for operation content indicating viewing a PDF or web page, difficulty information indicating a low difficulty level may be associated.

[0064] Similar to the first embodiment described above, the linking unit 235 links difficulty information indicating the difficulty level determined by the difficulty level determination unit 234 to each piece of biometric information and stores it in the storage device 14. More specifically, in this embodiment, whenever biometric information is acquired by the biometric information acquisition unit 231 while the user is performing work using the PC 200, the linking unit 235 links the biometric information, difficulty information indicating the difficulty level of the work at that time, and psychological state information indicating the user's psychological state determined based on the biometric information to generate the aforementioned unit information, and adds the generated unit information to the linking registration information already stored in the storage device 220 to update the linking registration information.

[0065] The extraction unit 236 refers to the linked registration information and extracts biological information linked to difficulty information indicating a low difficulty level for the task, as reference biological information that represents the standard biological information.

[0066] The update unit 237 updates the reference value when the number of extracted reference biological information items exceeds a predetermined number. The update method is the same as described in the first embodiment above.

[0067] If the reference value is updated by the update unit 237, the state determination unit 232 uses the updated reference value to determine the user's psychological state. If the reference value has not been updated, the state determination unit 232 uses the standard reference value described above to determine the user's psychological state.

[0068] The output control unit 238 controls the output of various types of information. For example, the output control unit 238 controls the output of information based on psychological state information, which indicates the user's psychological state as determined by the state determination unit 232. In this embodiment, the output control unit 238 transmits the above-mentioned linked registration information to the server 20 via the communication module 30 each time the user completes a task or after a certain period of time has elapsed. The output control unit 238 also generates information indicating the user's psychological state while performing the task (hereinafter referred to as a "psychological state report") based on the above-mentioned linked registration information each time the user completes a task or after a certain period of time has elapsed, and outputs the generated psychological state report from the output device 240. The output device 240 is composed of a display device such as a liquid crystal display. Furthermore, the output control unit 238 transmits the generated psychological state report to the server 20 or the output device 40 via the communication module 30. The server 20, upon receiving the psychological state report, transmits the received psychological state report to the output device 50, which can then output it. Furthermore, the output device 40, which receives the psychological state report, can output the received psychological state report. Here, the psychological state report can be considered an example of information based on psychological state information that indicates the user's psychological state as determined by the state determination unit 232.

[0069] For example, the psychological state report may be an image as shown in Figure 9. For example, as in Figure 9(A), the psychological state report may display the changes in biometric information while showing how the high and low levels of biometric information correspond to different psychological states, along with the changes in difficulty level information. Alternatively, as in Figure 9(B), the psychological state report may display a pie chart showing the proportion of each of the multiple types of psychological state information set as the user's psychological state information, and a pie chart showing the proportion of each of the multiple levels of difficulty information set as difficulty level information, side by side.

[0070] In the above case, the "output" of the psychological state report is in the form of displaying an image of the psychological state report on a display unit (display) such as the output device 240. This concludes the explanation of the functions of the information processing device 230.

[0071] Based on the above description, the information processing device 230 of this embodiment acquires first biometric information indicating the user's biometric information during a first period in which the user is performing work (an example of a task) that includes at least one of business and learning using the PC 200. For each piece of first biometric information, difficulty information indicating the difficulty level of the work is associated with and stored. The first biometric information associated with the lowest difficulty level is extracted as reference biometric information indicating the reference biometric information. The information processing device 15 then acquires second biometric information indicating the user's biometric information during a second period in which the user is performing work (a task in the same phase as the task in the first period, and at least a state in which the PC 200 is being operated). Based on the reference biometric information and the second biometric information, the device can be considered to be configured to determine the user's psychological state.

[0072] Furthermore, the information processing device 230 can be considered to be configured to determine the user's psychological state based on a reference value that is updated according to the extraction status of the reference biological information, and second biological information. A specific example of the operation of the information processing device 230 will be described later.

[0073] The functions of server 20 are basically the same as those described in the first embodiment above. For example, server 20 may generate the psychological state report based on the linked registration information received from information processing device 230, and control the output of the generated psychological state report from each unit (output device 240, output device 40, output device 50, etc.).

[0074] Next, an example of the operation of the information processing device 230 in this embodiment will be described. Figure 10 is a flowchart showing an example of the operation of the information processing device 230 while a user is performing a task. Note that the order of processing in each step described below can be changed arbitrarily and is not limited to the example in Figure 10. Also, in the specific details of each process described below, explanations will be omitted as appropriate where they overlap with the explanations of the parts described above.

[0075] As shown in Figure 10, first, the biometric information acquisition unit 231 acquires the user's biometric information (step S11). Next, the state determination unit 232 determines the user's psychological state based on the biometric information acquired in step S11 and a reference value (step S12).

[0076] Next, the operation information acquisition unit 233 acquires operation information (step S13). Then, the difficulty level determination unit 234 determines the difficulty level of the user's work based on the operation information acquired in step S13 (step S14). For example, the difficulty level determination unit 234 can identify the operation content based on the operation information acquired in step S13 and operation information over a certain period prior to that, and select difficulty level information corresponding to the identified operation content.

[0077] Next, the linking unit 235 links and registers the biometric information acquired in step S11, the psychological state information indicating the user's psychological state determined in step S12, and the difficulty level information indicating the difficulty level determined in step S14 (step S15).

[0078] Next, the extraction unit 235 refers to the linked registration information after step S15 and extracts the above-mentioned reference biological information for the most recent period, and the update unit 237 determines whether the number of reference biological information extracted by the extraction unit 235 is greater than or equal to a predetermined number (step S16). If the result of step S16 is positive (step S16: Yes), the update unit 237 updates the reference value (step S17). On the other hand, if the result of step S16 is negative (step S16: No), the update by the update unit 237 is not performed, and the current reference value is maintained (step S18).

[0079] After step S17 or step S18, the information processing device 230 determines whether the user's work has been completed (step S19). If the result of step S19 is negative (step S19: No), the information processing device 230 repeats the processing from step S11 onwards. If the result of step S19 is positive (step S19: Yes), the processing ends.

[0080] After the above processing is completed, as described above, the information processing device 230 transmits the above-mentioned linked registration information to the server 20 via the communication module 30. The information processing device 230 also generates the above-mentioned psychological state report based on the above-mentioned linked registration information and outputs the generated psychological state report from the output device 240. Furthermore, the output control unit 238 transmits the generated psychological state report to the server 20 or the output device 40 via the communication module 30.

[0081] (Function and Effects) As described above, the information processing device 230 of this embodiment determines the psychological state of a user during work using reference values ​​based on the user's biometric information (reference biometric information) while the user is performing a low-difficulty task (a task that includes at least one of work using the PC 200 and learning). Since the characteristics of users, including their level of alertness during work, differ from user to user, this embodiment improves the accuracy of estimating the user's psychological state during work by using reference values ​​that reflect the user's characteristics. Furthermore, similar to the first embodiment described above, the information processing device 230 of this embodiment sequentially updates the reference values ​​according to the extraction status of reference biometric information, so as the accumulation of reference biometric information progresses, it becomes possible to set reference values ​​that better reflect the user's characteristics. Therefore, it is possible to further improve the accuracy of estimating the user's psychological state during work.

[0082] (Modification 1 of the second embodiment) For example, all or part of the functions of the information processing device 230 described above may be installed on the server 20. In other words, all of the functions shown in Figure 8 may be provided by the server 20 instead of the information processing device 230, or the functions shown in Figure 8 may be distributed and installed on both the information processing device 230 and the server 20.

[0083] (Modification 2 of the second embodiment) In the second embodiment described above, the information processing device 230 registers information obtained at the corresponding time for each of the reference values ​​and psychological state information included in the linked registration information described above. However, it is not limited to this, and for example, the information processing device 230 may update each of the reference values ​​and psychological state information included in the linked registration information described above based on the latest reference values ​​each time the user's work is completed or after a certain period of time has elapsed. For example, all reference values ​​included in the linked registration information described above may be updated to the latest reference values, and all psychological state information may be updated to the judgment result using the latest reference values. Then, the psychological state report described above may be generated and output based on the updated linked registration information.

[0084] Alternatively, for example, the information processing device 230 may store pairs of biometric information and difficulty information without determining the user's psychological state while the user is performing work using the PC 200 or until a certain period of time has elapsed. After the user finishes their work or a certain period of time has elapsed, the information processing device 230 may extract one or more biometric pieces of information linked to difficulty information indicating that the work is easy from the previously stored information, and determine a reference value based on the extracted one or more reference biometric pieces of information. The method for determining the reference value may be the same as the method for updating the reference value described above. The information processing device 230 may then use the determined reference value to collectively determine the user's psychological state at the time each piece of biometric information was acquired.

[0085] In other words, the information processing device 230 may acquire first biometric information indicating the user's biometric information during a first period indicating the period during which the user is performing work (an example of a task) using the PC 200, store difficulty information indicating the difficulty level of the work associated with each piece of first biometric information, extract first biometric information associated with low difficulty information as reference biometric information indicating a standard biometric information, and determine the user's psychological state during the first period based on the reference biometric information and the first biometric information. In combination with the second embodiment described above, the information processing device 230 can be configured to perform at least one of the following: acquire first biometric information indicating the user's biometric information during a first period; associate and store difficulty information with each piece of first biometric information; extract first biometric information associated with low difficulty information as reference biometric information indicating a standard biometric information; determine the user's psychological state during the first period based on the reference biometric information and the first biometric information; and acquire second biometric information indicating the user's biometric information during a second period, which is a period after the first period during which the user is performing a task; and determine the user's psychological state during the second period based on the reference biometric information and the second biometric information.

[0086] (Third Embodiment) Next, a third embodiment will be described. Parts common to the embodiments described above will be omitted as appropriate. Figure 11 shows an example of the schematic configuration of the information processing system 3 according to this embodiment. The information processing system 3 of this embodiment is a system that estimates and outputs the psychological state of a user based on the user's biometric information while the user is performing work in a factory (in this embodiment, physical work by the user is assumed as an example of a "task"). The specific details will be described below.

[0087] As shown in Figure 11, the information processing system 3 includes a biosensor 11, a PC 300, a server 20, a communication module 30, an output device 40, and an output device 50. The PC 300 is connected to the server 20 and the output device 40 via the communication module 30. The configurations other than the PC 300 are the same as those in the embodiments described above, so a detailed explanation is omitted.

[0088] The configuration of PC300 will now be described. As shown in Figure 11, PC300 includes an input device 310, a storage device 320, an information processing device 330, and an output device 340. Note that the hardware configuration of PC300 is not limited to the example in Figure 11, and may include hardware elements other than those illustrated in Figure 11.

[0089] The input device 310 is a device that receives operations from the user and inputs information corresponding to those operations into the information processing device 330. "Information corresponding to the operation" includes, for example, the work ID described later. For example, the input device 310 may be a touch panel, a keyboard, or a mouse.

[0090] The storage device 320 is a device for storing various types of information. For example, the storage device 320 can store programs executed by the information processing device 330, information indicating the results of processing by the information processing device 330, and so on.

[0091] The information processing device 330 is a device that determines the psychological state of a user based on the user's biometric information detected by the biosensor 11, various types of information input by the input device 310, etc. For example, the information processing device 330 may be a processing device that includes a processor.

[0092] Figure 12 shows an example of the functions of the information processing device 330. As shown in Figure 12, the information processing device 330 includes a biometric information acquisition unit 331, a state determination unit 332, a work information acquisition unit 333, a difficulty determination unit 334, a linking unit 335, an extraction unit 336, an update unit 337, and an output control unit 338. In the example in Figure 12, only the functions necessary for explaining the main parts of this embodiment are illustrated, but the functions of the information processing device 330 are not limited to these. In this embodiment, the information processing device 330 realizes the functions of each of the above parts by executing a program stored in the storage device 320. However, it is not limited to this, and some or all of these functions may be realized by dedicated hardware circuits (semiconductor integrated circuits, etc.).

[0093] The biometric information acquisition unit 331 acquires the user's biometric information detected by the biosensor 11.

[0094] The state determination unit 332 determines the user's psychological state based on the biological information acquired by the biological information acquisition unit 331 and a reference value. This is as described in each of the embodiments described above.

[0095] The work information acquisition unit 333 acquires work information indicating the work being performed by the user. Various methods can be considered for acquiring work information, but in this embodiment, the work information acquisition unit 333 acquires work information based on information input from the input device 310. More specifically, for example, the user performs an operation to input a work ID (an example of "work information") that identifies the content of the work, and the input device 310, which receives this operation, inputs the work ID to the information processing device 330, thereby enabling the work information acquisition unit 333 to acquire the work ID.

[0096] Furthermore, various methods are possible for acquiring work information, not limited to the forms described above. For example, if various tasks in a factory are managed by a time schedule, the work information acquisition unit 333 can identify the work ID corresponding to the current time as the work ID that the user is currently performing, based on the current time and a time schedule that shows information associating time with each work ID, and acquire that identified work ID. Alternatively, if various tasks in a factory are each linked to a predetermined area, the work information acquisition unit 333 can identify the work ID corresponding to the user's current location as the work ID that the user is currently performing, based on the user's current location and area-specific work information that shows information associating work IDs with each predetermined area in the factory, and acquire that identified work ID. Various known technologies can be used to determine the user's current location. For example, the work information acquisition unit 333 can also acquire location information indicating the user's location by receiving GPS signals that represent radio waves transmitted from each of multiple GPS satellites and calculating the user's location by 3D positioning based on the received GPS signals. Alternatively, the work information acquisition unit 333 may, for example, acquire a work ID indicating the work currently being performed by the user by inputting images of the factory into a pre-trained model that has been machine-learned to recognize the correspondence between images and tasks, and outputting a work ID indicating the task corresponding to the input image.

[0097] In short, the method by which the work information acquisition unit 333 acquires work information is arbitrary. The work information acquisition unit 333 can acquire work information based on information input from the input device 310, based on time information indicating the time, based on location information indicating the user's location, or based on images of the factory (images in which the user is reflected). In other words, the work information is determined based on one of the following: information input by the user, time information indicating the time, location information indicating the user's location, or images in which the user is reflected.

[0098] The difficulty level determination unit 334 determines the difficulty level of the user's work in the factory based on the work information acquired by the work information acquisition unit 333. For example, for each work ID, which is an example of work information indicating a work, difficulty level correspondence information indicating the difficulty level of the work identified by the work ID may be prepared and the difficulty level information corresponding to the work ID may be selected based on the work ID acquired by the work information acquisition unit 333 and its difficulty level correspondence information. In short, the difficulty level information indicating the difficulty level of the physical work performed by the user can be determined based on the work information indicating the physical work performed by the user.

[0099] Furthermore, instead of determining the difficulty level using difficulty-corresponding information that directly associates difficulty information with the work ID, as described above, the difficulty level may be determined using score-corresponding information, which associates a score (an example of information for determining difficulty) with each evaluation axis that indicates the criteria for evaluating the difficulty level of the work, as shown in Figure 13. In the example in Figure 13, four evaluation axes are set: accuracy requirement, skill level, physical load, and mental load, but the types and number of evaluation axes can be changed arbitrarily.

[0100] In the example in Figure 13, the criteria (indicators, parameters) for evaluating "accuracy requirements" are the error tolerance, with a score of "1" assigned for ±5 mm, a score of "2" for ±1 mm, and a score of "3" for ±0.1 mm. In the example in Figure 13, the criteria for evaluating "skill level" are proficiency, with a score of "1" assigned for tasks that anyone can do, a score of "2" assigned for tasks requiring training, and a score of "3" assigned for tasks requiring qualifications. In the example in Figure 13, the criteria for evaluating "physical load" are weight and posture, with a score of "1" assigned for light work, a score of "2" assigned for moderate work, and a score of "3" assigned for heavy work. In the example in Figure 13, the criteria for evaluating "mental load" are the presence or absence of judgment and abnormal response, with a score of "1" assigned for simple tasks (no judgment required, no abnormal response required), a score of "2" assigned for tasks requiring some kind of judgment, and a score of "3" assigned for tasks requiring advanced judgment. In this example, evaluation information is pre-prepared for each task ID to determine the content of each evaluation axis. Once a task ID is determined, the system is set up so that the score corresponding to each evaluation axis is also determined based on the evaluation information corresponding to that task ID. The sum of the scores corresponding to each evaluation axis becomes the score for the corresponding task ID (an example of information used to determine difficulty).

[0101] As an example of the work, we will explain using visual inspection (checking for scratches and dirt). The following information is provided as evaluation information corresponding to visual inspection (information for determining the content of each evaluation axis) (this information is provided in conjunction with the work ID indicating "visual inspection"): • Visually inspect the appearance of the product to check for scratches and dirt. • No work tolerances are required, but judgment must be made according to the visual quality standards. • Understanding of inspection standards is necessary. • Physical load is light (mostly seated work). • Mental load is moderate (judgment of whether the inspection standards are being followed is necessary).

[0102] This section explains an example of calculating a score corresponding to the "accuracy requirement" for visual inspection. Since the visual quality standard described in the evaluation information corresponding to visual inspection is a judgment of approximately ±1 mm, the error tolerance can be determined to be ±1 mm. As shown in Figure 13, a score of "2" is assigned in the case of ±1 mm, so the score corresponding to the "accuracy requirement" is calculated to be "2".

[0103] Next, we will explain an example of calculating the score corresponding to the "skill level" of visual inspection. From the evaluation information above, it can be determined that visual inspection only requires an understanding of the inspection standards and can be handled with simple training. Therefore, the proficiency level can be determined to be "possible for anyone". As shown in Figure 13, a score of "1" is assigned to tasks that anyone can perform, so the score corresponding to the "skill level" is calculated to be "1".

[0104] Next, we will explain an example of calculating the score corresponding to "physical load" in visual inspection. Based on the evaluation information above, since visual inspection is mainly a seated task, it can be determined that the weight and posture are "light work". As shown in Figure 13, a score of "1" is assigned for light work, so the score corresponding to "physical load" is calculated to be "1".

[0105] Next, we will explain an example of calculating the score corresponding to the "mental burden" of visual inspection. Based on the evaluation information above, since visual inspection is a task that requires judgment on whether or not it conforms to the inspection standards, the presence or absence of judgment and abnormality response can be determined to be "a task that requires some kind of judgment." As shown in Figure 13, a score of "2" is assigned to tasks that require some kind of judgment, so the score corresponding to "mental burden" is calculated to be "2".

[0106] Based on the above, the sum of the scores corresponding to each evaluation axis is 2 + 1 + 1 + 2 = 6 points. As an example of how to determine the difficulty level, if the average score of each evaluation axis is 2 points and the total score is 8 points, then if the total score is less than 8 points, the difficulty level can be determined to be low. In the above example of the visual inspection, the total score is 6 points, which is below the threshold (8 points), so the difficulty level is determined to be low.

[0107] Returning to Figure 12, the explanation continues. Similar to the embodiments described above, the linking unit 335 links difficulty information indicating the difficulty level determined by the difficulty level determination unit 334 to each piece of biometric information and stores it in the storage device 320. More specifically, in this embodiment, whenever biometric information is acquired by the biometric information acquisition unit 331 while the user is performing work in the factory, the linking unit 335 links the biometric information with difficulty information indicating the difficulty level of the work (physical work performed by the user) at that time and psychological state information indicating the user's psychological state determined based on the biometric information to generate the aforementioned unit information, and adds the generated unit information to the linking registration information already stored in the storage device 320 to update the linking registration information.

[0108] The extraction unit 336 refers to the linked registration information and extracts biological information linked to difficulty information indicating a low difficulty level for the task, as reference biological information indicating the standard biological information.

[0109] The update unit 337 updates the reference value when the number of extracted reference biological information items exceeds a predetermined number. The update method is the same as described in each of the embodiments described above.

[0110] If the reference value is updated by the update unit 337, the state determination unit 332 uses the updated reference value to determine the user's psychological state. If the reference value has not been updated, the state determination unit 332 uses the standard reference value described above to determine the user's psychological state.

[0111] The output control unit 338 controls the output of various types of information. For example, the output control unit 338 controls the output of information based on psychological state information indicating the user's psychological state as determined by the state determination unit 332. In this embodiment, the output control unit 338 transmits the above-mentioned linked registration information to the server 20 via the communication module 30 each time the user completes a task or after a certain period of time has elapsed. The output control unit 338 also generates a psychological state report based on the above-mentioned linked registration information each time the user completes a task or after a certain period of time has elapsed, and outputs the generated psychological state report from the output device 340. Furthermore, the output control unit 238 transmits the generated psychological state report to the server 20 or the output device 40 via the communication module 30. These contents are the same as those described in the second embodiment.

[0112] Based on the above description, the information processing device 330 of this embodiment acquires first biometric information indicating the user's biometric information during a first period, which represents the period during which the user is performing work (an example of a task) in a factory. For each piece of first biometric information, difficulty information indicating the difficulty level of the work is associated with and stored. The first biometric information associated with the lowest difficulty level is extracted as reference biometric information indicating the reference biometric information. The information processing device 330 then acquires second biometric information indicating the user's biometric information during a second period, which represents a period after the first period during which the user is performing work (a task in the same phase as the task in the first period). Based on the reference biometric information and the second biometric information, the device can be considered to be configured to determine the user's psychological state.

[0113] Furthermore, the information processing device 330 can be considered to be configured to determine the user's psychological state based on a reference value that is updated according to the extraction status of the reference biological information, and second biological information. Specific examples of the operation of the information processing device 330 will be described later. The functions of the server 20 are basically the same as those described in each of the embodiments above.

[0114] Next, an example of the operation of the information processing device 330 in this embodiment will be described. Figure 14 is a flowchart showing an example of the operation of the information processing device 330 while a user is performing work in a factory. Note that the order of processing in each step described below can be changed arbitrarily and is not limited to the example in Figure 14. In addition, in the specific details of each process described below, explanations will be omitted as appropriate where they overlap with the explanations of the parts described above.

[0115] As shown in Figure 14, first, the biometric information acquisition unit 331 acquires the user's biometric information (step S21). Next, the state determination unit 332 determines the user's psychological state based on the biometric information acquired in step S21 and a reference value (step S22).

[0116] Next, the work information acquisition unit 333 acquires work information (step S23). Then, the difficulty level determination unit 334 determines the difficulty level of the work to be performed by the user based on the work information acquired in step S23 (step S24).

[0117] Next, the linking unit 335 links and registers the biometric information acquired in step S21, the psychological state information indicating the user's psychological state determined in step S22, and the difficulty level information indicating the difficulty level determined in step S24 (step S25).

[0118] Next, the extraction unit 336 refers to the linked registration information after step S25 and extracts the above-mentioned reference biological information for the most recent period, and the update unit 337 determines whether the number of reference biological information extracted by the extraction unit 335 is greater than or equal to a predetermined number (step S26). If the result of step S26 is positive (step S26: Yes), the update unit 337 updates the reference value (step S27). On the other hand, if the result of step S26 is negative (step S26: No), the update unit 337 does not perform an update, and the current reference value is maintained (step S28).

[0119] After step S27 or step S28, the information processing device 330 determines whether the user's work has been completed (step S29). If the result of step S29 is negative (step S29: No), the information processing device 330 repeats the processing from step S21 onwards. If the result of step S29 is positive (step S29: Yes), the processing ends.

[0120] After the above processing is completed, as described above, the information processing device 330 transmits the above-mentioned linked registration information to the server 20 via the communication module 30. The information processing device 330 also generates the above-mentioned psychological state report based on the above-mentioned linked registration information and outputs the generated psychological state report from the output device 240. Furthermore, the output control unit 338 transmits the generated psychological state report to the server 20 or the output device 40 via the communication module 30.

[0121] (Function and Effects) As described above, the information processing device 330 of this embodiment determines the psychological state of a user during work using reference values ​​based on the user's biometric information (reference biometric information) while the user is performing a low-difficulty task (factory work). Since the characteristics of each user, including the degree of alertness during work, differ, this embodiment improves the accuracy of estimating the user's psychological state during work by using reference values ​​that reflect the user's characteristics. Furthermore, similar to the embodiments described above, the information processing device 330 of this embodiment sequentially updates the reference values ​​according to the extraction status of reference biometric information, so as the accumulation of reference biometric information progresses, it becomes possible to set reference values ​​that better reflect the user's characteristics. Therefore, it is possible to further improve the accuracy of estimating the user's psychological state during work.

[0122] (Modification 1 of the third embodiment) For example, all or part of the functions of the information processing device 330 described above may be installed on the server 20. In other words, all of the functions shown in Figure 12 may be provided by the server 20 instead of the information processing device 330, or the functions shown in Figure 12 may be distributed and installed on both the information processing device 330 and the server 20.

[0123] (Modification 2 of the Third Embodiment) In the third embodiment described above, the information processing device 330 registers information obtained at the corresponding time for each of the reference values ​​and psychological state information included in the linked registration information described above. However, it is not limited to this, and for example, the information processing device 330 may update each of the reference values ​​and psychological state information included in the linked registration information described above based on the latest reference values ​​each time the user's work is completed or after a certain period of time has elapsed. For example, all reference values ​​included in the linked registration information described above may be updated to the latest reference values, and all psychological state information may be updated to the judgment result using the latest reference values. Then, the psychological state report described above may be generated and output based on the updated linked registration information.

[0124] Alternatively, for example, the information processing device 330 may store pairs of biometric information and difficulty information without determining the user's psychological state while the user is performing work in the factory or until a certain period of time has elapsed. After the user finishes work or a certain period of time has elapsed, it may extract one or more biometric pieces of information linked to difficulty information indicating that the work is easy from the previously stored information set, and determine a reference value based on the one or more reference biometric pieces of information extracted. The method for determining the reference value may be the same as the method for updating the reference value described above. The information processing device 330 may then use the determined reference value to collectively determine the user's psychological state at the time each piece of biometric information was acquired.

[0125] In other words, the information processing device 330 may acquire first biometric information indicating the user's biometric information during a first period of time in which the user is performing work (an example of a task) in a factory, associate and store difficulty information indicating the difficulty level of the work with each piece of first biometric information, extract first biometric information associated with low difficulty information as reference biometric information indicating a standard biometric information, and determine the user's psychological state during the first period based on the reference biometric information and the first biometric information. In combination with the third embodiment described above, the information processing device 330 can be configured to perform at least one of the following: acquire first biometric information indicating the user's biometric information during a first period; store difficulty information associated with each piece of first biometric information; extract first biometric information associated with low difficulty information as reference biometric information indicating a standard biometric information; determine the user's psychological state during the first period based on the reference biometric information and the first biometric information; and acquire second biometric information indicating the user's biometric information during a second period, which is a period after the first period during which the user is performing a task; and determine the user's psychological state during the second period based on the reference biometric information and the second biometric information.

[0126] (Modifications common to all embodiments) Next, modifications common to each of the embodiments described above will be explained. The following explanation will be based on the configuration of the first embodiment described above. In this modification, the output control unit 108 outputs a control signal to control the control devices that form the user's surrounding environment when the state determination unit 102 determines that the user's biometric information value is above a standard value (when it is determined that the user is in an "anxious" or "stressed (tension)" state).

[0127] For example, control devices that form the user's surrounding environment within the vehicle interior space in the first embodiment described above include air conditioning equipment (air conditioners, blowers), lighting equipment, sound equipment (speakers), and diffusers.

[0128] Let's take the case where the control device is an air conditioning device as an example. The output control unit 108 described above can output a control command to the air conditioning device to lower the temperature of the user's surrounding environment if it determines that the user's biometric information value is above a certain threshold. In other words, the control signal includes a control command to the air conditioning device to lower the temperature of the user's surrounding environment. For example, in the case of an air conditioner, the output control unit 108 described above can output a control command to the air conditioner to operate it so that the temperature of the user's surrounding environment is 1 to 5 degrees lower. For example, in the case of a fan, the output control unit 108 described above can output a control command to the fan to increase the fan's airflow strength.

[0129] Next, we will explain using the example of a lighting device as the control device. The output control unit 108 described above can output a control command to the lighting device to dim the lights so that the color temperature of the user's surrounding environment decreases, if it is determined that the user's biometric information value is above a standard value. In other words, the control signal includes a control command to the lighting device to dim the lights so that the color temperature of the user's surrounding environment decreases. For example, the output control unit 108 described above can output a control command to the lighting device to dim the lights so that the color temperature of the user's surrounding environment decreases by a predetermined amount. For example, it may output a control command to change the color temperature towards warmer colors (2700K to 4000K).

[0130] Next, let's explain using the case where the control device is an audio device as an example. The output control unit 108 described above can output a control command to the audio device to reduce the volume of the user's surrounding environment if it determines that the user's biometric information value is above a certain threshold. In other words, the control signal includes a control command to the audio device to reduce the volume of the user's surrounding environment. For example, the output control unit 108 described above can output a control command to the speaker to reduce the volume by 5% to 20%.

[0131] Next, let's explain using the example of a diffuser as the control device. The output control unit 108 described above can output a control command to the diffuser to spray a relaxing fragrance (for example, lavender, bergamot, chamomile, sandalwood, etc.) if it determines that the user's biometric information value is above a certain threshold. In other words, the control signal includes a control command to spray a relaxing fragrance to the diffuser.

[0132] Furthermore, if the user's biometric data is determined to be above a certain threshold, the system may, for example, activate all of the above-mentioned control devices, or it may selectively activate only some of the above-mentioned control devices depending on the level of stress and the user's characteristics (e.g., sensitivity to light, sound, smell). For example, for a user who is sensitive to sound, the speaker volume has a significant impact, so the system may selectively activate the speaker while keeping the other control devices inactive. Alternatively, the system may increase the number of control devices activated as the level of stress increases.

[0133] Figure 15 is a flowchart showing an example of the operation of the information processing device 15 while the user is driving the vehicle in this modified example. The difference from the flowchart shown in Figure 8 is that steps S101 and S102 are added between steps S2 and S3, and the processing content of the other steps is the same as in the first embodiment described above.

[0134] After step S2 described above, if the determination result by the state determination unit 102 is that the value of the biological information acquired in step S1 is equal to or greater than the reference value (step S101: Yes), the output control unit 108 outputs the control signals described above to the control devices (step S102). For example, the output control unit 108 can output a control command to the air conditioner to operate it so that the temperature of the user's surrounding environment is 1 to 5 degrees lower, or it can output a control command to the lighting equipment to dim the lights so that the color temperature of the user's surrounding environment is reduced by a predetermined amount, or it can output a control command to the speaker to reduce the volume by 5 to 20%, or it can output a control command to the diffuser to spray a relaxing fragrance. All of these control devices may be operated, or some of the control devices may be operated selectively.

[0135] On the other hand, if the determination result by the state determination unit 102 is that the value of the biological information acquired in step S1 is less than the reference value (step S101: No), the process proceeds to step S3 described above.

[0136] As described above, in this modified example, psychological burden can be reduced by auditory stimulation or reduction of auditory stimulation, and by stimulating the parasympathetic nervous system, thereby improving the work efficiency and safety of the task (driving in this example).

[0137] Furthermore, although this modified example was described based on the configuration of the first embodiment described above, it may also be based on the configuration of the second embodiment or the third embodiment described above.

[0138] While embodiments of this disclosure have been described above, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These novel embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents.

[0139] Furthermore, the effects described in the embodiments herein are merely illustrative and not limiting, and other effects may also occur.

[0140] 1,2 Information Processing System 10 Vehicle System 11 Biometric Sensor 12 Vehicle Sensor 13 GPS 14 Storage Device 15 Information Processing Device 16 Output Device 20 Server 30 Communication Module 101 Biometric Information Acquisition Unit 102 State Determination Unit 103 Driving Environment Information Acquisition Unit 104 Difficulty Determination Unit 105 Linking Unit 106 Extraction Unit 107 Update Unit 108 Output Control Unit 200 PC 210 PC Input Device 220 Storage Device 230 Information Processing Device 240 Output Device 231 Biometric Information Acquisition Unit 232 State Determination Unit 233 Operation Information Acquisition Unit 234 Difficulty Determination Unit 235 Linking Unit 236 Extraction Unit 237 Update Unit 238 Output Control Unit

Claims

1. An information processing method performed by an information processing device, comprising: acquiring first biometric information representing the user's biometric information during a first period indicating the period during which the user is performing a task; associating and storing difficulty information representing the difficulty of the task with each of the first biometric information; extracting the first biometric information associated with the difficulty information of a lower difficulty level as reference biometric information representing reference biometric information; acquiring second biometric information representing the user's biometric information during a second period that is after the first period and indicates the period during which the user is performing the task; and determining the user's psychological state based on the reference biometric information and the second biometric information.

2. The information processing method according to claim 1, which determines the user's psychological state based on a reference value updated according to the extraction status of the reference biological information and the second biological information.

3. The information processing method according to claim 2, wherein the reference value is updated when the number of extracted reference biological information items exceeds a predetermined number.

4. The information processing method according to claim 1, wherein the task is driving a vehicle, and the difficulty information indicates the difficulty level of driving the vehicle by the user.

5. The information processing method according to claim 4, which outputs an image in which information indicating the changes in the determined psychological state of the user is superimposed on route information indicating the route traveled by the vehicle.

6. The information processing method according to claim 4, which outputs an image in which information showing the changes in the difficulty level information is superimposed on route information showing the route traveled by the vehicle.

7. The information processing method according to claim 4, wherein the difficulty of driving the vehicle is determined based on information including the speed of the vehicle and the direction in which the vehicle is traveling, or on an image captured by a camera installed in the vehicle that captures the direction in which the vehicle is traveling.

8. The information processing method according to claim 1, wherein the task is an operation that includes at least one of work and learning using a terminal, and the difficulty information indicates the difficulty level of the operation performed by the user.

9. The information processing method according to claim 8, wherein the difficulty of the above-mentioned work is determined according to the amount of operation performed on the terminal.

10. The information processing method according to claim 1, wherein the task is a physical task performed by the user, and the difficulty information indicates the difficulty level of the physical task performed by the user.

11. The information processing method according to claim 10, wherein the difficulty information is determined based on work information indicating physical work performed by the user, and the work information is determined based on any of the following: information entered by the user, time information indicating the time, location information indicating the user's position, and an image in which the user is captured.

12. The information processing method according to claim 2, wherein if the value of the second biological information is equal to or greater than the reference value, a control signal is output to control a control device that forms the user's surrounding environment.

13. The information processing method according to claim 1, wherein determining the user's psychological state based on the reference biometric information and the second biometric information includes determining the user's psychological state during the second period, and further, determining the user's psychological state during the first period based on the reference biometric information and the first biometric information.

14. An information processing device comprising: an acquisition unit that acquires first biometric information indicating the user's biometric information during a first period indicating the period during which the user is performing a task; an association unit that associates difficulty information indicating the difficulty of the task with each of the first biometric information and stores it in a storage unit; an extraction unit that extracts the first biometric information associated with the difficulty information indicating that the difficulty of the task is low as reference biometric information indicating reference biometric data; and a determination unit that determines the user's psychological state based on the reference biometric information and second biometric information indicating the user's biometric information during a second period that is after the first period and indicates the period during which the user is performing the task.

15. An information processing method performed by an information processing device, comprising: acquiring first biometric information indicating the user's biometric information during a first period indicating the period during which the user is performing a task; associating and storing difficulty information indicating the difficulty of the task with each of the first biometric information; extracting the first biometric information associated with the difficulty information indicating a lower difficulty as reference biometric information; determining the user's psychological state during the first period based on the reference biometric information and the first biometric information; and acquiring second biometric information indicating the user's biometric information during a second period that is after the first period and indicates the period during which the user is performing the task; and determining the user's psychological state during the second period based on the reference biometric information and the second biometric information.

Images