Information processing device, determination method, and determination program

Abstract

An information processing device (100) communicates with at least one of a stimulus generation device and a reaction measurement device that are present in a space in which a user is present, and the information processing device includes at least one of the stimulus generation device and the reaction measurement device. The information processing device (100) comprises: an acquisition unit (120) that acquires information indicating a stimulus output to a space; and a control unit (130) that causes the stimulus generation device to output a stimulus that is different from the aforementioned stimulus, that acquires, from the reaction measurement device, reaction information indicating a reaction of the user to the stimulus, and that determines whether or not the user has reacted on the basis of the reaction information.

Description

Information processing device, determination method, and determination program

[0001] The present disclosure relates to an information processing device, a determination method, and a determination program.

[0002] There are cases where a user is concentrating on a task. A technique for calculating a concentration level has been proposed (see Patent Document 1). The concentration level prediction system in Patent Document 1 calculates a user's concentration level for an intellectual task based on a feature amount related to the user's biological activity.

[0003] International Publication No. 2022 / 070788

[0004] As with the above technology, it is possible to calculate a user's concentration level based on the feature amount of the user's biometric information and determine whether the user is in a state of concentration (hereinafter referred to as a "concentration state"). However, in order to calculate the concentration level based on the feature amount of the biometric information, it is necessary to continuously acquire the biometric information. Users are strongly reluctant to continuously acquire biometric information.

[0005] The purpose of the present disclosure is to reduce user resistance.

[0006] According to one aspect of the present disclosure, there is provided an information processing device. The information processing device communicates with at least one of a stimulus generation device and a reaction measurement device present in a space in which a user is present, or includes at least one of the stimulus generation device and the reaction measurement device. The information processing device has an acquisition unit that acquires information indicating a stimulus output in the space, and a control unit that causes the stimulus generation device to output a stimulus different from the stimulus, acquires reaction information indicating the user's reaction to the stimulus from the reaction measurement device, and determines whether the user has reacted based on the reaction information.

[0007] According to the present disclosure, it is possible to reduce the user's resistance.

[0008] FIG. 1 is a diagram showing a determination system of embodiment 1. FIG. 2 is a diagram showing hardware possessed by an information processing device of embodiment 1. FIG. 3 is a block diagram showing functions of an information processing device of embodiment 1. FIG. 4 is a flowchart showing an example of processing executed by an information processing device of embodiment 1. FIG. 5 is a diagram showing an example of an output sound table of embodiment 1. FIG. 6 is a flowchart showing an example of processing executed by an information processing device of embodiment 2. FIG. 7 is a diagram showing an example of an output sound table of embodiment 2. FIG. 8 is a flowchart showing an example of processing executed by an information processing device of embodiment 3. FIG. 9 is a diagram showing an example of schedule information of embodiment 6. FIG. 10 is a diagram showing an example of a notification determination table of embodiment 6. FIG. 11 is a diagram showing an example of a notification determination table of a modified example of embodiment 6.

[0009] Hereinafter, an embodiment will be described with reference to the drawings.

[0010] Embodiment 1. Fig. 1 is a diagram showing a determination system according to embodiment 1. The determination system includes an information processing device 100, a microphone 200, a speaker 300, and a camera 400. The information processing device 100, the microphone 200, the speaker 300, and the camera 400 communicate with each other via a network. The network may be a wired network or a wireless network. Fig. 1 shows a situation in which a user is working using a PC (Personal Computer).

[0011] For example, the information processing device 100 is a device that executes a determination method. Also, for example, the information processing device 100 is a server. The microphone 200 acquires sounds in the space in which the user is present (i.e., environmental sounds). In other words, environmental sounds are sounds that are being generated in that space. The speaker 300 outputs sounds. For example, the speaker 300 is located above the user. The speaker 300 is also referred to as a stimulus generating device. The camera 400 captures images of the user. The camera 400 is also referred to as an imaging device. The camera 400 is a reaction measurement device. The image obtained by the camera 400 is reaction information.

[0012] The microphone 200, the speaker 300, and the camera 400 may be included in the information processing device 100. For example, when the microphone 200, the speaker 300, and the camera 400 are included in the information processing device 100, the information processing device 100 is a smartphone, a tablet terminal, or a PC.

[0013] The information processing device 100 may also communicate with at least one of a speaker 300 and a camera 400 present in the space where the user is present. Furthermore, the information processing device 100 may include at least one of a speaker 300 and a camera 400.

[0014] Next, the hardware of the information processing device 100 will be described. Fig. 2 is a diagram showing the hardware of the information processing device of embodiment 1. The information processing device 100 is also called a computer. The information processing device 100 has a processor 101, a volatile storage device 102, and a non-volatile storage device 103.

[0015] The processor 101 controls the entire information processing device 100. For example, the processor 101 is a central processing unit (CPU) or a field programmable gate array (FPGA). The processor 101 may be a multiprocessor. The information processing device 100 may also include a processing circuit.

[0016] The volatile storage device 102 is a main storage device of the information processing device 100. For example, the volatile storage device 102 is a random access memory (RAM). The nonvolatile storage device 103 is an auxiliary storage device of the information processing device 100. For example, the nonvolatile storage device 103 is a hard disk drive (HDD) or a solid state drive (SSD).

[0017] Next, a description will be given of functions of the information processing device 100. Fig. 3 is a block diagram showing functions of the information processing device of embodiment 1. The information processing device 100 has a storage unit 110, an acquisition unit 120, and a control unit 130.

[0018] The storage unit 110 may be realized as a storage area secured in the volatile storage device 102 or the non-volatile storage device 103. The storage unit 110 may also be called a memory. A part or all of the acquisition unit 120 and the control unit 130 may be realized by a processing circuit. Furthermore, a part or all of the acquisition unit 120 and the control unit 130 may be realized as a program module executed by the processor 101. For example, the program executed by the processor 101 is also called a determination program or a determination program product. For example, the determination program is recorded on a recording medium.

[0019] The storage unit 110 stores various information. The functions of the acquisition unit 120 and the control unit 130 will be described later.

[0020] Next, the processing executed by the information processing device 100 will be described using a flowchart. FIG. 4 is a flowchart showing an example of the processing executed by the information processing device of embodiment 1. (Step S11) The acquisition unit 120 acquires an environmental sound signal indicating environmental sound from the microphone 200. The acquisition unit 120 may acquire the environmental sound signal via another device. (Step S12) The control unit 130 outputs a sound different from the environmental sound indicated by the environmental sound signal from the speaker 300. For example, this sound is a sound that is louder than the environmental sound, a sound with a different tone from the environmental sound, or a sound with a different sound period from the environmental sound. For example, the tone is expressed by sharpness (unit: acum), roughness (unit: asper), fluctuation intensity (unit: vacil), and tonality. For example, the sound period is expressed in units of bpm. The control unit 130 may also determine the sound to be output from the speaker 300 using an output sound table. An example of the output sound table is shown below.

[0021] 5 is a diagram showing an example of an output sound table according to the first embodiment. For example, the output sound table 111 is stored in the storage unit 110. The output sound table 111 indicates the correspondence between environmental sounds and output sounds. The control unit 130 may determine the sound to be output from the speaker 300 based on the output sound table 111.

[0022] (Step S13) The control unit 130 acquires video from the camera 400 capturing the user. (Step S14) The control unit 130 determines whether the user has reacted based on the video. More specifically, the control unit 130 analyzes the video. The control unit 130 determines whether the user has reacted based on the analysis results. If the user has reacted, the process proceeds to step S15. If the user has not reacted, the process proceeds to step S16. (Step S15) The control unit 130 determines that the user is not in a state of concentration. (Step S16) The control unit 130 determines that the user is in a state of concentration.

[0023] The processes of steps S13 and S14 are user response detection processes that detect whether the user has responded. In this way, the control unit 130 determines whether the user is in a state of concentration. The control unit 130 may notify the user of the determination result. For example, the control unit 130 notifies the PC used by the user of the determination result. This allows the user to recognize whether the user is in a state of concentration.

[0024] According to the first embodiment, the information processing device 100 determines whether the user is in a concentration state without using biometric information. That is, the information processing device 100 does not acquire biometric information from the user. Therefore, the information processing device 100 can reduce the user's resistance.

[0025] Furthermore, the control unit 130 may cause the speaker 300 to output a sound for determining whether the user is in a shallow state of concentration. For example, information about the sound is stored in the storage unit 110. When the sound is output from the speaker 300 and the user does not react based on the video, the control unit 130 determines that the user is in a shallow state of concentration.

[0026] When the control unit 130 determines that the user is in a state of concentration, the control unit 130 may cause the speaker 300 to output a sound different from the sound output from the speaker 300 (for example, a sound louder than the sound output from the speaker 300). The control unit 130 acquires video from the camera 400 capturing the user. When the user does not react based on the video, the control unit 130 determines that the user is in a state of deep concentration. When the user is in a state of deep concentration, the control unit 130 may prompt the user to take a break. For example, the control unit 130 transmits information prompting the user to take a break to the PC used by the user. In this way, the information processing device 100 can provide the user with an opportunity to take a break.

[0027] Next, modified examples will be described. In the first embodiment, a case has been described in which sound is used to determine whether or not the user is in a state of concentration. The information processing device 100 may use a stimulus other than sound to determine whether or not the user is in a state of concentration. Modified examples will be described below.

[0028] Variation 1 of Embodiment 1. The acquisition unit 120 acquires the illuminance of the space in which the user is present. For example, the acquisition unit 120 acquires the illuminance from a light meter. The control unit 130 controls the lighting in the space to change the illuminance. The control unit 130 acquires video from the camera 400 that is capturing the user. The control unit 130 determines whether the user has reacted based on the video. If the user does not react, the control unit 130 determines that the user is in a state of concentration. The lighting is also referred to as a stimulus generating device. According to Variation 1 of Embodiment 1, the information processing device 100 does not acquire biometric information, thereby reducing the user's resistance.

[0029] Modification 2 of Embodiment 1 The acquisition unit 120 acquires the color temperature of the space in which the user is present. For example, the acquisition unit 120 acquires the color temperature from a colorimeter or the camera 400. The control unit 130 controls the lighting in the space so as to change the color temperature. The control unit 130 acquires video from the camera 400 that is capturing the user. The control unit 130 determines whether the user has reacted based on the video. If the user does not react, the control unit 130 determines that the user is in a state of concentration. The lighting is also referred to as a stimulus generating device. According to Modification 2 of Embodiment 1, the information processing device 100 does not acquire biometric information, thereby reducing the user's sense of resistance.

[0030] Variation 3 of Embodiment 1. The acquisition unit 120 acquires the direction of the image displayed in the space where the user is present. For example, the acquisition unit 120 acquires the direction from a device controlling a projector displaying the image. For example, the image is an image of a circle moving from right to left. The control unit 130 controls the device controlling the projector outputting the image so that the direction is changed. For example, if the image displayed in the space where the user is present is an image of a circle moving from right to left, the control unit 130 controls the device controlling the projector so that the projector displays an image of a circle moving from left to right. The control unit 130 acquires an image from a camera 400 capturing an image of the user. The control unit 130 determines whether the user has responded based on the image. If the user does not respond, the control unit 130 determines that the user is in a state of concentration. The projector is also referred to as a stimulus generation device. According to Variation 3 of Embodiment 1, the information processing device 100 does not acquire biometric information, thereby reducing the user's resistance.

[0031] Variation 4 of Embodiment 1 The acquisition unit 120 acquires the temperature of the space in which the user is present. For example, the acquisition unit 120 acquires the temperature from a temperature sensor. The control unit 130 controls an air conditioner present in the space to change the temperature. The control unit 130 acquires video from a camera 400 capturing an image of the user. The control unit 130 determines whether the user has reacted based on the video. If the user does not react, the control unit 130 determines that the user is in a state of concentration. The air conditioner is also referred to as a stimulus generating device. According to Variation 4 of Embodiment 1, the information processing device 100 does not acquire biometric information, thereby reducing the user's resistance.

[0032] Variation 5 of Embodiment 1 The acquisition unit 120 acquires the direction of airflow output by an air conditioner present in the space where the user is present. For example, the acquisition unit 120 acquires the direction from the air conditioner. The control unit 130 controls the air conditioner to change the direction. The control unit 130 acquires video from a camera 400 capturing the user. The control unit 130 determines whether the user has reacted based on the video. If the user does not react, the control unit 130 determines that the user is in a state of concentration. The air conditioner is also referred to as a stimulus generating device. According to Variation 5 of Embodiment 1, the information processing device 100 does not acquire biometric information, thereby reducing the user's resistance.

[0033] Variation 6 of Embodiment 1. The acquisition unit 120 acquires information indicating an odor being output in a space where the user is present. For example, the acquisition unit 120 acquires information about the odor from an odor-emitting device (e.g., an aroma diffuser) that is outputting the odor. The control unit 130 controls the odor-emitting device so that the odor is changed. The control unit 130 acquires video from a camera 400 that is capturing an image of the user. The control unit 130 determines whether the user has responded based on the video. If the user does not respond, the control unit 130 determines that the user is in a state of concentration. Note that the odor-emitting device is also referred to as a stimulus-emitting device. According to Variation 6 of Embodiment 1, the information processing device 100 does not acquire biometric information, thereby reducing the user's sense of resistance.

[0034] In this way, the information processing device 100 can determine whether or not the user is in a concentration state by using stimuli other than sound.

[0035] As described above, the information processing device 100 is a device that communicates with at least one of a stimulus generator and a camera 400 that are present in a space where the user is present, or a device that includes at least one of a stimulus generator and a camera 400. The acquisition unit 120 acquires information indicating a stimulus being output in the space. The control unit 130 causes the stimulus generator to output a stimulus different from the stimulus. The control unit 130 acquires video from the camera 400 that is capturing the user. It may also be expressed that the control unit 130 causes the stimulus generator to output a stimulus different from the stimulus and acquires video from the camera 400 that is capturing the user. The control unit 130 determines whether the user has reacted based on the video.

[0036] Second Embodiment Next, a second embodiment will be described. In the second embodiment, differences from the first embodiment will be mainly described. Furthermore, in the second embodiment, descriptions of the commonalities between the first embodiment and the second embodiment will be omitted.

[0037] Next, the processing executed by the information processing device 100 will be described using a flowchart. Fig. 6 is a flowchart showing an example of the processing executed by the information processing device of embodiment 2. The processing in Fig. 6 differs from the processing in Fig. 4 in that steps S11a and 12a are executed. Therefore, steps S11a and 12a will be described in Fig. 6. Further, a description of the processing other than steps S11a and 12a will be omitted.

[0038] (Step S11a) The acquisition unit 120 acquires age information indicating the age of the user. For example, the acquisition unit 120 acquires the age information from the storage unit 110. Alternatively, for example, the acquisition unit 120 acquires the age information from an external device. Note that the external device is a device that exists outside the information processing device 100. For example, the external device is a cloud server, an external memory, etc. The external device is not shown in the drawing.

[0039] (Step S12a) The control unit 130 causes the speaker 300 to output a sound that is different from the environmental sound indicated by the environmental sound signal and that corresponds to the user's age. For example, if the user is in their 60s, the control unit 130 causes the speaker 300 to output a sound that is louder than a sound that corresponds to a user in their 20s. The control unit 130 may also use the output sound table 111a to determine the sound to be output from the speaker 300. An example of the output sound table 111a is shown below.

[0040] 7 is a diagram showing an example of an output sound table according to the second embodiment. For example, the output sound table 111a is stored in the storage unit 110. The output sound table 111a indicates the correspondence between age, environmental sound, and output sound. The control unit 130 may determine the sound to be output from the speaker 300 based on the output sound table 111a.

[0041] According to the second embodiment, the information processing device 100 can improve the accuracy of determining the concentration state by causing the speaker 300 to output a sound that takes age into consideration.

[0042] Furthermore, the acquisition unit 120 may acquire information indicating that the user has poor hearing from the storage unit 110 or an external device. Then, the control unit 130 causes the speaker 300 to output a sound that is different from the environmental sound indicated by the environmental sound signal and that is intended for people with poor hearing. For example, the control unit 130 causes the speaker 300 to output a sound that is louder than the environmental sound indicated by the environmental sound signal and that is intended for people with poor hearing. This allows the information processing device 100 to improve the accuracy of determining the concentration state.

[0043] Embodiment 3 Next, embodiment 3 will be described. In embodiment 3, differences from embodiment 1 will be mainly described. Furthermore, in embodiment 3, descriptions of matters common to embodiment 1 will be omitted.

[0044] Next, the processing executed by the information processing device 100 will be described using a flowchart. Fig. 8 is a flowchart showing an example of the processing executed by the information processing device of embodiment 3. The processing in Fig. 8 differs from the processing in Fig. 4 in that steps S11b and 14a are executed. Therefore, steps S11b and 14a will be described in Fig. 8. Further, a description of the processing other than steps S11b and 14a will be omitted.

[0045] (Step S11b) The acquisition unit 120 acquires habit information indicating the user's habit from the storage unit 110 or an external device. For example, the habit information indicates that the user has a habit of looking to the right. Also, for example, the habit information indicates that the user has a habit of looking slightly upward.

[0046] (Step S14a) The control unit 130 determines whether the user has responded based on the video. Furthermore, when making this determination, the control unit 130 takes the user's habits into consideration when determining whether the user has responded. For example, if the user has a habit of looking to the right and the video shows the user making a motion of looking to the right, the control unit 130 determines that the user has not responded. If the user has a habit of looking to the right and the video shows the user making a motion of looking to the left, the control unit 130 determines that the user has responded. Furthermore, for example, if the user has a habit of slightly looking up and the video shows the user making a motion of slightly looking up, the control unit 130 determines that the user has not responded. If the user has a habit of slightly looking up and the video shows the user making a motion of looking up significantly, the control unit 130 determines that the user has responded. If the user has a habit of slightly looking up and the video shows the user making a motion of looking up significantly, the control unit 130 determines that the user has responded. If the user has responded, the process proceeds to step S15. If the user has not responded, the process proceeds to step S16.

[0047] According to the third embodiment, the information processing device 100 can accurately determine the user's reaction by taking into account the user's habits.

[0048] Embodiment 4 Next, embodiment 4 will be described. In embodiment 4, differences from embodiment 1 will be mainly described. Furthermore, in embodiment 4, description of matters common to embodiment 1 will be omitted.

[0049] The control unit 130 periodically determines the user's concentration state. For example, the control unit 130 determines the user's concentration state every hour. This allows the control unit 130 to obtain multiple determination results.

[0050] The control unit 130 notifies the user of the multiple determination results. For example, the control unit 130 notifies the user of the multiple determination results to the PC used by the user. For example, the control unit 130 notifies the user of information such as "You have been in a state of concentration for three hours." This allows the user to recognize the state over a long period of time.

[0051] Embodiment 5 Next, embodiment 5 will be described. In embodiment 5, differences from embodiment 4 will be mainly described. Furthermore, in embodiment 5, descriptions of matters common to embodiment 4 will be omitted.

[0052] The acquisition unit 120 acquires the estimated rest time from the storage unit 110 or an external device. The estimated rest time is the time during which a break is generally required or the time during which fatigue is generally detected. For example, the estimated rest time is three hours.

[0053] If the state of concentration continues beyond the estimated break time based on multiple determination results, the control unit 130 executes a process to prompt the user to take a break. For example, the estimated break time is assumed to be three hours. The multiple determination results indicate that the user has been in a state of concentration for three hours. Since the state of concentration has continued for three hours, the control unit 130 prompts the user to take a break. For example, the control unit 130 transmits information prompting the user to take a break to the PC used by the user.

[0054] According to the fifth embodiment, the information processing device 100 can give the user a sense of rest.

[0055] Furthermore, if the state of concentration continues for the estimated break time, the control unit 130 may execute processing to cause the speaker 300 to output a sound to elicit a user's reaction. The sound is quite loud. The speaker 300 may gradually increase the volume. The sound may have a tone and period that is quite different from ambient sounds. By having the speaker 300 output the sound, the information processing device 100 can forcibly end the state of concentration. This prevents the user from becoming more fatigued.

[0056] Modification of the Fifth Embodiment In the fifth embodiment, the case where the estimated rest time is the time when a rest is generally required has been described. In a modification of the fifth embodiment, the estimated rest time is the time when the user requires a rest. Note that the estimated rest time may be measured using the stream-bounce, the double flash illusion, the McGurk effect, or the like.

[0057] The acquisition unit 120 acquires the estimated rest time from the storage unit 110 or an external device. The control unit 130 executes a process to prompt the user to take a break if the estimated rest time or the state of concentration continues based on a plurality of determination results, or a process to output a sound from the speaker 300 to elicit a response from the user.

[0058] According to the modification of the fifth embodiment, the information processing device 100 can execute processing tailored to the user.

[0059] Sixth Embodiment Next, a sixth embodiment will be described. In the sixth embodiment, differences from the fourth embodiment will be mainly described. Furthermore, in the sixth embodiment, descriptions of the commonalities between the fourth embodiment and the sixth embodiment will be omitted.

[0060] The acquisition unit 120 acquires the user's schedule information from the storage unit 110 or an external device. An example of the schedule information is shown below.

[0061] 9 is a diagram showing an example of schedule information according to the sixth embodiment. For example, the acquiring unit 120 acquires schedule information 500 from the storage unit 110. The schedule information 500 is information indicating future plans.

[0062] The acquisition unit 120 also acquires the notification determination table from the storage unit 110 or an external device. An example of the notification determination table is shown below.

[0063] FIG. 10 is a diagram showing an example of a notification determination table according to the sixth embodiment. For example, the notification determination table 112 is stored in the storage unit 110. The notification determination table 112 is also referred to as notification determination information. The notification determination table 112 is information for determining whether or not to encourage a break based on the duration of the concentration state and a schedule. Specifically, the notification determination table 112 has fields for duration of concentration state, schedule, and whether or not to notify. The field for duration of concentration state indicates the duration of the concentration state. The field for schedule indicates the contents of the schedule. The field for whether or not to notify registers information indicating whether or not to notify a user to take a break.

[0064] The control unit 130 determines whether to prompt the user to take a break using multiple determination results, the current time, the schedule information 500, and the notification determination table 112. For example, assume that multiple determination results indicate that the user has been in a state of concentration for three hours. The current time is 10:30. The control unit 130 identifies an upcoming appointment as a "meeting" based on the current time and the schedule information 500. The control unit 130 determines to prompt the user to take a break using multiple determination results (i.e., a three-hour state of concentration), the identified "meeting" appointment, and the notification determination table 112. The control unit 130 prompts the user to take a break. For example, the control unit 130 transmits information prompting the user to take a break to the PC used by the user.

[0065] For example, if a user is in a state of concentration for three hours, the burden on the user is high. If the user then participates in a conference while in a high-burden state, the user's burden will increase even more. Therefore, the information processing device 100 prompts the user to take a break before the conference. By taking a break before the conference, the user's burden is reduced. Then, the user can participate in the conference with a reduced burden.

[0066] According to the sixth embodiment, the information processing device 100 can allow the user to take a break according to the schedule.

[0067] Modification of Embodiment 6 The acquisition unit 120 acquires physical condition information indicating the physical condition of the user from the storage unit 110 or an external device.

[0068] The acquisition unit 120 also acquires the notification determination table 112a from the storage unit 110 or an external device. An example of the notification determination table 112a is shown below.

[0069] 11 is a diagram showing an example of a notification determination table according to a modification of the sixth embodiment. For example, the notification determination table 112a is stored in the storage unit 110. The notification determination table 112a is in a state in which a physical condition item has been added to the notification determination table 112. The notification determination table 112a contains information for determining whether or not to encourage a break based on the physical condition, the duration of the concentration state, and the schedule.

[0070] The control unit 130 determines whether to prompt the user to take a break using the physical condition information, the multiple determination results, the current time, the schedule information 500, and the notification determination table 112a. For example, assume that the physical condition information indicates poor health. Assume that the multiple determination results indicate that the user has been in a state of concentration for two hours. Assume that the current time is 10:30. Based on the current time and the schedule information 500, the control unit 130 identifies that the upcoming appointment is a "meeting." Using the physical condition information (i.e., poor health), the multiple determination results (i.e., a two-hour state of concentration), the identified appointment "meeting," and the notification determination table 112, the control unit 130 determines that the user should take a break. The control unit 130 prompts the user to take a break.

[0071] According to the modification of the sixth embodiment, the information processing device 100 can allow the user to take a rest in consideration of the user's physical condition.

[0072] The above describes a case where a user's reaction is determined using video. Alternatively, a user's reaction may be determined using biometric information. The processing will be described in detail. The acquisition unit 120 acquires the user's biometric information. For example, the acquisition unit 120 acquires the user's biometric information from a biometric information acquisition device. Here, the biometric information may include electroencephalograms, cerebral blood flow, electrocardiograms, heart rate, pulse waves, electrooculography, pupil diameter, electromyography, electrodermal activity, blood pressure, body temperature, respiratory rate, and pulse rate. The biometric information may also include blinking and gaze. When the biometric information is pulse rate, the biometric information acquisition device is a smartwatch or the like. The biometric information acquisition device is a reaction measurement device, and the biometric information is reaction information. The control unit 130 causes a stimulus generator to output a stimulus different from the current stimulus. The control unit 130 acquires the user's biometric information from the biometric information acquisition device. The control unit 130 determines whether the user has reacted based on the biometric information before the stimulus is output and the biometric information after the stimulus is output. The user response detection process is a process of determining whether the user has responded based on the biometric information before the stimulus is output and the biometric information after the stimulus is output. For example, if the biometric information is a pulse rate, the control unit 130 determines whether the user has responded based on the pulse rate before the stimulus is output and the pulse rate after the stimulus is output. If the pulse rate changes, the control unit 130 determines that the user has responded. The information processing device 100 does not continuously acquire the biometric information. Therefore, the information processing device 100 can reduce the user's resistance.

[0073] The acquisition unit 120 may acquire biometric information of the user in a normal state from the storage unit 110 or an external device. The biometric information of the normal state is measured in advance. The control unit 130 causes the stimulus generator to output a stimulus different from the current stimulus. The control unit 130 acquires the user's biometric information from the biometric information acquisition device. The control unit 130 determines whether the user has reacted based on the biometric information of the normal state and the biometric information after the stimulus has been output. For example, if the biometric information is a pulse, the control unit 130 determines whether the user has reacted based on the pulse rate in a normal state and the pulse rate after the stimulus has been output. If the pulse rate in a normal state and the pulse rate after the stimulus has been output are different, the control unit 130 determines that the user has reacted. In this way, the information processing device 100 can improve the accuracy of determination by using the biometric information of the user in a normal state.

[0074] The user's voice may also be used to determine the user's reaction. The process will be described in detail. The control unit 130 causes the stimulus generator to output a stimulus different from the current stimulus. The control unit 130 acquires the user's voice signal from the microphone 200. The microphone 200 may be a microphone built into the information processing device 100 or a microphone external to the information processing device 100. The microphone is a microphone. The microphone 200 is a reaction measurement device. The signal acquired by the microphone 200 is reaction information. The control unit 130 performs a user reaction detection process based on the voice signal. That is, the control unit 130 determines whether the user has reacted based on the voice signal. For example, if the voice based on the voice signal is a voice generated when responding to a stimulus such as "ah," the control unit 130 determines that the user has reacted. In this way, the information processing device 100 does not acquire biometric information, thereby reducing the user's resistance.

[0075] The acquisition unit 120 may acquire normal utterance information indicating the content of what the user normally utters from the storage unit 110 or an external device. The normal utterance information is created in advance. The control unit 130 causes the stimulus generator to output a stimulus different from the current stimulus. The control unit 130 acquires the user's voice signal from the microphone 200. The control unit 130 determines whether the user has reacted based on the voice signal and the normal utterance information. If the content of the voice indicated by the voice signal is not included in the normal utterance information, the control unit 130 determines that the user has reacted. In this way, the information processing device 100 can improve the accuracy of determination by using the normal utterance information.

[0076] Furthermore, the user's reaction may be determined using the user's key operation sound. The process will be described in detail. The control unit 130 causes the stimulus generator to output a stimulus different from the current stimulus. The control unit 130 acquires a sound signal from the microphone 200, which has been continuously receiving the user's key operation sound since before the stimulus was output. The microphone 200 is a reaction measurement device. The signal acquired by the microphone 200 is reaction information. The control unit 130 performs a user reaction detection process based on the sound signal. That is, the control unit 130 determines whether the user has reacted based on the audio signal. For example, if the sound signal indicates that the key operation sound has stopped after the stimulus was output, the control unit 130 determines that the user has reacted. In this way, the information processing device 100 does not acquire biometric information, thereby reducing the user's resistance.

[0077] The acquisition unit 120 may acquire the user's normal key operation pause time from the storage unit 110 or an external device. The normal key operation pause time is measured in advance. The control unit 130 causes the stimulus generator to output a stimulus different from the current stimulus. The control unit 130 acquires a sound signal from the microphone 200, which has been continuously receiving the user's key operation sound since before the stimulus was output. The control unit 130 determines whether the user has responded based on the sound signal and the normal key operation pause time. For example, the normal key operation pause time is set to 5 seconds. For example, if the sound signal indicates that the key operation sound has stopped for 10 seconds after the stimulus was output, the control unit 130 determines that the user has responded. In this way, the information processing device 100 can improve the accuracy of the determination by using the normal key operation pause time.

[0078] Each of the embodiments and the modified examples of each of the embodiments may be realized by a plurality of devices. For example, each of the embodiments and the modified examples of each of the embodiments may be realized by a server and a smartphone.

[0079] The embodiments and modifications thereof are merely examples, and various modifications are possible within the scope of the present disclosure. Furthermore, the features of the embodiments and modifications thereof can be combined with each other as appropriate.

[0080] REFERENCE SIGNS LIST 100 Information processing device, 101 Processor, 102 Volatile storage device, 103 Non-volatile storage device, 110 Storage unit, 111 Output sound table, 111a Output sound table, 112 Notification determination table, 112a Notification determination table, 120 Acquisition unit, 130 Control unit, 200 Microphone, 300 Speaker, 400 Camera, 500 Schedule information.

Claims

1. An information processing device that communicates with at least one of a stimulus generating device and a response measuring device present in a space in which a user is present, or that includes at least one of the stimulus generating device and the response measuring device, comprising: an acquisition unit that acquires information indicating a stimulus being output to the space; and a control unit that outputs a stimulus different from the stimulus from the stimulus generating device, acquires response information indicating the user's response to the stimulus from the response measuring device, and determines whether the user has responded based on the response information.

2. An information processing device that communicates with at least one of a stimulus generating device and a camera present in a space in which a user is present, or that includes at least one of the stimulus generating device and the camera, comprising: an acquisition unit that acquires information indicating a stimulus being output to the space; and a control unit that outputs a stimulus different from the stimulus from the stimulus generating device, acquires an image from the camera that is filming the user, and determines whether the user has reacted based on the image.

3. The information processing device of claim 2, wherein the stimulus generating device is a speaker, the acquisition unit acquires an environmental sound signal indicating the environmental sound of the space, and the control unit outputs a sound different from the environmental sound indicated by the environmental sound signal from the speaker, acquires video from the camera capturing the user, and determines whether the user has reacted based on the video.

4. The information processing device according to claim 3, wherein the control unit notifies the user of the determination result.

5. The information processing device of claim 3, wherein the control unit outputs a sound from the speaker to determine whether the user is in a shallow state of concentration, and if the user does not react, determines that the user is in a shallow state of concentration, outputs a sound different from the sound to determine whether the user is in a shallow state of concentration from the speaker, acquires video from the camera capturing the user, and, based on the video, determines that the user is in a deep state of concentration if the user does not react.

6. The information processing device according to claim 5, wherein the control unit urges the user to take a break when the user is in a state of deep concentration.

7. The information processing device described in claim 3, wherein the acquisition unit acquires age information indicating the age of the user, and the control unit outputs from the speaker a sound that is different from the environmental sound indicated by the environmental sound signal and that corresponds to the age of the user.

8. The information processing device described in claim 3, wherein the acquisition unit acquires information indicating that the user has poor hearing, and the control unit outputs from the speaker a sound that is different from the environmental sound indicated by the environmental sound signal and is intended for people with poor hearing.

9. The information processing device according to claim 3, wherein the acquisition unit acquires habit information indicating the habits of the user, and the control unit, when making the determination, determines whether the user has responded by taking into account the habits of the user.

10. The information processing device according to any one of claims 3 to 9, wherein the control unit periodically determines the concentration state of the user.

11. The information processing device described in claim 10, wherein the acquisition unit acquires an estimated break time, which is the time a break is required, and the control unit executes, based on the multiple judgment results, a process to prompt the user to take a break if the estimated break time or the state of concentration continues, or a process to output a sound from the speaker to elicit a response from the user.

12. The information processing device described in claim 10, wherein the acquisition unit acquires an estimated break time, which is the time when the user needs to take a break, and the control unit executes, based on the multiple judgment results, a process to prompt the user to take a break if the estimated break time or the state of concentration continues, or a process to output a sound from the speaker to elicit a response from the user.

13. The information processing device described in claim 10, wherein the acquisition unit acquires notification determination information, which is information for determining whether or not to encourage the user to take a break, based on the user's schedule information and the duration of the concentration state and the schedule; and the control unit uses multiple determination results, the current time, the schedule information, and the notification determination information to determine whether or not to encourage the user to take a break, and if it is determined that the user should be encouraged to take a break, encourages the user to take a break.

14. The information processing device described in claim 10, wherein the acquisition unit acquires schedule information of the user, physical condition information indicating the user's physical condition, and notification determination information that is information for determining whether or not to encourage the user to take a break based on the physical condition, the duration of the concentration state, and the schedule; and the control unit uses the physical condition information, multiple determination results, the current time, the schedule information, and the notification determination information to determine whether or not to encourage the user to take a break, and if it is determined that the user should be encouraged to take a break, encourages the user to take a break.

15. The information processing device of claim 2, wherein the stimulus generating device is a light, the acquisition unit acquires the illuminance of the space in which the user is present, the control unit controls the light so that the illuminance is changed, acquires video from the camera capturing the user, and determines whether the user has reacted based on the video.

16. The information processing device of claim 2, wherein the stimulus generating device is a light, the acquisition unit acquires the color temperature of the space in which the user is present, the control unit controls the light so that the color temperature is changed, acquires video from the camera capturing the user, and determines whether the user has reacted based on the video.

17. The information processing device of claim 2, wherein the stimulus generating device is a projector, the acquisition unit acquires the direction in which an image displayed in the space in which the user is present moves, the control unit controls a device that controls the projector so that the direction is changed, acquires an image from the camera that is photographing the user, and determines whether the user has reacted based on the image.

18. The information processing device of claim 2, wherein the stimulus generating device is an air conditioner, the acquisition unit acquires the temperature of the space in which the user is present, the control unit controls the air conditioner to change the temperature, acquires video from the camera capturing the user, and determines whether the user has reacted based on the video.

19. The information processing device of claim 2, wherein the stimulus generating device is an air conditioner, the acquisition unit acquires the direction of the air being output by the air conditioner, the control unit controls the air conditioner to change the direction, acquires video from the camera capturing the user, and determines whether the user has reacted based on the video.

20. The information processing device of claim 2, wherein the stimulus generating device is an odor generating device, the acquisition unit acquires information indicating the odor being output in the space in which the user is present, the control unit controls the odor generating device so as to change the odor, acquires video from the camera capturing the user, and determines whether the user has reacted based on the video.

21. An information processing device that communicates with a stimulus generating device and a biometric information acquiring device present in a space in which a user is present, comprising: an acquisition unit that acquires information indicating a stimulus being output to the space and acquires the user's biometric information from the biometric information acquiring device; and a control unit that outputs a stimulus different from the stimulus from the stimulus generating device, acquires the user's biometric information from the biometric information acquiring device, and determines whether the user has reacted based on the biometric information before the different stimulus is output and the biometric information after the different stimulus is output.

22. The information processing device described in claim 21, wherein the acquisition unit acquires biometric information of the user in a normal state, and the control unit causes the stimulus generating device to output a stimulus different from the stimulus, acquires the user's biometric information from the biometric information acquisition device, and determines whether the user has reacted based on the biometric information in the normal state and the biometric information after the different stimulus has been output.

23. An information processing device that communicates with at least one of a stimulus generating device and a microphone present in a space in which a user is present, or that includes at least one of the stimulus generating device and the microphone, comprising: an acquisition unit that acquires information indicating a stimulus being output to the space; and a control unit that causes the stimulus generating device to output a stimulus different from the stimulus, acquires the user's voice signal from the microphone, and determines whether the user has reacted based on the voice signal.

24. The information processing device described in claim 23, wherein the acquisition unit acquires normal speech information indicating what the user normally says, and the control unit causes the stimulus generating device to output a stimulus different from the stimulus, acquires the user's voice signal from the microphone, and determines whether the user has responded based on the voice signal and the normal speech information.

25. An information processing device that communicates with at least one of a stimulus generating device and a microphone present in a space in which a user is present, or that includes at least one of the stimulus generating device and the microphone, comprising: an acquisition unit that acquires information indicating a stimulus being output to the space; and a control unit that causes the stimulus generating device to output a stimulus different from the stimulus, acquires a sound signal from the microphone that has been continuously receiving key operation sounds from the user since before the different stimulus was output, and determines whether the user has reacted based on the sound signal.

26. The information processing device according to claim 25, wherein the acquisition unit acquires a normal key operation stop time of the user, and the control unit causes the stimulus generation device to output a stimulus different from the stimulus, acquires a sound signal from the microphone into which the user's key operation sound has been continuously input since before the different stimulus was output, and determines whether the user has reacted based on the sound signal and the normal key operation stop time.

27. A determination method in which an information processing device that communicates with at least one of a stimulus generating device and a response measuring device present in a space in which a user is present, or that includes at least one of the stimulus generating device and the response measuring device, acquires information indicating a stimulus being output to the space, causes the stimulus generating device to output a stimulus different from the stimulus, acquires response information indicating the user's response to the stimulus from the response measuring device, and determines whether the user has responded based on the response information.

28. A determination method in which one or more devices that communicate with at least one of a stimulus generating device and a camera present in a space in which a user is present, or that include at least one of the stimulus generating device and the camera, acquire information indicating a stimulus being output in the space, output a stimulus different from the stimulus from the stimulus generating device, acquire video from the camera filming the user, and determine whether or not the user has reacted based on the video.

29. A determination method in which an information processing device that communicates with a stimulus generating device and a biometric information acquiring device present in a space in which a user is present, acquires information indicating a stimulus being output to the space, acquires the user's biometric information from the biometric information acquiring device, causes the stimulus generating device to output a stimulus different from the stimulus, acquires the user's biometric information from the biometric information acquiring device, and determines whether the user has reacted based on the biometric information before the different stimulus is output and the biometric information after the different stimulus is output.

30. A determination method in which an information processing device that communicates with at least one of a stimulus generating device and a microphone present in a space in which a user is present, or that includes at least one of the stimulus generating device and the microphone, acquires information indicating a stimulus being output to the space, causes the stimulus generating device to output a stimulus different from the stimulus, acquires an audio signal of the user from the microphone, and determines whether the user has responded based on the audio signal.

31. A determination method in which an information processing device that communicates with at least one of a stimulus generating device and a microphone present in a space in which a user is present, or that includes at least one of the stimulus generating device and the microphone, acquires information indicating a stimulus being output to the space, causes the stimulus generating device to output a stimulus different from the stimulus, acquires a sound signal from the microphone that has continuously input the user's key operation sound since before the different stimulus was output, and determines whether the user has reacted based on the sound signal.

32. A determination program that causes an information processing device that communicates with at least one of a stimulus generating device and a response measuring device present in a space in which a user is present, or that includes at least one of the stimulus generating device and the response measuring device, to execute the following processes: acquire information indicating a stimulus being output to the space; output a stimulus different from the stimulus from the stimulus generating device; acquire response information indicating the user's response to the stimulus from the response measuring device; and determine whether or not the user has responded based on the response information.

33. A determination program that causes an information processing device that communicates with at least one of a stimulus generating device and a camera present in a space in which a user is present, or that includes at least one of the stimulus generating device and the camera, to execute the following processes: acquire information indicating a stimulus being output in the space; output a stimulus different from the stimulus from the stimulus generating device; acquire video from the camera that is filming the user; and determine whether or not the user has reacted based on the video.

34. A determination program that causes an information processing device that communicates with a stimulus generating device and a biometric information acquiring device present in a space in which a user is present to execute the following processes: acquire information indicating a stimulus being output to the space; acquire the user's biometric information from the biometric information acquiring device; output a stimulus different from the stimulus from the stimulus generating device; acquire the user's biometric information from the biometric information acquiring device; and determine whether the user has reacted based on the biometric information before the different stimulus is output and the biometric information after the different stimulus is output.

35. A determination program that causes an information processing device that communicates with at least one of a stimulus generating device and a microphone present in a space in which a user is present, or that includes at least one of the stimulus generating device and the microphone, to execute the following processes: acquire information indicating a stimulus being output to the space; output a stimulus different from the stimulus from the stimulus generating device; acquire the user's voice signal from the microphone; and determine whether the user has responded based on the voice signal.

36. A determination program that causes an information processing device that communicates with at least one of a stimulus generating device and a microphone present in a space in which a user exists, or that includes at least one of the stimulus generating device and the microphone, to execute the following processes: acquire information indicating a stimulus being output to the space; output a stimulus different from the stimulus from the stimulus generating device; acquire a sound signal from the microphone that has continuously input the user's key operation sound since before the different stimulus was output; and determine whether the user has reacted based on the sound signal.

Images