Information processing device and information processing method

Abstract

This information processing device (100, 200, 300, 400) comprises: an imaging information acquisition unit (101) that acquires imaging information of a driver (3) of a vehicle (1); and a stimulation behavior detection unit (104) that detects, on the basis of the imaging information acquired by the imaging information acquisition unit (101), a stimulation behavior including a behavior of the driver (3) to give a pain stimulation to the driver (3).

Description

Information processing device and information processing method

[0001] The present disclosure relates to an information processing device and an information processing method.

[0002] Conventionally, a driving mode switching device that switches a driving mode of a vehicle between automatic driving and manual driving has been disclosed (see, for example, Patent Document 1). This driving mode switching device is configured to determine a driver's drowsiness level based on the degree to which the driver's eyelids are open while driving the vehicle, and to switch the driving mode between automatic driving and manual driving based on the determination result.

[0003] Japanese Patent Application Laid-Open No. 2020-013248

[0004] Generally, when the degree of eyelid opening is smaller than normal, the driver's physical condition has a significant impact on driving. Therefore, even if a change in the driver's physical condition is detected after a change in physical condition that has a significant impact on driving occurs, it may be difficult to provide effective driving assistance to the driver.

[0005] The present disclosure was made in response to the recognition of the above-mentioned problem, and aims to provide an information processing device and an information processing method that can detect signs of changes in the physical condition of a vehicle driver that affect the driving operation.

[0006] The information processing device according to the present disclosure is characterized by including an imaging information acquisition unit that acquires imaging information of a vehicle driver, and a stimulating behavior detection unit that detects stimulating behavior, including behavior by the driver to inflict pain stimulation on himself, based on the imaging information acquired by the imaging information acquisition unit.

[0007] The information processing device of the present disclosure can detect signs of a change in the driver's physical condition that will affect the driver's driving operation by detecting stimulating behavior, including behavior by the driver to inflict pain stimulation on himself, when the driver of a vehicle becomes aware of a change in his physical condition that will affect the driver's driving operation.

[0008] 1 is a side view showing a vehicle according to embodiment 1. FIG. 2 is a block diagram showing a schematic configuration of an information processing system according to embodiment 1. FIG. 3 is a block diagram showing an example of a hardware configuration of an information processing device according to embodiment 1. FIG. 4 is a block diagram showing an example of a hardware configuration of an information processing device according to embodiment 1. FIG. 5 is a flowchart showing an example of processing performed by the information processing device according to embodiment 1. FIG. 6 is a block diagram showing a schematic configuration of an information processing system according to embodiment 2. FIG. 7 is a flowchart showing an example of processing performed by the information processing device according to embodiment 2. FIG. 8 is a graph showing an example of change over time in the alertness of a driver. FIG. 9 is a block diagram showing a schematic configuration of an information processing system according to embodiment 3. FIG. 10 is a flowchart showing an example of processing performed by the information processing device according to embodiment 3. FIG. 11 is a flowchart showing an example of processing performed by the information processing device according to embodiment 4.

[0009] Hereinafter, embodiments according to the present disclosure will be described in detail with reference to the drawings. Embodiment 1. First, a schematic configuration of an information processing system according to embodiment 1 will be described with reference to FIGS. 1 and 2. FIG. 1 is a side view showing a vehicle according to embodiment 1, and FIG. 2 is a block diagram showing a schematic configuration of the information processing system according to embodiment 1. The information processing system according to embodiment 1 is a system for detecting signs of changes in the physical condition of a driver 3 who drives a vehicle 1, which may affect the driving operation of the driver 3. As shown in FIGS. 1 and 2, the information processing system according to embodiment 1 includes an imaging device 10, an external device 20, and an information processing device 100, which are electrically connected wirelessly or by wire so as to be able to communicate with each other.

[0010] The imaging device 10 captures an image of the driver 3 of the vehicle 1. For example, the imaging device 10 is held in the vehicle body 2 of the vehicle 1 and captures an image of a specific range within the cabin R of the vehicle 1, including the driver 3, to generate imaging information indicating visual information within the cabin R, including the driver 3. For example, the imaging device 10 has an imaging element (image sensor) and a lens (not shown), and generates imaging information by converting light captured through the lens into an electrical signal using the imaging element. The imaging device 10 has an imaging element such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. For example, the imaging device 10 is disposed within the cabin R of the vehicle 1 and captures an image of the cabin R at a preset frame rate to generate imaging information of a range including the face of the driver 3 seated in the driver's seat. The imaging device 10 outputs the generated imaging information to the information processing device 100 .

[0011] Note that the imaging of the driver 3 by the imaging device 10 may be any device that generates imaging information that indicates the visual information of the driver 3, and the imaging device 10 may be a camera that generates imaging information based on visible light, an infrared camera that generates imaging information based on infrared light, a thermography camera that generates imaging information based on infrared light, or a millimeter-wave imaging radar that generates imaging information by detecting reflected waves of output millimeter waves.

[0012] The external device 20 performs a specific process based on the information input from the information processing device 100. For example, the external device 20 is held in the vehicle body 2 of the vehicle 1, and is configured to function as an alarm device that issues visual or audio alerts to occupants of the vehicle 1 or persons outside the vehicle 1 based on the information input from the information processing device 100. Furthermore, for example, the external device 20 is held in the vehicle body 2 of the vehicle 1, and functions as a driving control device that controls the driving of the vehicle 1 based on the information input from the information processing device 100. Specifically, the external device 20 is configured to function as a driving control device that assists the driver in driving operations by controlling one or both of the moving speed and moving direction of the vehicle 1.

[0013] The information processing device 100 includes an image capture information acquisition unit 101 and a stimulating behavior detection unit 104 .

[0014] The imaging information acquisition unit 101 acquires imaging information from the imaging device 10 that captures an image of the driver 3 of the vehicle 1. For example, the imaging information acquisition unit 101 acquires imaging information that indicates changes in the appearance of the driver 3 over time at a preset frame rate.

[0015] The stimulating behavior detection unit 104 detects stimulating behaviors of the driver 3, including behaviors for giving himself / herself a pain stimulus, based on the imaging information acquired by the imaging information acquisition unit 101. For example, the stimulating behavior detection unit 104 detects, as the stimulating behaviors of the driver 3, pain stimulating behaviors for giving himself / herself a pain stimulus and non-pain stimulating behaviors for giving himself / herself a stimulus other than a pain stimulus. Generally, changes in a driver's physical condition that affect the driver's driving operation include an abnormal physiological state, such as when a driver's level of alertness decreases due to drowsiness, or an abnormal psychological state, such as when a driver has a panic attack. Generally, when a driver becomes aware of a change in their physical condition that affects their driving operation while driving, they may voluntarily perform a stimulating behavior that gives themselves a stimulus in order to suppress the change in their physical condition.

[0016] The above-described stimulating behavior may occur before the driver exhibits noticeable appearance characteristics resulting from a change in physical condition that significantly affects driving, such as closing the eyes, looking down, or leaning the body. In other words, the above-described stimulating behavior can be treated as a sign of a change in the driver's physical condition that will affect the driver's driving performance. The information processing device 100 according to the first embodiment is configured to detect the above-described stimulating behavior as a sign of a change in the driver's physical condition that will affect the driver's driving performance.

[0017] Pain stimulation behavior is, for example, behavior in which the driver 3 inflicts a pain sensation on himself / herself by hitting himself / herself, specifically, behavior in which the driver 3 inflicts a pain sensation on himself / herself by repeatedly hitting his / her face, cheek, head, or any other body part or multiple parts thereof with his / her hand at a frequency of twice or more per second, behavior in which the driver 3 inflicts a pain sensation on himself / herself by pinching himself / herself with his / her hand, etc. Non-pain stimulation behavior is, for example, behavior in which the driver 3 inflicts a pain sensation on himself / herself by rotating his / her head or any other body part at a frequency of once or more per two seconds, behavior in which the driver 3 inflicts a pain sensation on himself / herself by taking a deep breath, behavior in which the driver 3 inflicts a pain sensation on himself / herself by massaging his / her body, etc.

[0018] The stimulating behavior detection unit 104 extracts feature amounts of the imaging information acquired by the imaging information acquisition unit 101, and detects the stimulating behavior of the driver 3 based on changes over time of the extracted feature amounts. For example, the stimulating behavior detection unit 104 learns using a plurality of data sets including imaging information of the driver captured over a specific period and information indicating whether the driver is engaging in pain stimulating behavior or non-pain stimulating behavior, and detects the stimulating behavior of the driver 3 using a trained model that outputs information indicating whether the driver is engaging in pain stimulating behavior or non-pain stimulating behavior based on input of the imaging information acquired by the imaging information acquisition unit 101.

[0019] Next, the hardware configuration of the information processing device 100 will be described with reference to Figures 3 and 4. Figure 3 is a diagram showing an example of the hardware configuration of the information processing device 100, and Figure 4 is a diagram showing an example of the hardware configuration of the information processing device 100 that is different from that shown in Figure 3. For example, as shown in Figure 3, the information processing device 100 is configured as a computer having a processor 100a, a memory 100b, and an I / O port 100c, and is configured so that the processor 100a reads and executes a program stored in the memory 100b. The memory 100b is configured, for example, by RAM (Random Access Memory), ROM (Read Only Memory), flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), etc.

[0020] 4, the information processing device 100 is configured as a computer that includes a processing circuit 100d, which is dedicated hardware, and an I / O port 100c, and executes a program. The processing circuit 100d is configured, for example, by a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. Each function of the information processing device 100 is realized by the processor 100a or the processing circuit 100d, which is dedicated hardware, executing a program. The information processing device 100 may also include hardware other than those described above, such as a hardware timer.

[0021] Next, details of the processing performed by the information processing device 100 will be described with reference to Fig. 1, Fig. 2, and Fig. 5. Fig. 5 is a flowchart showing an example of the processing performed by the information processing device 100 according to the first embodiment. As shown in Fig. 5, when the information processing device 100 starts the processing, it first acquires imaging information from the imaging device 10 (step ST01). In this processing, the information processing device 100 acquires, via the imaging information acquisition unit 101, imaging information generated by the imaging device 10 by capturing an image of the driver operating the vehicle 1.

[0022] After performing the process of step ST01, the information processing device 100 determines whether or not a stimulating behavior of the driver 3 has been detected (step ST04). In this process, the information processing device 100 determines whether or not a stimulating behavior of the driver 3 has been detected by the stimulating behavior detection unit 104 based on the imaging information acquired by the imaging information acquisition unit 101.

[0023] If the information processing device 100 detects the stimulating behavior of the driver 3 in the processing of step ST04 (YES in step ST04), it outputs the detection result to the external device 20 (step ST05). For example, in this processing, the information processing device 100 outputs information indicating that the stimulating behavior of the driver 3 has been detected to the external device 20 as an alarm device. For example, the external device 20 as an alarm device notifies the occupants that the stimulating behavior of the driver 3 has been detected based on the information from the information processing device 100. This allows the committee members other than the driver 3 to recognize that a change has occurred in the driver's physical condition, and enables them to take action, for example, by urging the driver 3 to stop the vehicle or by taking over driving duties from the driver 3. Also, for example, in this processing, the information processing device 100 outputs information indicating that the stimulating behavior of the driver 3 has been detected to the external device 20 as a driving control device. For example, the external device 20 as a driving control device controls one or both of the moving speed and moving direction of the vehicle 1 based on information from the information processing device 100 so as to assist the driver in driving operations.

[0024] If the information processing device 100 does not detect a stimulating behavior of the driver 3 in the processing of step ST04 (NO in step ST04), or if the information processing device 100 has performed the processing of step ST05, it determines whether or not the driver 3 has finished driving the vehicle 1 (step ST15). In this processing, the information processing device 100 determines whether or not the driver 3 has finished driving the vehicle 1 based on, for example, the state of the power switch of the vehicle 1, the state of the ignition switch, the position of the shift lever, etc.

[0025] If the information processing device 100 determines that the driver 3 has not finished driving in the processing of step ST15 (NO in step ST15), it returns the processing to step ST01, and if the driver 3 has finished driving in the processing of step ST15 (YES in step ST15), it ends the processing.

[0026] As described above, the information processing device 100 according to the first embodiment includes the imaging information acquisition unit 101 that acquires imaging information from the imaging device 10 that images the driver 3 of the vehicle 1, and the stimulating behavior detection unit 104 that detects stimulating behavior, including behavior by the driver 3 to inflict pain stimulation on himself, based on the imaging information acquired by the imaging information acquisition unit 101. With this configuration, the information processing device 100 according to the first embodiment can detect signs of a change in the physical condition of the driver 3 of the vehicle 1 before any noticeable external characteristics appear that are due to a change in the physical condition of the driver 3 that will affect the driving operation of the driver 3.

[0027] In the first embodiment, the information processing system is configured to, when the information processing device 100 detects the driver 3's stimulating behavior, cause the external device 20 to issue a notification as a notification device or cause the external device 20 to control the vehicle as a vehicle control device; however, this is not limited to this. The information processing device may be configured to issue both a notification and control the vehicle when the driver's stimulating behavior is detected, or is not necessarily limited to issuing a notification or controlling the vehicle when the driver's stimulating behavior is detected. For example, some drivers may have a habit of hitting parts of their body while driving. Therefore, the information processing system may be configured to detect the driver's stimulating behavior and, when the driver's physiological state is not normal, issue a notification and / or control the vehicle.

[0028] Embodiment 2 Next, an information processing system according to embodiment 2 will be described with reference to Fig. 1 and Fig. 6 to Fig. 8. The information processing system according to embodiment 2 differs from the information processing system according to embodiment 1 in the configuration for acquiring physiological information of the driver and making a notification based on the acquired physiological information, but the other configurations are the same, and the same configurations as those in embodiment 1 are assigned the same names and symbols as those in embodiment 1, and descriptions thereof will be omitted.

[0029] Fig. 6 is a block diagram showing a schematic configuration of an information processing system according to embodiment 2. The information processing system according to embodiment 2 is a system for detecting signs of changes in physical condition that may affect the driving operation of a driver 3 who drives a vehicle 1, and for making a notification based on the detection results. As shown in Fig. 6, the information processing system according to embodiment 2 includes an imaging device 10, a physiological information detection device 30, a notification device 40, and an information processing device 200, which are electrically connected wirelessly or by wire so as to be able to communicate with each other.

[0030] The physiological information detection device 30 generates physiological information indicating the physiological state of the driver 3, and outputs the generated physiological information to the information processing device 200. For example, the physiological information detection device 30 is configured with a contact or non-contact physiological sensor, and generates physiological information indicating one or more of the following as the physiological state of the driver 3: heart rate, pulse rate, blood pressure, respiratory rate, sweat rate, body temperature, and oxygen saturation, and outputs the generated physiological information to the information processing device 200. Specifically, the physiological information detection device 30 is disposed on a steering wheel (not shown), and emits light toward the palms of the driver 3, converts the reflected light from the palms into an electrical signal, and outputs the electrical signal, thereby outputting physiological information, which is a signal indicating the heart rate of the driver 3, to the information processing device 200.

[0031] The notification device 40 issues a specific notification based on information input from the information processing device 100. For example, the notification device 40 is held in the vehicle body 2 of the vehicle 1, and issues a specific notification to the occupants of the vehicle 1 or persons outside the vehicle 1 based on information input from the information processing device 200. Specifically, the notification device 40 is configured to function as one or more of the following devices: a display device having a liquid crystal display panel that issues specific visual information to the occupants based on information input from the information processing device 200; an audio notification device having a speaker that issues specific audio information to the occupants based on information input from the information processing device 200; and a communication device that communicates with a device outside the vehicle 1 based on information input from the information processing device 200 and transmits specific information to the device outside the vehicle 1 to issue a specific notification to persons outside the vehicle 1.

[0032] The information processing device 200 includes an imaging information acquisition unit 101 , a physiological information acquisition unit 102 , a physiological state determination unit 103 , a stimulating behavior detection unit 104 , and a notification determination unit 105 .

[0033] The physiological information acquisition unit 102 acquires physiological information indicating the physiological state of the driver 3. For example, the physiological information acquisition unit 102 acquires, from the physiological information detection device 30, physiological information indicating one or more of the following as the physiological state of the driver 3: heart rate, pulse rate, blood pressure, respiratory rate, amount of sweat, body temperature, and oxygen saturation.

[0034] The physiological state determination unit 103 determines which of a plurality of preset states the physiological state of the driver 3 is in. For example, the physiological state determination unit 103 determines which of a plurality of preset states the physiological state of the driver 3 is in, based on the physiological information acquired by the physiological information acquisition unit 102. Specifically, the physiological state determination unit 103 calculates the alertness of the driver 3 based on a change in the heart rate of the driver 3 indicated by the physiological information acquired by the physiological information acquisition unit 102, and determines which of a plurality of preset states the alertness as a physiological state of the driver 3 is in, based on the calculated alertness. For example, the physiological state determination unit 103 learns using multiple data sets including the difference between the driver's heart rate in a normal state and the heart rate at a specific time point, and information indicating which of multiple preset states the driver's alertness is in, and determines which of multiple preset states the driver's alertness as a physiological state is in using a trained model that outputs which of multiple preset states the driver's alertness is in based on the input physiological information acquired by the physiological information acquisition unit 102.

[0035] For example, the physiological state determination unit 103 determines whether the physiological state of the driver 3 is in one of a plurality of states including a first state, a second state that has a smaller impact on driving than the first state, or a normal state, based on the physiological information acquired by the physiological information acquisition unit 102. The first state is, for example, a physical condition state that has a relatively large impact on the driving of the driver 3, specifically, the first state is a state in which the alertness of the driver 3 has decreased to such an extent that it has a significant impact on the driving of the driver 3. The second state is, for example, a physical condition state that has a smaller impact on the driving of the driver 3 than the first state, specifically, the second state is a state in which the alertness of the driver 3 is between the first state and the normal state, which is the driver's normal state, and is slightly lower than the normal state.

[0036] The notification determination unit 105 determines whether to issue a notification based on the determination result by the physiological state determination unit 103 and the detection result by the stimulating behavior detection unit 104. For example, when the physiological state of the driver 3 is in the first state, or when the physiological state of the driver 3 is in the second state and a stimulating behavior of the driver 3 is detected, the notification determination unit 105 determines to issue a notification based on the determination result by the physiological state determination unit 103 and the detection result by the stimulating behavior detection unit 104. For example, when the physiological state of the driver 3 is in the first state, it is desirable for the driver 3 to suspend driving. Therefore, the notification determination unit 105 determines to issue a notification to prompt the driver 3 to suspend driving or to prompt another occupant of the vehicle 1 to take over driving, based on the fact that the physiological state of the driver 3 is in the first state. Furthermore, for example, when the physiological state of the driver 3 is in the second state and a stimulating behavior of the driver 3 is detected, it is also desirable for the driver 3 to suspend driving. Therefore, based on the fact that the physiological state of the driver 3 is in the second state and that the driver 3's stimulating behavior has been detected, the notification determination unit 105 decides to issue a notification to encourage the driver 3 to stop driving, or to encourage other occupants of the vehicle 1 to take over driving.

[0037] Furthermore, for example, when the physiological state of the driver 3 is in the first state, or when the physiological state of the driver 3 is in the second state and a stimulating behavior of the driver 3 is detected, the notification determination unit 105 determines to issue a notification to a person outside the vehicle 1 based on the determination result by the physiological state determination unit 103 and the detection result by the stimulating behavior detection unit 104. Specifically, when the physiological state of the driver 3 is in the first state, or when the physiological state of the driver 3 is in the second state and a stimulating behavior of the driver 3 is detected, the notification determination unit 105 determines to issue a notification to a person outside the vehicle 1 by transmitting information indicating that the physiological state of the driver 3 is in a state that has a significant impact on driving to a device outside the vehicle 1 based on the determination result by the physiological state determination unit 103 and the detection result by the stimulating behavior detection unit 104. By issuing such a notification, for example, an outside person who receives the notification can dispatch personnel to the vicinity of the vehicle 1 to assist the driver 3.

[0038] The hardware configuration of the information processing device 200 according to the second embodiment is similar to the hardware configuration of the information processing device 100 according to the first embodiment, and therefore a description thereof will be omitted.

[0039] Next, details of the processing performed by the information processing device 200 will be described with reference to Fig. 1 and Fig. 6 to Fig. 8. Fig. 7 is a flowchart showing an example of the processing performed by the information processing device 200 according to embodiment 2. Note that part of the processing performed by the information processing device 200 according to embodiment 2 is similar to the processing performed by the information processing device 100 according to embodiment 1, and therefore, a description of the processing similar to embodiment 1 will be omitted. As shown in Fig. 7, when the information processing device 200 starts processing, it first acquires imaging information from the imaging device 10 (step ST01).

[0040] After completing the process of step ST01, the information processing device 200 acquires physiological information from the physiological information detection device 30 (step ST02). In this process, the information processing device 200 acquires, for example, physiological information for determining the wakefulness level of the driver 3.

[0041] After performing the process of step ST02, the information processing device 200 determines whether the physiological state of the driver 3 is in the first state (step ST03). In this process, the information processing device 200 determines, for example, whether the alertness level as the physiological state of the driver 3 is in the first state.

[0042] 8 is a graph showing an example of changes in a driver's alertness over time. The graph shows a state in which the driver's alertness decreases over time from a high alertness state DL5 to a low alertness state DL0. For example, DL5 is a fully alert state, characterized by appearances such as rapid blinking, wide eye opening, and good posture. DL4 is a state of weakly decreased alertness, characterized by noticeable blinking with long eyelid closure times and a slightly smaller degree of eyelid opening. DL3 is a state of moderately decreased alertness, characterized by appearances such as blinking with long eyelid closure times, an even smaller degree of eyelid opening than DL4, and yawning and changes in facial expression depending on the individual, resulting in a slightly sluggish driving performance. DL2 is a state of severely reduced alertness, characterized by external characteristics such as repeated loss of posture and recovery, frequent blinking with long eyelid closure periods, for example, frequent long-term eyelid closure periods of approximately 3 seconds, etc., which causes microsleep and significantly affects driving, requiring recovery of alertness to continue driving. DL1 is a lightly dozing state, characterized by external characteristics such as loss of posture and completely closed eyelids, and is a state in which the driver can respond to visual or audio information but has difficulty continuing driving. DL0 is a deeply dozing state, characterized by external characteristics such as persistent loss of posture and completely closed eyelids, is unable to respond to visual or audio information, and is unable to continue driving. Normally, a driver's alertness does not decrease to DL0 while driving a vehicle, but it may decrease to DL0 due to severe fatigue or the effects of alcohol. For example, the information processing device 200 is configured to determine DL0 to DL2 as a first state using the physiological state determination unit 103.

[0043] If the physiological state of the driver 3 is not the first state in the process of step ST03 (NO in step ST03), the information processing device 200 determines whether or not a stimulating behavior of the driver 3 has been detected (step ST04). In this process, the information processing device 200 determines, by the stimulating behavior detection unit 104, whether or not a stimulating behavior of the driver 3 that stimulates himself or herself has been detected.

[0044] When the information processing device 200 detects the stimulating behavior of the driver 3 in the processing of step ST04 (YES in step ST04), it determines whether the physiological state of the driver 3 is in the second state (step ST08). In this processing, for example, as shown in FIG. 8 , DL3 and DL4 are the second state, and DL5 is the normal state, and the information processing device 200 determines whether the physiological state of the driver 3 is in the second state or the normal state using the physiological state determination unit 103. Note that the thresholds for determining the first state, the second state, and the normal state are not limited to those described above, and the information processing device 200 may make the determination based on other thresholds. Furthermore, it is also possible to identify the driver who drives the vehicle and set different thresholds for each driver, or to determine the first state, the second state, and the normal state based on an index other than the level of wakefulness.

[0045] If the physiological state of the driver 3 is in the first state in the processing of step ST03 (YES in step ST03), and if the physiological state of the driver 3 is in the second state in the processing of step ST08 (YES in step ST08), the information processing device 200 issues a notification (step ST14). In this processing, the information processing device 200 issues a notification that the driver 3 is in a physical condition that will affect driving operation, based on the fact that the physiological state of the driver 3 is in the first state and that the physiological state of the driver 3 is in the second state and a stimulating behavior of the driver 3 has been detected.

[0046] If the information processing device 200 does not detect a stimulating behavior of the driver 3 in the processing of step ST04 (NO in step ST04), if the physiological state of the driver 3 is not in the second state in the processing of step ST08 (NO in step ST08), or if the information processing device 200 performs the processing of step ST14, it determines whether the driver 3 has finished driving (step ST15).

[0047] If the information processing device 200 determines that the driver 3 has not finished driving in the processing of step ST15 (NO in step ST15), it returns the processing to step ST01, and if the driver 3 has finished driving in the processing of step ST15 (YES in step ST15), it ends the processing.

[0048] As described above, the information processing device 200 according to the second embodiment includes an imaging information acquisition unit 101 that acquires imaging information of the driver 3 of the vehicle 1, a stimulating behavior detection unit 104 that detects stimulating behavior, including behavior by the driver 3 to inflict pain stimulation on himself, based on the imaging information acquired by the imaging information acquisition unit 101, a physiological information acquisition unit 102 that acquires physiological information indicating the physiological state of the driver 3, a physiological state determination unit 103 that determines which of a plurality of pre-set states the physiological state of the driver 3 is in, based on the physiological information acquired by the physiological information acquisition unit 102, and a notification determination unit 105 that determines whether to issue a notification based on the determination result by the physiological state determination unit 103 and the detection result by the stimulating behavior detection unit 104.

[0049] With this configuration, the information processing device 200 issues a notification based on the physiological state of the driver 3 in addition to whether or not a stimulating behavior by the driver 3 has been detected, and therefore the reliability of the information to be notified can be improved compared to when the notification is issued based only on whether or not a stimulating behavior by the driver 3 has been detected. For example, by issuing a notification based on the physiological state of the driver 3 in addition to whether or not a stimulating behavior by the driver 3 has been detected, it is possible to suppress a notification based on the driver hitting a part of their body while driving due to the driver's habit, even though the driver's physiological state is normal, and thereby improve the reliability of the information to be notified.

[0050] In addition, the information processing device 200 according to the second embodiment is configured to determine whether the driver's physiological state is in one of a plurality of states including a first state, a second state that has less impact on driving than the first state, and a normal state, and to decide to issue an alert based on the determination result by the physiological state determination unit and the detection result by the stimulating behavior detection unit when the driver's physiological state is in the first state, or when the driver's physiological state is in the second state and the driver's stimulating behavior is detected.

[0051] Configured in this manner, the information processing device 200 can issue an alert when a physical condition that affects the driving operation of the driver 3 is detected, and when signs of a physical condition that will affect the driving operation of the driver 3 are detected.

[0052] In the second embodiment, the information processing device 200 has the physiological state determination unit 103 determine the physiological state of the driver 3 based on the physiological information acquired by the physiological information acquisition unit 102. However, this is not limiting. The physiological state determination unit may be configured to determine the physiological state of the driver 3. For example, the physiological state determination unit may be configured to determine the physiological state of the driver based on the imaging information acquired by the imaging information acquisition unit, or may be configured to determine the physiological state of the driver based on the physiological information acquired by the physiological information acquisition unit and the imaging information acquired by the imaging information acquisition unit. For example, the physiological state determination unit may be configured to determine which of a plurality of preset states the driver's level of alertness as a physiological state is based on a change in a feature amount extracted from the imaging information acquired by the imaging information acquisition unit, utilizing the fact that external features change in response to a change in the driver's level of alertness. Alternatively, the physiological state determination unit may be configured to determine the driver's physiological state based on the driver's facial expression, posture, facial direction, degree of eyelid opening, frequency of reflexes such as yawning, etc.

[0053] Embodiment 3 Next, an information processing system according to embodiment 3 will be described with reference to Figures 1, 9, and 10. The information processing system according to embodiment 3 differs from the information processing system according to embodiment 2 in the configuration for assisting the driver in driving operations, but the other configurations are the same, and the same names and symbols as those in embodiment 2 will be used to designate the same configurations as those in embodiment 2, and descriptions thereof will be omitted.

[0054] Fig. 9 is a block diagram showing a schematic configuration of an information processing system according to embodiment 3. The information processing system according to embodiment 3 is a system for detecting signs of changes in physical condition that affect the driving operation of a driver 3 who drives a vehicle 1, and for providing an alert based on the detection results and controlling the vehicle to assist the driver in performing the driving operation. As shown in Fig. 9, the information processing system according to embodiment 3 includes an imaging device 10, a physiological information detection device 30, an alert device 40, a driving control device 50, and an information processing device 300, which are electrically connected wirelessly or by wire so as to be able to communicate with each other.

[0055] The driving control device 50 controls one or both of the speed and direction of travel of the vehicle 1. For example, the driving control device 50 automatically controls one or both of the speed and direction of travel of the vehicle 1 based on information for controlling the vehicle 1 from the information processing device 300 so as to assist the driver 3 in driving operations.

[0056] The information processing device 300 includes an imaging information acquisition unit 101 , a physiological information acquisition unit 102 , a physiological state determination unit 103 , a stimulating behavior detection unit 104 , a driving support determination unit 106 , and a control information generation unit 107 .

[0057] The driving assistance determination unit 106 determines whether to control the vehicle 1 to assist the driving operation of the driver 3, based on the determination result by the physiological state determination unit 103 and the detection result by the stimulating behavior detection unit 104. For example, when the physiological state of the driver 3 is in the first state, or when the physiological state of the driver 3 is in the second state and a stimulating behavior of the driver 3 is detected, the driving assistance determination unit 106 determines to control the vehicle 1 to assist the driving operation of the driver 3.

[0058] The control information generation unit 107 generates driving assistance information for controlling the driving control device 50 based on the determination result by the driving assistance determination unit 106, and outputs the generated driving assistance information to the driving control device 50. For example, the control information generation unit 107 outputs driving assistance information, which is information for changing the autonomous driving level by the driving control device 50, to the driving control device 50 based on the determination result by the driving assistance determination unit 106.

[0059] The hardware configuration of the information processing device 300 according to the third embodiment is similar to the hardware configuration of the information processing device 100 according to the first embodiment, and therefore a description thereof will be omitted.

[0060] Next, details of the processing performed by the information processing device 300 will be described with reference to Fig. 1 and Fig. 9 to Fig. 10. Fig. 10 is a flowchart showing an example of the processing performed by the information processing device 300 according to embodiment 3. Note that part of the processing performed by the information processing device 300 according to embodiment 3 is similar to the processing performed by the information processing device 200 according to embodiment 2, and therefore, a description of the processing similar to embodiment 2 will be omitted.

[0061] As shown in FIG. 10 , when the information processing device 300 starts processing, it first acquires imaging information from the imaging device 10 (step ST01). After performing the processing of step ST01, the information processing device 300 acquires physiological information from the physiological information detection device 30 (step ST02). After performing the processing of step ST02, the information processing device 300 determines whether the physiological state of the driver 3 is in the first state (step ST03). If the physiological state of the driver 3 is not in the first state in the processing of step ST03 (NO in step ST03), the information processing device 300 determines whether a stimulating behavior of the driver 3 has been detected (step ST04). If the information processing device 300 detects a stimulating behavior of the driver 3 in the processing of step ST04 (YES in step ST04), it determines whether the physiological state of the driver 3 is in the second state (step ST08).

[0062] If the physiological state of the driver 3 is in the first state in the processing of step ST03 (YES in step ST03), or if the physiological state of the driver 3 is in the second state in the processing of step ST08 (YES in step ST08), the information processing device 300 generates driving assistance information (step ST09). In this processing, if the physiological state of the driver 3 is in the first state, or if the physiological state of the driver 3 is in the second state and a stimulating behavior of the driver 3 is detected, the information processing device 300 determines to control the vehicle 1 to assist the driving operation of the driver 3, and generates driving assistance information for controlling the vehicle 1 to assist the driving operation of the driver 3 by the driving control device 50. For example, in this processing, the information processing device 300 determines to assist the driving operation of the driver 3 by increasing the autonomous driving level by the driving control device 50.

[0063] After completing the process of step ST09, the information processing device 300 outputs the driving assistance information generated in the process of step ST09 (step ST10). In this process, the information processing device 300 outputs the driving assistance information generated in the process of step ST09 to the driving control device 50.

[0064] After completing the process of step ST10, the information processing device 300 issues a notification (step ST14). In this process, the information processing device 300 determines to issue a notification via the notification device 40 that the driving control device 50 will assist the driver 3 in driving operation, and outputs information to the notification device 40 to cause the notification device 40 to issue a notification based on the determination result by the physiological state determination unit 103 and the detection result by the stimulating behavior detection unit 104.

[0065] If the information processing device 300 does not detect a stimulating behavior of the driver 3 in the processing of step ST04 (NO in step ST04), if the physiological state of the driver 3 is not in the second state in the processing of step ST08 (NO in step ST08), or if the information processing device 300 performs the processing of step ST14, it determines whether the driver 3 has finished driving (step ST15).

[0066] If the information processing device 300 determines that the driver 3 has not finished driving in the processing of step ST15 (NO in step ST15), it returns the processing to step ST01, and if the driver 3 has finished driving in the processing of step ST15 (YES in step ST15), it ends the processing.

[0067] As described above, the information processing device 300 according to the third embodiment includes an imaging information acquisition unit 101 that acquires imaging information of the driver 3 of the vehicle 1, a stimulating behavior detection unit 104 that detects stimulating behavior, including behavior by the driver 3 to inflict pain stimulation on himself, based on the imaging information acquired by the imaging information acquisition unit 101, a physiological information acquisition unit 102 that acquires physiological information indicating the physiological state of the driver 3, a physiological state determination unit 103 that determines which of a plurality of preset states the physiological state of the driver 3 is in, based on the physiological information acquired by the physiological information acquisition unit 102, and a driving assistance determination unit 106 that determines whether or not to control the vehicle 1 to assist the driving operation of the driver 3, based on the determination result by the physiological state determination unit 103 and the detection result by the stimulating behavior detection unit 104.

[0068] With this configuration, the information processing device 300 controls the vehicle 1 to assist driving operation based on the physiological state of the driver 3 in addition to whether a stimulating behavior by the driver 3 has been detected, and therefore can prevent the vehicle 1 from being controlled when driving operation assistance is not required, compared to when the vehicle 1 is controlled based only on whether a stimulating behavior by the driver 3 has been detected. For example, by providing a notification based on the physiological state of the driver 3 in addition to whether a stimulating behavior by the driver 3 has been detected, it can prevent the vehicle 1 from being controlled to assist driving operation based on the driver's habit of hitting a part of their body while driving, even though their physiological state is normal.

[0069] In addition, the information processing device 300 according to the third embodiment determines whether the physiological state of the driver 3 is in one of a plurality of states including a first state, a second state which has less impact on driving than the first state, and a normal state, and decides to control the vehicle 1 to assist the driving operation of the driver 3 when the physiological state of the driver 3 is in the first state, or when the physiological state of the driver 3 is in the second state and stimulating behavior of the driver 3 is detected.

[0070] In the third embodiment, the information processing device 300 has the physiological state determination unit 103 determine the physiological state of the driver 3 based on the physiological information acquired by the physiological information acquisition unit 102, but this is not limiting. The physiological state determination unit may be configured to be able to determine the physiological state of the driver, and for example, similar to the second embodiment, the physiological state determination unit may be configured to determine the physiological state of the driver based on the imaging information acquired by the imaging information acquisition unit, or may be configured to determine the physiological state of the driver based on the physiological information acquired by the physiological information acquisition unit and the imaging information acquired by the imaging information acquisition unit.

[0071] Furthermore, the information processing device 300 may be configured to perform control and notification to assist the driving operation of the driver 3 based on the length of time the stimulating behavior continues. For example, when the physiological state of the driver 3 is in a first state and the information processing device 300 has detected a stimulating behavior of the driver 3, the information processing device 300 may be configured to perform control and notification to assist the driving operation of the driver 3 if the stimulating behavior of the driver 3 continues for a first time or more, and when the physiological state of the driver 3 is in a normal state and the information processing device 300 has detected a stimulating behavior of the driver 3, the information processing device 300 may be configured to perform control and notification to assist the driving operation of the driver 3 if the stimulating behavior of the driver 3 continues for a second time or more that is longer than the first time. Specifically, the information processing device 300 may be configured to perform control and alert to assist the driver 3 in driving operations when the driver's physiological state is in a first state and the driver's stimulating behavior is detected, and when the driver's stimulating behavior continues for 20 seconds or more, and to perform control and alert to assist the driver 3 in driving operations when the driver's physiological state is in a normal state and the driver's stimulating behavior is detected, and when the driver's stimulating behavior continues for 1 minute or more.

[0072] Embodiment 4 Next, an information processing system according to embodiment 4 will be described with reference to Figures 1, 9, and 11. The information processing system according to embodiment 4 differs from the information processing system according to embodiment 3 in the configuration for assisting the driver in driving operations, but the other configurations are the same, and the same names and symbols as those in embodiment 3 will be used and descriptions thereof will be omitted.

[0073] The information processing system according to the third embodiment is a system for detecting signs of changes in physical condition that may affect the driving operation of a driver 3 who drives a vehicle 1, and for providing an alert based on the detection results and controlling the vehicle to assist the driver in the driving operation. As shown in Fig. 9, the information processing system according to the fourth embodiment includes an imaging device 10, a physiological information detection device 30, an alert device 40, a driving control device 50, and an information processing device 400, which are electrically connected wirelessly or by wire so as to be able to communicate with each other.

[0074] The information processing device 400 includes an imaging information acquisition unit 101 , a physiological information acquisition unit 102 , a physiological state determination unit 103 , a stimulating behavior detection unit 104 , a driving support determination unit 206 , and a control information generation unit 207 .

[0075] The driving assistance determination unit 206 determines whether to control the vehicle 1 to assist the driving operation of the driver 3, based on the determination result by the physiological state determination unit 103 and the detection result by the stimulating behavior detection unit 104. For example, when the physiological state of the driver 3 is in the first state, or when the physiological state of the driver 3 is in the second state and a stimulating behavior of the driver 3 is detected, the driving assistance determination unit 206 determines to control the vehicle 1 to assist the driving operation of the driver 3. Furthermore, for example, when the physiological state of the driver 3 is in the first state or when the physiological state of the driver 3 is in the second state and a stimulating behavior of the driver 3 is detected, the driving assistance determination unit 206 determines to control the vehicle 1 so as to reduce the burden of the driving operation of the driver 3 compared to when the physiological state of the driver 3 is in a normal state and a stimulating behavior of the driver is detected.

[0076] The control information generation unit 207 generates first driving assistance information or second driving assistance information for controlling the driving control device 50 based on the determination result by the driving assistance determination unit 206, and outputs one of the generated first driving assistance information and second driving assistance information to the driving control device 50. For example, the control information generation unit 107 generates first driving assistance information, which is information for setting the autonomous driving level by the driving control device 50 to a first level that is equal to or greater than 1, or second driving assistance information, which is information for setting the autonomous driving level by the driving control device 50 to a second level that is higher than the first level, based on the determination result by the driving assistance determination unit 106, and outputs the generated second driving assistance information to the driving control device 50.

[0077] The hardware configuration of the information processing device 400 according to the fourth embodiment is similar to the hardware configuration of the information processing device 100 according to the first embodiment, and therefore a description thereof will be omitted.

[0078] Next, details of the processing performed by the information processing device 400 will be described with reference to Figures 1, 9, and 11. Figure 11 is a flowchart showing an example of the processing performed by the information processing device 400 according to embodiment 4. Note that part of the processing performed by the information processing device 400 according to embodiment 4 is similar to the processing performed by the information processing device 300 according to embodiment 3, and therefore, a description of the processing similar to embodiment 3 will be omitted.

[0079] 11 , when the information processing device 400 starts processing, it first acquires imaging information from the imaging device 10 (step ST01). After performing the processing of step ST01, the information processing device 400 acquires physiological information from the physiological information detection device 30 (step ST02). After performing the processing of step ST02, the information processing device 400 determines whether the physiological state of the driver 3 is in the first state (step ST03). If the physiological state of the driver 3 is not in the first state in the processing of step ST03 (NO in step ST03), the information processing device 400 determines whether a stimulating behavior of the driver 3 has been detected (step ST04).

[0080] When the information processing device 400 detects a stimulating behavior of the driver 3 in the processing of step ST04 (YES in step ST04), it generates first driving assistance information (step ST06). In this processing, the information processing device 400 generates the first driving assistance information, which is information for setting the autonomous driving level of the driving control device 50 to a first level that is 1 or higher. For example, if the autonomous driving level of the vehicle 1 is 0, i.e., an autonomous driving level corresponding to fully manual driving, in the state before the processing of step ST06, the information processing device 400 generates the first driving assistance information, which is information for setting the adaptive cruise control (ACC), the autonomous emergency braking (AEB), and the lane keep assist system (LKAS) to on.

[0081] After performing the process of step ST06, the information processing device 400 outputs the first driving assistance information generated in the process of step ST06 (step ST07). Generally, the stimulating behavior of the driver 3 is a movement different from the driving operation, and therefore the stimulating behavior of the driver 3 may affect the driving operation of the driver 3. Therefore, in the process of step ST07, the information processing device 400 outputs the first driving assistance information generated in the process of step ST06 to the driving control device 50, thereby causing the driving control device 50 to control the vehicle 1 to assist the driving operation of the driver 3 in the vehicle 1 in order to suppress the effect of the stimulating behavior of the driver 3 on the driving operation.

[0082] After performing the process of step ST07, the information processing device 400 determines whether the physiological state of the driver 3 is in the second state (step ST08).

[0083] If the physiological state of the driver 3 is in the first state in the processing of step ST03 (YES in step ST03), and if the physiological state of the driver 3 is in the second state in the processing of step ST08 (YES in step ST08), the information processing device 400 generates second driving support information (step ST12). In this processing, the information processing device 400 determines to control the vehicle 1 by the driving control device 50 so as to reduce the burden of the driving operation on the driver 3 compared to the control of the vehicle 1 performed by the driving control device 50 in the processing of step ST07, and generates second driving support information for causing the driving control device 50 to perform the control. In other words, in this processing, when the physiological state of the driver 3 is in the first state and when the physiological state of the driver 3 is in the second state and a stimulating behavior of the driver 3 is detected, the information processing device 400 determines to control the vehicle so as to reduce the burden of the driving operation on the driver 3 compared to the case when the physiological state of the driver 3 is in the normal state and a stimulating behavior of the driver 3 is detected.

[0084] For example, in the processing of step ST12, the information processing device 400 generates second driving assistance information that is information for setting the autonomous driving level to a second level that is higher than the first level, which is the autonomous driving level of the control of the vehicle 1 performed by the driving control device 50 in the processing of step ST07. Specifically, in the processing of step ST12, the information processing device 400 generates second driving assistance information that is information for controlling the driving control device 50 to move the vehicle 1 to the side of the road and then stop the vehicle 1 by causing the driving control device 50 to perform fully autonomous driving.

[0085] After performing the process of step ST12, the information processing device 400 outputs the second driving assistance information generated in the process of step ST12 (step ST13). In this process, the information processing device 400 outputs the generated second driving assistance information to the driving control device 50, thereby causing the driving control device 50 to control the vehicle 1.

[0086] After performing the process of step ST13, the information processing device 400 makes a notification (step ST14).

[0087] If the information processing device 400 does not detect a stimulating behavior of the driver 3 in the processing of step ST04 (NO in step ST04), if the physiological state of the driver 3 is not in the second state in the processing of step ST08 (NO in step ST08), or if the information processing device 400 performs the processing of step ST14, it determines whether the driver 3 has finished driving (step ST15).

[0088] If the information processing device 400 determines that the driver 3 has not finished driving in the processing of step ST15 (NO in step ST15), it returns the processing to step ST01, and if the driver 3 has finished driving in the processing of step ST15 (YES in step ST15), it ends the processing.

[0089] As described above, the information processing device 400 according to the fourth embodiment is configured to determine to control the vehicle so as to reduce the burden of the driving operation on the driver 3 when the physiological state of the driver 3 is in the first state and when the physiological state of the driver 3 is in the second state and a stimulating behavior of the driver 3 is detected, compared to when the physiological state of the driver 3 is normal and a stimulating behavior of the driver 3 is detected. With this configuration, the information processing device 400 can suppress the influence of the stimulating behavior of the driver 3 on the driving operation even when the physiological state of the driver 3 is normal, and can reduce the burden of the driving operation on the driver 3 when the physiological state of the driver 3 is not normal.

[0090] In any of the above-described embodiments, the information processing device may be configured to determine, when a pain-stimulating behavior of the driver is detected, to control the vehicle using the driving control device so as to reduce the burden on the driver in driving operations compared to when a pain-stimulating behavior of the driver is not detected and a non-pain-stimulating behavior of the driver is detected. For example, pain-stimulating behavior is more likely to occur when there is a change in physical condition that has a greater impact on the driver's driving operations than non-pain-stimulating behavior. Therefore, by controlling the vehicle so as to reduce the burden on the driver in driving operations when a pain-stimulating behavior is detected, it is possible to suppress the impact of the driver's physical condition on driving operations.

[0091] Furthermore, in any of the above-described embodiments, the information processing device may be configured to store the acquired information, the processing results, and the information output to the outside as time-series data in a memory unit (not shown) provided in the information processing device, and may be configured to store this information in the memory unit in association with the vehicle positioning information acquired by a positioning device (not shown).

[0092] Furthermore, in any of the above-described embodiments, the information processing device may include some or all of the other configurations of the information processing system, or some of the configurations of the information processing device may be provided in another device that is communicatively connected to the information processing device, or the information processing device may be communicatively connected to the other configurations of the information processing system via another device, computer, or communication network not shown.

[0093] In addition, the present disclosure allows for free combination of the respective embodiments, modification of any of the components of the respective embodiments, or omission of any of the components of the respective embodiments.

[0094] The information processing device according to the present disclosure can be used, for example, to provide effective driving assistance to a vehicle driver by detecting signs of changes in the driver's physical condition that affect the driver's driving operation.

[0095] 1 Vehicle, 2 Vehicle body, 3 Driver, 10 Imaging device, 20 External device, 30 Physiological information detection device, 40 Notification device, 50 Driving control device, 100 Information processing device, 101 Imaging information acquisition unit, 102 Physiological information acquisition unit, 103 Physiological state determination unit, 104 Stimulation behavior detection unit, 105 Notification determination unit, 106 Driving assistance determination unit, 107 Control information generation unit, 200 Information processing device, 206 Driving assistance determination unit, 207 Control information generation unit, 300 Information processing device, 400 Information processing device, R Vehicle compartment.

Claims

1. An information processing device comprising: an imaging information acquisition unit that acquires imaging information of a vehicle driver; and a stimulating behavior detection unit that detects stimulating behavior, including behavior by the driver intended to inflict pain stimulation on himself / herself, based on the imaging information acquired by the imaging information acquisition unit.

2. The information processing device according to claim 1, further comprising: a physiological state determination unit that determines which of a plurality of pre-set states the driver's physiological state is in based on the imaging information acquired by the imaging information acquisition unit; and a notification determination unit that determines whether or not to issue a notification based on the determination result by the physiological state determination unit and the detection result by the stimulating behavior detection unit.

3. An information processing device as described in claim 1, characterized in that it comprises: a physiological information acquisition unit that acquires physiological information indicating the physiological state of the driver; a physiological state determination unit that determines which of a plurality of pre-set states the driver's physiological state is in based on the physiological information acquired by the physiological information acquisition unit; and an alarm determination unit that determines whether to issue an alarm based on the determination result by the physiological state determination unit and the detection result by the stimulating behavior detection unit.

4. The information processing device according to claim 1, further comprising: a physiological state determination unit that determines which of a plurality of pre-set states the driver's physiological state is in based on the imaging information acquired by the imaging information acquisition unit; and a driving assistance determination unit that determines whether or not to control the vehicle to assist the driver's driving operation based on the determination result by the physiological state determination unit and the detection result by the stimulating behavior detection unit.

5. An information processing device as described in claim 1, characterized in that it comprises: a physiological information acquisition unit that acquires physiological information indicating the physiological state of the driver; a physiological state determination unit that determines which of a plurality of pre-set states the driver's physiological state is in based on the physiological information acquired by the physiological information acquisition unit; and a driving assistance determination unit that determines whether to control the vehicle to assist the driver in driving operations based on the determination result by the physiological state determination unit and the detection result by the stimulating behavior detection unit.

6. The information processing device described in claim 2 or 3, characterized in that the physiological state determination unit determines whether the driver's physiological state is in a first state, a second state which has less impact on driving than the first state, or a normal state, and the notification determination unit decides to issue a notification based on the determination result by the physiological state determination unit and the detection result by the stimulating behavior detection unit when the driver's physiological state is in the first state, or when the driver's physiological state is in the second state and a stimulating behavior of the driver is detected.

7. The information processing device according to claim 4 or 5, characterized in that the physiological state determination unit determines whether the driver's physiological state is in a first state, a second state which has less impact on driving than the first state, or a plurality of states including a normal state, and the driving assistance determination unit decides to control the vehicle to assist the driver in driving operations when the driver's physiological state is in the first state, or when the driver's physiological state is in the second state and a stimulating behavior of the driver is detected.

8. The information processing device according to claim 7, characterized in that the driving assistance determination unit determines to control the vehicle so as to reduce the burden of driving operations on the driver when the driver's physiological state is in the first state and when the driver's physiological state is in the second state and a stimulating behavior of the driver is detected, compared to when the driver's physiological state is in the normal state and a stimulating behavior of the driver is detected.

9. The information processing device according to claim 4 or 5, characterized in that the stimulating behavior detection unit detects, as stimulating behaviors, pain stimulating behaviors in which the driver applies pain stimuli to himself / herself and non-pain stimulating behaviors in which the driver applies stimuli other than pain stimuli based on the imaging information acquired by the imaging information acquisition unit, and the driving assistance determination unit, when a pain stimulating behavior of the driver is detected, decides to control the vehicle so as to reduce the burden of driving operations on the driver compared to when a pain stimulating behavior of the driver is not detected and a non-pain stimulating behavior of the driver is detected.

10. An information processing device as claimed in any one of claims 1 to 5, characterized in that the stimulation behavior detection unit detects, based on the imaging information acquired by the imaging information acquisition unit, an action of the driver hitting part of himself as a pain stimulation behavior for inflicting a pain stimulation on himself.

11. An information processing method performed by a device having an image information acquisition unit and a stimulating behavior detection unit, comprising: a step in which the image information acquisition unit acquires image information of a vehicle driver; and a step in which the stimulating behavior detection unit detects stimulating behavior, including behavior by the driver intended to inflict a pain stimulus on himself, based on the image information acquired by the image information acquisition unit.

Images