State estimation system

Abstract

This state estimation system comprises an acquisition unit for acquiring heartbeat information relating to the heartbeat of the driver of a moving body and the skin potential of the driver, a computation unit for calculating a first index value based on the heartbeat information and a second index value based on the skin potential, and an estimation unit for estimating the state of the driver on the basis of the first index value and the second index value.

Description

State Estimation System

[0001] The present invention relates to a state estimation system.

[0002] In recent years, technologies for estimating a driver's state based on the driver's biological information have been used in vehicle driving assistance systems, etc. For example, a technology has been disclosed for estimating a driver's state (e.g., whether the driver is absentminded or not) based on the driver's line of sight and eye movement detected from a captured image of the driver.

[0003] Japanese Patent Application Laid-Open No. 2021-077135

[0004] However, it tends to take a long time from when a driver's state changes until a change in appearance, such as a change in line of sight, occurs. Furthermore, there tends to be relatively large individual differences in the changes in appearance that occur with changes in the driver's state. Therefore, there is room for improvement in the estimation speed and accuracy of methods for estimating a driver's state based on changes in the driver's appearance.

[0005] The present invention has been made in view of the above, and provides a state estimation system that can estimate a change in the driver's state quickly and with high accuracy.

[0006] A state estimation system according to one embodiment of the present invention includes an acquisition unit that acquires heart rate information relating to the heart rate of a driver of a mobile body and the driver's skin potential, a calculation unit that calculates a first index value based on the heart rate information and a second index value based on the skin potential, and an estimation unit that estimates the driver's state based on the first index value and the second index value.

[0007] According to the state estimation system of the present invention, it is possible to estimate changes in the driver's state quickly and with high accuracy.

[0008] FIG. 1 is a diagram showing an example of the configuration of a vehicle according to the first embodiment. FIG. 2 is a block diagram showing an example of the system configuration and hardware configuration of the vehicle according to the first embodiment. FIG. 3 is a diagram showing an example of the functional configuration of an information estimation system according to the first embodiment. FIG. 4 is a diagram showing an example of a state map according to the first embodiment. FIG. 5 is a diagram showing an example of the relationship between the state map and plots according to the first embodiment. FIG. 6 is a flowchart showing an example of processing in the state estimation system according to the first embodiment. FIG. 7 is a diagram showing an example of the functional configuration of a state estimation system according to the second embodiment. FIG. 8 is a diagram showing an example of a state map corrected by a correction unit according to the second embodiment.

[0009] Exemplary embodiments of the present invention are disclosed below. The configurations of the embodiments described below, as well as the actions, results, and advantages brought about by the configurations, are merely examples. The present invention can be realized with configurations other than those disclosed in the following embodiments, and it is possible to obtain at least one of the various advantages based on the basic configurations and derivative advantages.

[0010] (First embodiment) Fig. 1 is a diagram showing an example of the configuration of a vehicle 1 according to a first embodiment. The vehicle 1 is an example of a moving body on which a state estimation system is mounted. The state estimation system according to this embodiment estimates the state of a driver 2 who is driving the vehicle 1.

[0011] The vehicle 1 according to this embodiment includes a Doppler sensor 11 , an air pressure sensor 12 , an FMCW (Frequency Modulated Continuous Wave) radar 13 , a camera 14 , a steering contact sensor 15 , and a vibration generator 16 .

[0012] The Doppler sensor 11 is a sensor that transmits radio waves toward the driver 2 and receives reflected waves from the driver 2 to obtain heartbeat information about the driver 2. The Doppler sensor 11 is disposed in the backrest 22 of the seat 21, transmits radio waves (transmitted waves) toward a predetermined part of the driver 2 (for example, the back near the heart), and receives the reflected waves of the transmitted waves reflected by the driver 2. The Doppler sensor 11 obtains information about the heartbeat (pulse) of the driver 2, such as the heartbeat interval, based on changes in the frequency of the transmitted and received radio waves (the frequency difference between the transmitted waves and the received waves). The frequency of the transmitted waves from the Doppler sensor 11 should be selected appropriately depending on the usage situation, but may be, for example, about 24 GHz.

[0013] Like the Doppler sensor 11, the air pressure sensor 12 is a sensor for acquiring heartbeat information of the driver 2. The air pressure sensor 12 is installed inside the seat back 22 and has an air bag (bladder) located near the position of the driver's 2 heart. The air pressure sensor 12 detects the movement of the body surface of the driver 2 based on the pressure acting on the air bag, and acquires heartbeat information based on the detection result. Note that, in this embodiment, a configuration including the Doppler sensor 11 and the air pressure sensor 12 is illustrated as an example of means for acquiring heartbeat information, but the means for acquiring heartbeat information is not limited to this. For example, a configuration using either the Doppler sensor 11 or the air pressure sensor 12 may be used, or a configuration using a sensor other than the Doppler sensor 11 and the air pressure sensor 12 may be used.

[0014] The FMCW radar 13 is a sensor capable of acquiring information relating to the presence or absence of the driver 2 and the body movements of the driver 2. The FMCW radar 13 illustrated here is disposed on the roof 31 inside the vehicle and can acquire distance information such as TOF (Time Of Flight) corresponding to the distance between the installation position of the FMCW radar 13 and the driver 2 (head or its vicinity). The presence or absence and the body movements of the driver 2 can be estimated based on the distance information acquired by the FMCW radar 13. The specific configuration of the FMCW radar 13 is not particularly limited, but may be, for example, one that utilizes electromagnetic waves with a frequency of about 60 GHz.

[0015] The camera 14 is a device that acquires image data of the driver 2 seated in the seat 21. The camera 14 illustrated here is placed near the boundary between the roof 31 and the windshield 32, and acquires image data including the face of the driver 2 from a position diagonally above and in front of the driver 2. By analyzing the image data, information regarding changes in the appearance of the driver 2, such as line of sight movement, eye movement, and body movement, can be acquired.

[0016] The steering wheel touch sensor 15 is a sensor disposed on the steering wheel 41 and capable of measuring the skin potential of the palm of the driver 2 gripping the steering wheel 41. The skin potential is an example of biological information relating to a physiological phenomenon other than the heart rate of the driver 2. The skin potential is a value related to, for example, the amount of electricity flowing through the skin, and changes depending on, for example, the amount of sweating from the skin. For example, the skin potential becomes higher as the amount of sweating from the palm of the driver 2 increases.

[0017] The vibration generator 16 is an example of a drive unit that exerts a physical action on the driver 2 seated in the seat 21. The vibration generator 16 illustrated here is a device that is disposed inside the backrest 22 and provides a massage effect or the like to the back of the driver 2. The specific configuration of the vibration generator 16 is not particularly limited, but it may, for example, utilize a pneumatic structure that generates vibrations using air pressure. The air bag described above may be the pneumatic structure.

[0018] 2 is a block diagram showing an example of a system configuration and a hardware configuration of the vehicle 1 according to the first embodiment. The vehicle 1 according to this embodiment includes a state estimation system 111 and a vehicle control system 112.

[0019] The state estimation system 111 is a system that estimates the state of the driver 2 of the vehicle 1, and includes a Doppler sensor 11, an air pressure sensor 12, an FMCW radar 13, a camera 14, a steering contact sensor 15, a vibration generator 16, a processor 121, and the like.

[0020] The processor 121 is an information processing device that performs various arithmetic processing according to a program, and is configured using, for example, a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), SSD (Solid State Drive), I / F (Interface), etc. The processor 121 loads programs stored in the ROM or SSD into the RAM, and executes arithmetic processing and control processing for estimating the state of the driver 2. The processor 121 transmits and receives various information to and from other devices via the I / F.

[0021] The processor 121 according to this embodiment executes processing to estimate the state of the driver 2 based on heart rate information acquired by at least one of the Doppler sensor 11 and the air pressure sensor 12, and skin potential acquired by the steering wheel contact sensor 15. At this time, TOF data acquired by the FMCW radar 13, image data acquired by the camera 14, etc. may also be used. The processor 121 also drives the vibration generator 16 in accordance with the estimation result, i.e., the state of the driver 2. State information indicating the state of the driver 2 estimated by the processor 121 (state estimation system 111) is output to the vehicle control system 112.

[0022] The vehicle control system 112 includes an ECU (Electronic Control Unit) 131, a drive mechanism 132, a braking mechanism 133, a steering mechanism 134, a user I / F 135, etc. The drive mechanism 132 is a mechanism including a drive source (engine, motor, etc.) of the vehicle 1. The braking mechanism 133 is a mechanism that decelerates and stops the vehicle 1. The steering mechanism 134 is a mechanism that changes the traveling direction of the vehicle 1. The user I / F 135 is a display, speaker, operation unit, etc. that are provided inside the vehicle. The ECU 131 is an information processing device that executes various processes to control the drive mechanism 132, the braking mechanism 133, the steering mechanism 134, the user I / F 135, etc. The ECU 131 according to this embodiment executes predetermined control using state information output from the state estimation system 111. The ECU 131 controls the drive mechanism 132, the braking mechanism 133, the steering mechanism 134, the user I / F 135, etc. so that a danger avoidance action is taken based on the state information. The danger avoidance action may be, for example, a warning to the driver 2, decelerating or stopping the vehicle 1, etc.

[0023] 3 is a diagram showing an example of the functional configuration of the state estimation system 111 according to the first embodiment. The state estimation system 111 according to the present embodiment includes an acquisition unit 201, a calculation unit 202, a storage unit 203, an estimation unit 204, an output unit 205, and a drive unit 206. These functional units are realized by cooperation between hardware elements and software elements (programs, etc.) of the state estimation system 111. Furthermore, at least one of these functional units may be configured by dedicated hardware (circuits, etc.).

[0024] The acquisition unit 201 acquires heart rate information related to the heart rate of the driver 2 of the vehicle 1 and biometric information related to physiological phenomena other than the heart rate of the driver 2. The acquisition unit 201 of this embodiment acquires, as the biometric information, the skin potential of the palm of the driver 2. In addition to the skin potential, the acquisition unit 201 may acquire, as the biometric information, information related to the driver 2's gaze movement, eye movement, body movement, brain waves, etc.

[0025] The calculation unit 202 calculates a first index value based on the heart rate information acquired by the acquisition unit 201. The first index value is, for example, an index value related to the emotion of the driver 2, and may be valence indicating the degree of pleasantness or unpleasantness of the emotion of the driver 2. Various methods can be used to calculate valence from the heart rate information. For example, valence can be calculated based on fluctuations in the heart rate interval (RRI: RR Interval). In this case, valence can be calculated such that the greater the time-series fluctuations in the heart rate interval, the stronger the negative (unpleasant) emotion. In the following description, the first index value will be described as valence, but the first index value is not limited to valence.

[0026] The calculation unit 202 also calculates a second index value based on the skin potential as biological information acquired by the acquisition unit 201. The second index value may be, for example, an index value relating to the degree of arousal of the driver 2, such as arousal indicating the degree of activity of the subject's central nervous system. Various methods can be used to calculate the arousal level from the skin potential. For example, the arousal level can be calculated such that the higher the skin potential (the greater the amount of sweating on the palm), the higher the arousal level. The skin potential can also be estimated from heart rate information. In the following description, the second index value will be described as arousal level, but the second index value is not limited to arousal level.

[0027] The storage unit 203 stores a state map M used to estimate the state of the driver 2 .

[0028] FIG. 4 is a diagram showing an example of the state map M of the first embodiment. As shown in FIG. 4, the state map M is a four-quadrant matrix with valence as the first axis and arousal as the second axis, and multiple regions R1 to R4 are set for each type of state of the driver 2. In this embodiment, a higher valence indicates a stronger positive (pleasant) emotion, and a lower valence indicates a stronger negative (unpleasant) emotion. Furthermore, a higher arousal value indicates a more lucid state of consciousness, and a lower arousal value indicates a more confused state of consciousness. Note that, although an example is shown here in which the horizontal axis is the first axis and the vertical axis is the second axis, the first and second axes may be reversed.

[0029] The state map M of this embodiment includes an origin O, a first quadrant, a second quadrant, a third quadrant, and a fourth quadrant. The origin O is a point based on the valence and arousal level calculated at a predetermined reference point in time, and in this case, is a point at which the valence and arousal level calculated at the reference point in time are set to zero. The reference point in time is a reference point in time when estimation of the state of the driver 2 begins, and may be, for example, a point in time when the driver 2 is seated in the seat 21 of the vehicle 1 in a stopped state and is recognized to be in a mentally and physically stable state. The first quadrant is a region where valence is higher and arousal level is higher than the origin O. The second quadrant is a region where valence is lower and arousal level is higher than the origin O. The third quadrant is a region where valence is lower and arousal level is lower than the origin O. The fourth quadrant is a region where valence is higher and arousal level is lower than the origin O.

[0030] The state map M of this embodiment includes a first absentminded state region R1 corresponding to a first absentminded state, a second absentminded state region R2 corresponding to a second absentminded state, a resting state region R3 corresponding to a resting state, and a normal driving state region R4 corresponding to a normal driving state. The first absentminded state is a absentminded state caused by a predetermined first factor. The first factor may be, for example, distraction. The second absentminded state is a absentminded state caused by a predetermined second factor different from the first factor. The second factor may be, for example, fatigue or drowsiness. The resting state is a state in which the driver 2 is not driving and is seated in a normal position in the seat 21. The normal driving state is a state in which the driver 2 is driving normally. Hereinafter, the first absentminded state region R1, the second absentminded state region R2, the resting state region R3, and the normal driving state region R4 may be referred to as region R1, region R2, region R3, and region R4, respectively.

[0031] In this embodiment, the first absentminded state region R1 does not include the origin O and is set to straddle the first, second, third, and fourth quadrants. The second absentminded state region R2 does not include the origin O and the first absentminded state region R1 and is set to straddle the third and fourth quadrants. The resting state region R3 does not include the first absentminded state region R1, the second absentminded state region R2, and the normal driving state region R4, but is set to include the origin O and straddle the first, second, third, and fourth quadrants. The normal driving state region R4 does not include the first absentminded state region R1, the second absentminded state region R2, the resting state region R3, and the origin O, but is set to straddle the first, second, third, and fourth quadrants. Note that the method of setting the above regions R1 to R4 is not limited to the above.

[0032] 3 , the estimation unit 204 estimates the state of the driver 2 based on the valence and arousal level that change over time and are calculated by the calculation unit 202. The estimation unit 204 estimates the state of the driver 2 based on, for example, which region (in this embodiment, the first absentminded state region R1, the second absentminded state region R2, the calm state region R3, or the normal driving state region R4) a plot on the state map M, which is determined based on the valence and arousal level acquired while driving the vehicle 1, belongs to.

[0033] FIG. 5 is a diagram illustrating an example of the relationship between the state map M and the plot P in the first embodiment. As shown in FIG. 5 , the position of the plot P on the state map M changes over time in response to changes in valence and arousal level during driving, i.e., changes in the driver's 2 heart rate information and biological information (such as skin potential). The state of the driver 2 is estimated based on whether the plot P belongs to the first absentminded state region R1, the second absentminded state region R2, the resting state region R3, or the normal driving state region R4 on the state map M. That is, if the plot P is located within the first absentminded state region R1, the driver 2 is estimated to be in the first absentminded state. If the plot P is located within the second absentminded state region R2, the driver 2 is estimated to be in the second absentminded state. If the plot P is located within the resting state region R3, the driver 2 is estimated to be in the resting state. If the plot P is located within the normal driving state region R4, the driver 2 is estimated to be in the normal driving state.

[0034] In this case, it is preferable that a threshold be set as a condition for determining that the driver 2 is in a certain state (e.g., the first absentminded state), such as the plot P remaining continuously within a region corresponding to that state (e.g., the first absentminded state region R1) for a predetermined period of time or more. Specifically, if the plot P remains continuously within the first absentminded state region R1 for a predetermined period of time or more, the driver 2 is determined to be in the first absentminded state. The same applies to the second absentminded state, the resting state, and the normal driving state. This makes it possible to prevent the state estimation result from changing too sensitively in response to instantaneous changes in the driver 2's heart rate, skin potential, etc.

[0035] Furthermore, it is preferable that the threshold for transitioning from the first absentminded state or the second absentminded state to the normal driving state be set higher than the threshold for transitioning from the normal driving state to the first absentminded state or the second absentminded state. For example, if a plot P that has remained in the first absentminded state region R1 or the second absentminded state region R2 for a predetermined time or longer transitions to the normal driving state region R4, it is determined that the driver has transitioned to the normal driving state if the plot P remains in the normal driving state region R4 for a time T1 or longer. Also, if a plot P that has remained in the normal driving state region R4 for a predetermined time or longer transitions to the first absentminded state region R1 or the second absentminded state region R2, it is determined that the driver has transitioned to the first absentminded state or the second absentminded state if the plot P remains in the first absentminded state region R1 or the second absentminded state region R2 for a time T2 or longer. In this case, time T1 is set to be longer than time T2. This allows a strict determination to return to a normal driving state once the vehicle has entered a careless or unsafe state, thereby improving safety.

[0036] The state may also be estimated taking into consideration the trajectory of the plot P on the state map M. For example, the threshold for transitioning to the first absentminded state may be changed based on whether or not the driver has passed through a resting state or a second absentminded state during the transition from the normal driving state to the first absentminded state. This allows the state of the driver 2 to be estimated with higher accuracy according to the progress of the change in the state of the driver 2.

[0037] 3 , the output unit 205 outputs the estimation result by the estimation unit 204, i.e., state information indicating the state of the driver 2. The output unit 205 outputs the state information to, for example, the vehicle control system 112. The state information is used, for example, in processing to realize danger avoidance behavior by the vehicle control system 112.

[0038] Based on the estimation result (state information) by the estimation unit 204, the driving unit 206 performs a predetermined physical action on the driver 2 (for example, massaging the back of the driver 2 using the vibration generating device 16).

[0039] 6 is a flowchart showing an example of processing in the state estimation system 111 of the first embodiment. In step S101, the acquisition unit 201 acquires heart rate information and biological information (skin potential, etc.), and in step S102, the calculation unit 202 calculates valence based on the heart rate information and calculates arousal level based on the biological information.

[0040] Then, in step S103, the estimation unit 204 sets the state map M based on the valence and arousal level at a reference time point. That is, the origin O and each of the regions R1 to R4 are set based on the valence and arousal level calculated at a reference time point (for example, when the driver is in a resting state before starting to drive). In step S104, the estimation unit 204 estimates the state of the driver 2 based on the position on the state map M of the plot P that corresponds to the current valence and arousal level (while the vehicle 1 is being driven). In step S105, the output unit 205 outputs state information indicating the estimation result by the estimation unit 204 to the vehicle control system 112, etc.

[0041] In step S106, the vehicle control system 112 that has acquired the state information determines whether the driver 2 is in the first absentminded state or the second absentminded state. If the driver 2 is not in either the first absentminded state or the second absentminded state (S106: No), this routine ends. If the driver 2 is in the first absentminded state or the second absentminded state (S106: Yes), the vehicle control system 112 executes safety control (risk avoidance action) in step S107. At this time, the drive unit 206 may take physical action on the driver 2, such as massaging.

[0042] As described above, according to this embodiment, the state of the driver 2 is estimated based on the first index value (valence) calculated based on heart rate information and the second index value (alertness) calculated based on biological information related to physiological phenomena other than heart rate (e.g., skin potential). That is, the state of the driver 2 is estimated based on internal changes, such as changes in the driver's 2 heart rate and physiological phenomena other than heart rate (e.g., sweating on the palms of the hands). Internal changes accompanying changes in the state of the driver 2 generally appear earlier than external changes such as gaze movement and eye movement, and there is relatively little individual variation. Therefore, according to this embodiment, it is possible to estimate changes in the state of the driver 2 quickly and with high accuracy.

[0043] Furthermore, in the above embodiment, the estimation unit 204 estimates the state of the driver 2 based on the first index value, the second index value, and the state map M, which is a four-quadrant matrix with the first index value on the first axis and the second index value on the second axis, and in which a plurality of regions are set for each type of state of the driver 2. This makes it possible to identify which of the plurality of states the driver 2 is in.

[0044] In the above embodiment, the state map M includes a first absentminded state region R1 indicating that the driver 2 is in a first absentminded state caused by a predetermined first factor. This makes it possible to detect when the driver 2 has entered the first absentminded state caused by the first factor, which is determined as a state in which the driver 2 is likely to cause an accident, and to prevent the occurrence of an accident.

[0045] In the above embodiment, the state map M includes a second absentminded state region R2 indicating that the driver 2 is in a second absentminded state caused by a predetermined second factor. This makes it possible to detect when the driver 2 has entered the second absentminded state caused by a second factor different from the first factor, which is determined to be a state in which the driver 2 is likely to cause an accident, and to more effectively prevent accidents from occurring.

[0046] Furthermore, in the above embodiment, the state map M includes an origin O based on the first index value (emotional valence) and the second index value (arousal level) calculated at a predetermined reference time point, a first quadrant which is a region where the first index value is higher than the origin O and the second index value is higher, a second quadrant which is a region where the first index value is lower than the origin O and the second index value is higher, a third quadrant which is a region where the first index value is lower than the origin O and the second index value is lower, and a fourth quadrant which is a region where the first index value is higher than the origin O and the second index value is lower, and a fourth quadrant which is a region where the first index value is higher than the origin O, and the first absentminded state region R1 is set to not include the origin O but to straddle the first, second, third, and fourth quadrants. This makes it possible to appropriately set the first absentminded state region R1 based on the state at the reference time point.

[0047] Furthermore, in the above embodiment, the state map M includes an origin O based on the first index value (emotional valence) and the second index value (arousal level) calculated at a predetermined reference time point, a first quadrant which is a region where the first index value is higher than the origin O and the second index value is higher, a second quadrant which is a region where the first index value is lower than the origin O and the second index value is higher, a third quadrant which is a region where the first index value is lower than the origin O and the second index value is lower, and a fourth quadrant which is a region where the first index value is higher than the origin O and the second index value is lower, and the second absentminded state region R2 does not include the origin O or the first absentminded state region R1, but is set to straddle the third and fourth quadrants. This makes it possible to appropriately set the second absentminded state region R2 based on the state at the reference time point.

[0048] In the above embodiment, the state map M includes a normal driving state region R4, which indicates that the driver 2 is driving normally, within an area that does not include the first absentminded state region R1 or the second absentminded state region R2. This makes it possible to estimate the state of the driver 2 in more detail.

[0049] In the above embodiment, the threshold for transitioning from the first absentminded state or the second absentminded state to the normal driving state is set higher than the threshold for transitioning from the normal driving state to the first absentminded state or the second absentminded state. This allows for stricter determination of whether to return to the normal driving state once the vehicle has entered the first absentminded state or the second absentminded state, thereby improving safety.

[0050] In the above embodiment, the state map M includes a resting state region R3, which indicates that the driver 2 is in a resting state where the driver 2 is not driving but is seated in the seat, within an area that does not include the first absentminded state region R1, the second absentminded state region R2, and the normal driving state region R4. This makes it possible to estimate the state of the driver 2 in more detail.

[0051] In the above embodiment, the estimation unit 204 estimates the state in consideration of the trajectory of the plot P on the state map M determined from the first index value (valence) and the second index value (arousal level). This makes it possible to estimate the state of the driver 2 with higher accuracy in accordance with the progress of changes in the state of the driver 2.

[0052] In the above embodiment, the driving unit 206 is provided to take a physical action on the driver 2 when it is estimated that the driver 2 is in the first absentminded state or the second absentminded state. This allows the driver 2 to return to a normal state (for example, a normal driving state), thereby improving safety.

[0053] In the above embodiment, the skin potential is acquired from the palm of the driver 2 gripping the steering wheel. This makes it possible to acquire the skin potential of the driver 2 driving the vehicle 1 naturally and with high accuracy.

[0054] In the above embodiment, the estimation unit estimates the state of the driver 2 based on the skin potential that changes depending on the amount of sweat on the palms of the hands. This makes it possible to estimate the state of the driver 2 with high accuracy based on the amount of sweat on the palms of the driver 2, which changes sensitively depending on the driving situation.

[0055] Other embodiments of the present invention will be described below, but descriptions of parts that are the same as or similar to the first embodiment will be omitted as appropriate.

[0056] Second Embodiment Fig. 7 is a diagram showing an example of the functional configuration of a state estimation system 111 according to a second embodiment. The state estimation system 111 according to this embodiment includes a correction unit 211 in addition to the functional configuration of the first embodiment.

[0057] The correction unit 211 corrects the state map M according to the characteristics of the driver 2. The characteristics of the driver 2 may be, for example, the personality (mental tendency) or physical characteristics of the driver 2. Such correction of the state map M can be achieved by various methods. For example, the vehicle 1 may acquire the characteristics of the driver 2 by some means before or while the vehicle 1 is traveling, and the state map M used during driving may be corrected according to the acquired characteristics. Various means may be adopted as a means for the vehicle 1 to acquire the characteristics of the driver 2. For example, the driver 2 may input the characteristics themselves via a user I / F installed in the vehicle 1, or the characteristics of the driver 2 may be estimated from biological information acquired while the vehicle 1 is traveling. Furthermore, how the state map M is specifically corrected should be determined appropriately depending on the usage situation, etc., but for example, the regions set on the state map M (in this embodiment, the first absentminded state region R1, the second absentminded state region R2, the calm state region R3, and the normal driving state region R4) may be changed according to the characteristics of the driver 2.

[0058] FIG. 8 is a diagram showing an example of the state map M corrected by the correction unit 211 of the second embodiment. FIG. 8 illustrates an example of correcting the state map M according to the personality of the driver 2. Here, the example illustrates correcting the state map M corresponding to an average personality to a state map M' corresponding to an irritable personality or a state map M'' corresponding to an anxious personality, and shows that the first absentminded state region R1, the second absentminded state region R2, the calm state region R3, and the normal driving state region R4 change according to the personality. Note that this correction method is merely an example, and the method of correcting the state map M is not limited to this.

[0059] The estimation unit 204 (see FIG. 7) of this embodiment estimates the state of the driver 2 who is driving the vehicle 1 based on the state maps M′, M″ corrected according to the characteristics of the driver 2 as described above.

[0060] As described above, according to this embodiment, the state of the driver 2 is estimated using the state maps M', M'' that are optimized according to the characteristics of the driver 2. This makes it possible to improve the accuracy of estimating the state of the driver 2.

[0061] A program for causing a computer (such as the processor 121) to realize the functions of the state estimation system 111 of the above embodiment may be provided by being recorded in an installable or executable file format on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a digital versatile disk (DVD).

[0062] The program may also be configured to be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network, or may be configured to be provided or distributed via a network such as the Internet.

[0063] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims.

[0064] [Summary of the Present Embodiment] The present embodiment has at least the following configuration.

[0065] [1] A state estimation system comprising: an acquisition unit (201) that acquires heart rate information relating to the heart rate of a driver (2) of a moving body (1) and the skin potential of the driver (2); a calculation unit (202) that calculates a first index value based on the heart rate information and a second index value based on the skin potential; and an estimation unit (204) that estimates the state of the driver (2) based on the first index value and the second index value.

[0066] This configuration makes it possible to quickly and accurately estimate changes in the state of the driver (2).

[0067] [2] The state estimation system described in [1], wherein the estimation unit (204) estimates the state of the driver (2) based on the first index value, the second index value, and a state map (M) which is a four-quadrant matrix with the first index value on the first axis and the second index value on the second axis, and in which multiple regions are set for each type of state of the driver (2).

[0068] This configuration makes it possible to identify which of a plurality of states the driver (2) is in.

[0069] [3] The state estimation system described in [2], wherein the state map (M) includes a first absentminded state region (R1) indicating that the driver (2) is in a first absentminded state caused by a predetermined first factor.

[0070] According to this configuration, it is possible to detect when the driver (2) has entered a first absentminded state caused by a first factor, which is defined as a state that is likely to cause an accident, and to prevent the occurrence of an accident.

[0071] [4] The state estimation system described in [3], wherein the state map (M) further includes a second absentminded state region (R2) indicating that the driver (2) is in a second absentminded state caused by a predetermined second factor.

[0072] According to this configuration, it is possible to detect when the driver (2) has entered a second absentminded state caused by a second factor different from the first factor, which is defined as a state in which there is a high possibility of an accident, and to more effectively prevent the occurrence of an accident.

[0073] [5] The state map (M) includes an origin (O) based on the first index value and the second index value calculated at a predetermined reference time point, a first quadrant which is a region where the first index value is higher and the second index value is higher than the origin (O), a second quadrant which is a region where the first index value is lower and the second index value is higher than the origin (O), a third quadrant which is a region where the first index value is lower and the second index value is lower than the origin (O), and a fourth quadrant which is a region where the first index value is higher and the second index value is lower than the origin (O), and the first absentminded state region (R1) does not include the origin (O) and is set to straddle the first quadrant, the second quadrant, the third quadrant, and the fourth quadrant.

[0074] According to this configuration, it is possible to appropriately set the first absentminded state region (R1) based on the state at the reference time point.

[0075] [6] The state map (M) includes an origin (O) based on the first index value and the second index value calculated at a predetermined reference time point, a first quadrant which is a region where the first index value is higher and the second index value is higher than the origin (O), a second quadrant which is a region where the first index value is lower and the second index value is higher than the origin (O), a third quadrant which is a region where the first index value is lower and the second index value is lower than the origin (O), and a fourth quadrant which is a region where the first index value is higher and the second index value is lower than the origin (O), and the second absentminded state region (R2) does not include the origin (O) and the first absentminded state region (R1), and is set to straddle the third quadrant and the fourth quadrant.

[0076] According to this configuration, it is possible to appropriately set the second absentminded state region (R2) based on the state at the reference time point.

[0077] [7] The state estimation system described in [4], wherein the state map (M) includes a normal driving state area indicating that the driver (2) is in a normal driving state in which the driver (2) is driving normally within an area that does not include the first absentminded state area (R1) and the second absentminded state area (R2).

[0078] This configuration makes it possible to estimate the state of the driver (2) in more detail.

[0079] [8] The state estimation system described in [7], wherein a threshold value for transitioning from the first absentminded state or the second absentminded state to the normal driving state is set higher than a threshold value for transitioning from the normal driving state to the first absentminded state or the second absentminded state.

[0080] According to this configuration, once the first absentminded state or the second absentminded state has been reached, a strict determination can be made to return to the normal driving state, thereby improving safety.

[0081] [9] The state map (M) includes a resting state region (R3) indicating that the driver (2) is in a resting state where he / she is not driving but is sitting in a seat, within an area that does not include the first absentminded state region (R1), the second absentminded state region (R2), and the normal driving state region. This is the state estimation system described in [7].

[0082] This configuration makes it possible to estimate the state of the driver (2) in more detail.

[0083]

[10] The state estimation system according to [9], wherein the estimation unit (204) estimates the state in consideration of a trajectory of a plot (P) on the state map (M) determined from the first index value and the second index value.

[0084] According to this configuration, it is possible to estimate the state of the driver (2) with higher accuracy in accordance with the progress of changes in the state of the driver (2).

[0085]

[11] The state estimation system described in [4], further comprising a drive unit (206) that takes physical action on the driver (2) when the driver (2) is estimated to be in the first absentminded state or the second absentminded state.

[0086] According to this configuration, it is possible to return the driver (2) to a normal state (for example, a normal driving state), thereby improving safety.

[0087]

[12] The state estimation system according to any one of [1] to

[11] , wherein the skin potential is acquired from the palm of the driver (2) gripping the steering wheel.

[0088] According to this configuration, it is possible to acquire the skin potential of the driver (2) who is driving the vehicle naturally and with high accuracy.

[0089]

[13] The state estimation system according to

[12] , wherein the estimation unit (204) estimates the state of the driver (2) based on the skin potential that changes depending on the amount of sweat on the palm of the hand.

[0090] This configuration makes it possible to estimate the state of the driver (2) with high accuracy based on the amount of sweat on the palms of the driver (2), which changes sensitively depending on the driving situation.

[0091]

[14] The state estimation system according to any one of [2] to

[11] , further comprising: a correction unit (211) that corrects the state map (M) according to the characteristics of the driver (2).

[0092] This configuration makes it possible to improve the accuracy of estimating the state of the driver (2).

[0093] 1...vehicle, 111...state estimation system, 201...acquisition unit, 202...calculation unit, 203...storage unit, 204...estimation unit, 205...output unit, 206...drive unit, 211...correction unit, M, M', M''...state map, O...origin, P...plot, R1...first absentminded state region, R2...second absentminded state region, R3...rested state region, R4...normal driving state region

Claims

1. A state estimation system comprising: an acquisition unit that acquires heart rate information relating to the heart rate of a driver of a mobile body and the driver's skin potential; a calculation unit that calculates a first index value based on the heart rate information and a second index value based on the skin potential; and an estimation unit that estimates the driver's state based on the first index value and the second index value.

2. The state estimation system according to claim 1, wherein the estimation unit estimates the state of the driver based on the first index value, the second index value, and a state map which is a four-quadrant matrix with the first index value on the first axis and the second index value on the second axis, and in which a plurality of regions are set for each type of state of the driver.

3. The state estimation system according to claim 2, wherein the state map includes a first absentminded state region indicating that the driver is in a first absentminded state caused by a predetermined first factor.

4. The state estimation system according to claim 3, wherein the state map further includes a second absentminded state region indicating that the driver is in a second absentminded state caused by a predetermined second factor.

5. The state map includes an origin based on the first index value and the second index value calculated at a predetermined reference time point, a first quadrant which is an area where the first index value is higher and the second index value is higher than the origin, a second quadrant which is an area where the first index value is lower and the second index value is higher than the origin, a third quadrant which is an area where the first index value is lower and the second index value is lower than the origin, and a fourth quadrant which is an area where the first index value is higher and the second index value is lower than the origin, and the first absentminded state area is set to not include the origin and to straddle the first quadrant, the second quadrant, the third quadrant, and the fourth quadrant.

6. The state map includes an origin based on the first index value and the second index value calculated at a predetermined reference time point, a first quadrant which is an area where the first index value is higher and the second index value is higher than the origin, a second quadrant which is an area where the first index value is lower and the second index value is higher than the origin, a third quadrant which is an area where the first index value is lower and the second index value is lower than the origin, and a fourth quadrant which is an area where the first index value is higher and the second index value is lower than the origin, and the second absentminded state area is set to straddle the third and fourth quadrants, excluding the origin and the first absentminded state area.

7. The state estimation system according to claim 4, wherein the state map includes a normal driving state area indicating that the driver is in a normal driving state in which the driver is driving normally, within an area that does not include the first absentminded state area and the second absentminded state area.

8. The state estimation system according to claim 7, wherein a threshold value for transitioning from the first absentminded state or the second absentminded state to the normal driving state is set higher than a threshold value for transitioning from the normal driving state to the first absentminded state or the second absentminded state.

9. The state estimation system described in claim 7, wherein the state map includes a resting state region indicating that the driver is in a resting state where the driver is not driving but is seated in a seat, within an area that does not include the first absentminded state region, the second absentminded state region, and the normal driving state region.

10. The state estimation system according to claim 9, wherein the estimation unit estimates the state in consideration of a trajectory of a plot on the state map determined from the first index value and the second index value.

11. The state estimation system according to claim 4, further comprising a drive unit that takes a physical action on the driver when it is estimated that the driver is in the first absentminded state or the second absentminded state.

12. The state estimation system according to any one of claims 1 to 11, wherein the skin potential is acquired from the palm of the driver's hand gripping the steering wheel.

13. The state estimation system according to claim 12, wherein the estimation unit estimates the state of the driver based on the skin potential that changes depending on the amount of sweat on the palm of the hand.

14. The state estimation system according to any one of claims 2 to 11, further comprising a correction unit that corrects the state map in accordance with the characteristics of the driver.

Images