Psychophysiological state estimation device

Abstract

This psychophysiological state estimation device (10) comprises: a measurement unit (12) that uses a pair of electrodes (30) disposed on a steering body (32) to measure biological information indicating fluctuations in the heartbeat or heartbeat of an occupant seated on a driver's seat, and that outputs a biological waveform corresponding to the biological information; an estimation unit (16) that uses a plurality of analysis windows in which different time lengths are set, and estimates the psychophysiological state of the occupant from a feature quantity obtained by analyzing the biological waveform in the analysis windows; and an output unit (18) that outputs an estimation result from the estimation unit (16).

Description

Mental and physical state estimation device

[0001] The present disclosure relates to a mental and physical state estimation device.

[0002] Japanese Patent Application Laid-Open Publication No. 2013-138705 discloses a drowsiness detection applicability determination device that determines the applicability of a drowsiness detection system that uses heartbeat features to a subject. The drowsiness detection applicability determination device compares the level of periodicity of time fluctuations in heartbeat intervals calculated at a predetermined cycle from time-series data on the heartbeat intervals with a predetermined threshold.

[0003] In addition, the drowsiness detection applicability determination device divides the heart rate signal data into windows of a predetermined time width, calculates the autocorrelation of the heart rate signal data at a predetermined period for each window, and determines whether drowsiness can be properly detected using the weighted average of the autocorrelation values ​​of all windows.

[0004] Furthermore, Japanese Patent Application Laid-Open Publication No. 2018-161432 discloses a method for detecting non-REM sleep, in which a window of a predetermined length is set for time-series data of heartbeat intervals, and vector analysis is performed on the time-series data for each window to determine whether or not non-REM sleep has occurred.

[0005] As described above, in estimating the mental and physical state, a window having a predetermined time as a window width is set for the occupant's biometric information, such as an electrocardiogram waveform (electrocardiogram data) and a heart rate (pulse wave) waveform (heart rate data), and the mental and physical state is estimated for each window. In this case, the accuracy of estimating the mental and physical state is improved by lengthening the window time (widening the window width).

[0006] However, if the window width is widened, it takes time to estimate the physical and mental state, and especially if signals such as biometric information and electrocardiogram data are cut off midway, it takes even longer to estimate the physical and mental state. For this reason, there is room for improvement in the estimation of physical and mental states.

[0007] The present disclosure provides a mental and physical state estimation device that enables improvement in the accuracy of estimating mental and physical states.

[0008] The first aspect of the mental and physical state estimation device includes a measurement unit that uses a pair of electrodes arranged on a steering body to measure biometric information indicating the heart rate or heart rate fluctuations of an occupant seated in the driver's seat and outputs a biometric waveform corresponding to the biometric information, an estimation unit that uses a plurality of analysis windows with different time lengths and estimates the mental and physical state of the occupant from features obtained by analyzing the biometric waveforms within the analysis windows, and an output unit that outputs the estimation results of the estimation unit.

[0009] A second aspect of the mental and physical state estimation device is the first aspect, and includes that the output unit preferentially outputs estimation results obtained using an analysis window that is set to a longer time as the analysis window.

[0010] A third aspect of the mental and physical state estimation device is the first aspect, wherein the estimation unit uses an estimation model in which an estimated value corresponding to the occupant's mental and physical state is calculated from one or more feature quantities obtained by analyzing the bio-waveform, and calculates the estimated value from the feature quantities obtained from the bio-waveform in each of the analysis windows.

[0011] A fourth aspect of the mental and physical state estimation device is the third aspect, wherein the estimation unit includes a first estimation unit that uses a first analysis window having a time length of a first hour as the analysis window and calculates a first estimated value as the estimated value from the feature obtained from the biowaveform within the first analysis window, and a second estimation unit that uses a second analysis window having a time length of a second hour that is longer than the first time as the analysis window and calculates a second estimated value as the estimated value from the feature obtained from the biowaveform within the second analysis window, and the output unit outputs an estimated value corresponding to the first estimated value and the second estimated value as an estimated value indicating the mental and physical state of the occupant.

[0012] A fifth aspect of the mental and physical state estimation device is the fourth aspect, wherein the output unit outputs the second estimated value as an estimated value indicating the mental and physical state of the occupant in preference to the first estimated value.

[0013] A sixth aspect of the mental and physical state estimation device is the fourth aspect, wherein the output unit outputs the second estimated value when the second estimated value is output from the second estimation unit, and outputs the first estimated value output from the first estimation unit when the second estimated value is not output from the second estimation unit.

[0014] The seventh aspect of the mental and physical state estimation device is any one of the first to sixth aspects, and further includes a grip determination unit, wherein the measurement unit detects potential changes in each of the pair of electrodes, the grip determination unit determines whether the occupant is gripping the steering wheel with both hands based on the potential changes detected by the measurement unit, and the output unit outputs an estimation result of the mental and physical state of the occupant based on the determination result of the grip determination unit.

[0015] The eighth aspect of the mental and physical state estimation device is any one of the first to seventh aspects, and includes an appropriateness determination unit that determines the periodicity of the biological waveform and determines whether the biological information is being detected appropriately, and the output unit outputs an estimation result of the occupant's mental and physical state based on the determination result of the appropriateness determination unit.

[0016] In the mental and physical state estimation device according to the first aspect of the present disclosure, a measurement unit measures biometric information indicating the heart rate or heart rate fluctuations of an occupant seated in the driver's seat using a pair of electrodes arranged on a steering body and outputs a biometric waveform corresponding to the biometric information. An estimation unit estimates the mental and physical state of the occupant from features obtained by analyzing the biometric waveform, and an output unit outputs the estimation result of the estimation unit.

[0017] The estimation unit uses multiple analysis windows with different time lengths to estimate the occupant's physical and mental state by analyzing the bioelectrical waveforms within each analysis window. This allows for improved accuracy in estimating the physical and mental state by including the estimation results of analysis windows with longer time lengths.

[0018] In the second aspect of the mental and physical state estimation device, the output unit outputs estimation results using analysis windows with longer durations in preference to estimation results using analysis windows with shorter durations, thereby improving the accuracy of mental and physical state estimation.

[0019] In the third aspect of the mental and physical state estimation device, an estimation model is used in which an estimated value corresponding to the occupant's mental and physical state is calculated from one or more feature quantities obtained by analyzing the biometric waveform, and an estimation unit calculates an estimated value from the feature quantities obtained from the biometric waveform in each analysis window, thereby easily improving the accuracy of estimating the mental and physical state.

[0020] In a fourth aspect of the mental and physical state estimation device, a first estimation unit uses a first analysis window having a time length of a first hour as the analysis window and calculates a first estimated value as an estimated value from features obtained from the biological waveform within the first analysis window, and a second estimation unit uses a second analysis window having a time length of a second hour that is longer than the first time as the analysis window and calculates a second estimated value as an estimated value from features obtained from the biological waveform within the second analysis window.

[0021] The output unit outputs an estimated value according to the first estimated value and the second estimated value as an estimated value indicating the mental and physical state of the occupant. By including the second estimated value using the second time period, the time period of which is longer than the first time period, the accuracy of estimating the mental and physical state can be further improved.

[0022] In the mental and physical state estimation device of the fifth aspect, the output unit outputs the second estimated value as an estimated value indicating the mental and physical state of the occupant in preference to the first estimated value, thereby easily improving the accuracy of estimating the mental and physical state.

[0023] In the sixth aspect of the mental and physical state estimation device, when a second estimated value is output from the second estimating unit, the second estimated value is output, and when a second estimated value is not output from the second estimating unit, the first estimated value output from the first estimating unit is output, thereby reducing the time during which an estimation result of the mental and physical state of an occupant is not output.

[0024] In the mental and physical state estimation device of the seventh aspect, the measurement unit detects a change in the potential of each of the pair of electrodes, the grip determination unit determines whether the occupant is gripping the steering wheel with both hands based on the change in potential detected by the measurement unit, and the output unit outputs an estimation result of the occupant's mental and physical state based on the determination result of the grip determination unit. This makes it possible to prevent the estimation result from being output when the occupant has released at least one hand from the steering wheel, and to suppress a decrease in the estimation accuracy of the output mental and physical state.

[0025] In the mental and physical state estimation device of the eighth aspect, the appropriateness determination unit determines whether the biological information has been properly detected by determining the periodicity of the biological waveform, and the output unit outputs an estimation result of the occupant's mental and physical state based on the determination result of the appropriateness determination unit. This makes it possible to prevent an estimation result from being output even when the biological information has not been properly measured, and to suppress a decrease in the estimation accuracy of the output mental and physical state.

[0026] 4A and 4B are diagrams showing the outline of the estimation device according to the present embodiment; FIG. 4B is a diagram showing an example of a heartbeat waveform; FIG. 4C is a diagram showing an example of a change in RRI; and FIG. 4D is a diagram showing an example of a feature quantity. (A) and (B) are diagrams showing an outline of the extraction of biological waveform data for each analysis window, where (A) corresponds to a first estimation unit and (B) corresponds to a second estimation unit. FIG. 4C is a diagram showing the change in the estimated value according to the extraction results of FIG. 4A and FIG. 4D.

[0027] Hereinafter, an embodiment of the present disclosure will be described in detail with reference to the drawings. This embodiment estimates the level of drowsiness, fatigue, stress, and the like, as a mental and physical state of a vehicle occupant. This embodiment includes measuring biometric information indicating the heart rate or heart rate fluctuations of an occupant seated in the driver's seat using a pair of electrodes arranged on a steering body, outputting a biometric waveform corresponding to the biometric information, using a plurality of analysis windows with different time lengths, estimating the occupant's mental and physical state from feature quantities obtained by analyzing the biometric waveforms within the analysis windows, and outputting the estimation results. This embodiment also includes preferentially outputting estimation results obtained using analysis windows with longer time lengths.

[0028] In addition, this embodiment includes using an estimation model in which an estimated value corresponding to the occupant's physical and mental state is calculated from one or more feature quantities obtained by analyzing the bio-waveform, and calculating an estimated value from the feature quantities obtained from the bio-waveform in each analysis window.

[0029] Furthermore, this embodiment includes using a first analysis window having a time length of a first hour as the analysis window, calculating a first estimate as an estimate from a feature quantity obtained from the biometric waveform within the first analysis window, and using a second analysis window having a time length of a second hour longer than the first hour as the analysis window, calculating a second estimate as an estimate from the feature quantity obtained from the biometric waveform within the second analysis window, and outputting an estimate corresponding to the first estimate and the second estimate as an estimate indicating the mental and physical state of the occupant.In this case, this embodiment includes outputting the second estimate as an estimate indicating the mental and physical state of the occupant in preference to the first estimate.

[0030] Furthermore, this embodiment includes outputting the second estimated value when the second estimated value is output, and outputting the first estimated value when the second estimated value is not output.

[0031] This embodiment further includes the output unit averaging the first estimated value and the second estimated value as an estimated value indicating the mental and physical state of the occupant, and outputting a weighted average in which the first estimated value and the second estimated value are weighted according to the time length of the analysis window.

[0032] This embodiment also includes detecting a change in the potential of each of the pair of electrodes to determine whether the occupant is gripping the steering wheel with both hands, and outputting an estimation result of the occupant's mental and physical state based on the detection result. In this case, this embodiment includes outputting the estimation result of the occupant's mental and physical state when it is determined that the occupant is gripping the steering wheel with both hands.

[0033] This embodiment also includes determining the periodicity of the biological waveform to determine whether the biological information has been properly detected, and outputting an estimation result of the occupant's mental and physical state based on the determination result. In this case, this embodiment also includes outputting the estimation result of the occupant's mental and physical state when it is determined that the biological information has been properly detected.

[0034] The mental and physical state estimation device 10 according to this embodiment is installed in a vehicle (not shown) and is used to estimate the mental and physical state of an occupant (driver) seated in the driver's seat. Fig. 1 shows a schematic configuration of the estimation device 10 in a block diagram.

[0035] 1 , the estimation device 10 includes a measurement unit 12, a waveform generation unit 14, an estimation unit 16, an output unit 18, a grip determination unit 20, and an appropriateness determination unit 22. The estimation unit 16 includes a first estimation unit 24 and a second estimation unit 26. The estimation device 10 also includes a pair of sensor electrodes 30. The sensor electrodes 30 are paired with a sensor electrode 30L and a sensor electrode 30R, and the sensor electrodes 30L and 30R are each electrically connected to the measurement unit 12.

[0036] The estimation device 10 includes a computer (not shown) including a central processing unit (CPU), read-only memory (ROM), random access memory (RAM), a non-volatile storage medium such as a solid-state drive (SSD), and other required functional circuits. In the estimation device 10, the CPU reads out a program pre-stored in memory such as the ROM or SSD, and executes the program while expanding it into the RAM. As a result, the estimation device 10 realizes the functions of the measurement unit 12, waveform generation unit 14, estimation unit 16, output unit 18, grip determination unit 20, and appropriateness determination unit 22, as well as the functions of a first estimation unit 24 and a second estimation unit 26 in the estimation unit 16.

[0037] The vehicle is equipped with a steering device (not shown), which includes a gripping body that is gripped and operated by the occupant, and a steering body, such as a steering wheel 32. The steering wheel 32 is composed of a substantially annular rim portion 34A, a boss portion 34B provided at the center of the rim portion 34A, and a stay portion 34C that connects the rim portion 34A and the boss portion 34B.

[0038] The steering wheel 32 has a metal core (not shown) that forms the framework, and is made up of a substantially annular rim core of the rim portion 34A, a boss core of the boss portion 34B, and a stay core of the stay portion 34C. The rim core and the boss core are connected by the stay core, and the rim portion 34A, boss portion 34B, and stay portion 34C are integrated into one body.

[0039] The vehicle (steering device) also includes a steering shaft (not shown), which is rotatably supported by the vehicle body on the vehicle front side of a seat (driver's seat, not shown) where an occupant sits, with its axis oriented in the longitudinal direction of the vehicle. The steering wheel 32 is supported by the vehicle body so as to be rotatable integrally with the steering shaft, with the boss core portion of the boss portion 34B fixed to the vehicle rear end of the steering shaft.

[0040] As a result, the steering wheel 32 is located in front of the driver's seat where the occupant sits, and can be turned by the occupant. When the steering wheel 32 is turned to rotate the steering shaft, the steered wheels (front wheels) are turned and the vehicle is steered (the direction of travel is changed). Note that Figure 1 shows a front view of the steering wheel 32 as seen from the occupant when the vehicle is traveling straight ahead.

[0041] The rim portion 34A of the steering wheel 32 is made of an insulating resin material such as urethane, and a rim core metal portion is inserted into a base (not shown) having a generally circular cross section (or a generally elliptical cross section) by insert molding. A decorative portion 36 is disposed on the radially outer side of the base of the rim portion 34A, and the decorative portion 36 is made of leather or resin (a portion may be made of wood) and has insulating properties. The entire circumference of the rim portion 34A in the radial cross section of the steering wheel 32 and the entire circumferential circumference (entire area) of the base are covered with the decorative portion 36.

[0042] Sensor electrodes 30L, 30R are arranged on the steering wheel 32. Each of the sensor electrodes 30L, 30R is made of a conductive material and formed into a generally sheet-like shape, and is disposed on the outer peripheral surface of the base in the rim portion 34A and covered with a decorative portion 36. When the vehicle is traveling straight, the sensor electrode 30L is disposed over approximately half the circumference on the left side of the rim portion 34A, and the sensor electrode 30R is disposed over approximately half the circumference on the right side of the rim portion 34A.

[0043] As a result, when the occupant grips the steering wheel 32 (rim portion 34A) with his / her left hand, the occupant approaches (comes into contact with) the sensor electrode 30L. When the occupant grips the steering wheel 32 with his / her right hand, the occupant approaches (comes into contact with) the sensor electrode 30R. When the occupant's left and right hands touch the steering wheel 32, the sensor electrodes 30L and 30R are in electrical contact (contact via a predetermined impedance) between the occupant's right and left hands, which are sandwiching the occupant's heart.

[0044] When the occupant touches each of the sensor electrodes 30L, 30R, a capacitance is generated (changes) between the occupant and each of the sensor electrodes 30L, 30R. Furthermore, the occupant experiences a change in ionic current due to electrical activity during their heartbeat. Therefore, the measurement unit 12 can detect the capacitance generated when the occupant grips the steering wheel 32 with their left and right hands and touches the sensor electrodes 30L, 30R, as well as the change in capacitance, and detect the current change in the occupant's left and right hands, along with the contact of the occupant's left and right hands with the sensor electrodes 30L, 30R.

[0045] The measurement unit 12 detects the potential (change in potential) corresponding to the capacitance of the sensor electrodes 30L, 30R and outputs the detected potential to the grip determination unit 20. The grip determination unit 20 determines whether the occupant (the left hand) is in contact with the sensor electrode 30L and whether the occupant (the right hand) is in contact with the sensor electrode 30R from the change in potential of each of the sensor electrodes 30L, 30R, and determines whether the occupant is gripping the steering wheel 32 with both hands. Furthermore, if the grip determination unit 20 determines that at least one hand of the occupant is not gripping the steering wheel 32, it obtains time information (information indicating the time from start to finish) and outputs the obtained information to the output unit 18.

[0046] The measurement unit 12 detects a differential voltage between the potential detected by the sensor electrode 30L and the potential detected by the sensor electrode 30R, and outputs a change in the differential voltage (electrocardiogram waveform) as biological information of the occupant to the waveform generation unit 14. Fig. 2A shows a schematic diagram of an electrocardiogram waveform, and Fig. 2B shows a schematic diagram of RRI data indicating a change in RRI obtained from the electrocardiogram waveform. In Fig. 2A, the horizontal axis represents time (e.g., seconds), and the vertical axis represents the amplitude of the electrocardiogram waveform (a value corresponding to a voltage value). In Fig. 2B, the horizontal axis represents time, and the vertical axis represents the period (time) of the RRI.

[0047] 2A, an electrocardiogram waveform periodically shows R waves whose amplitude increases as the heart alternates between expansion and contraction. The waveform generator 14 generates a waveform (hereinafter referred to as biological waveform data) that indicates changes in RRI (R-R interval), which is the interval between R waves, from the electrocardiogram waveform. The waveform generator 14 also outputs the generated biological waveform data to the estimation unit 16 and the appropriateness determination unit 22.

[0048] The appropriateness determination unit 22 determines whether the RRI is too long or there is a break in the biowaveform data from the biowaveform data, and determines whether an appropriate mental and physical state can be estimated. If the appropriateness determination unit 22 determines that there is a break in the biowaveform data, it acquires time information (information indicating the time from start to end) and outputs the acquired information to the output unit 18.

[0049] Meanwhile, the estimation device 10 is configured with an estimation model 40 for estimating the occupant's physical and mental state from one or more feature quantities obtained from the biometric information. There is a correlation between the feature quantities obtained from the biometric information and the physical and mental state, and the first estimation unit 24 and the second estimation unit 26 of the estimation unit 16 each use the estimation model 40 to calculate an estimated value as an index showing the occupant's physical and mental state from the feature quantities. Fig. 3 shows an example of feature quantities obtained from the biometric information in a table.

[0050] As shown in FIG. 3, the feature quantities include LF, which indicates the power spectrum of the frequency band of 0.04 to 0.15 Hz obtained by frequency analysis of the change in RRI (biological waveform data), HF, which indicates the power spectrum of the frequency band of 0.15 to 0.4 Hz, MeanNN (msec), which indicates the average value of RRI (RR interval) over 5 minutes, and the area of ​​the Lorenz plot (LorenzS).

[0051] 3, various feature quantities obtained by analyzing biological waveform data, the natural logarithm of the feature quantity (for example, the power spectrum of various frequency bands, the natural logarithm of the power spectrum of the frequency bands), etc. Furthermore, various feature quantities obtained by analyzing data indicating heart rate variability can be used as the feature quantity.

[0052] The estimation model 40 is configured to be able to estimate at least one of the following mental and physical conditions of the occupant: drowsiness (presence or absence of drowsiness, intensity of drowsiness, etc.), fatigue state (presence or absence of fatigue, level of fatigue, etc.), and stress (presence or absence of stress, intensity of stress, etc.). The estimation model 40 is configured as a regression equation using a multiple regression method such as SVR (Support Vector Regression). In the present embodiment, as an example, the estimation model 40 is configured to be used to calculate an estimated value for mainly estimating drowsiness as the mental and physical condition of the occupant.

[0053] The estimation model 40 may be established, for example, by machine learning. In this case, electrocardiogram waveforms are measured from multiple subjects to obtain electrocardiogram waveforms and other biometric information. Multiple feature quantities are extracted from the obtained electrocardiogram waveforms, and the subjects' physical and mental conditions at the time of measurement are input to obtain training data associating the feature quantities with the physical and mental conditions. The estimation model 40 may be generated by using this training data to generate a trained model that estimates the physical and mental conditions from the feature quantities, and by reading one or more feature quantities and weighting coefficients for each of the feature quantities used in the trained model. This enables the estimation model 40 to estimate the physical and mental conditions of the occupant with high accuracy. Furthermore, using multiple electrocardiogram waveforms obtained from the occupant makes it possible to generate an estimation model 40 tailored to the occupant.

[0054] The estimation model 40 uses, for example, feature quantities Xa, Xb, and Xc, and coefficients Ka, Kb, and Kc are set as weighting coefficients for the feature quantities Xa, Xb, and Xc, respectively. As a result, the estimated value Y obtained from the estimation model 40 is expressed as follows: Y = Ka.Xa + Kb.Xb + Kc.Xc

[0055] The estimation model 40 is not limited to the three types of feature quantities Xa, Xb, and Xc. The feature quantity X used in the estimation model 40 may be one or more types, and the estimation model 40 may be configured by setting a weighting coefficient K for each of the feature quantities X.

[0056] The estimation unit 16 uses an analysis window of a predetermined time length and performs required analysis processing on the biological waveform data within the analysis window to extract the feature quantity X. At this time, the estimation unit 16 uses analysis windows of different time lengths and calculates an estimated value of the mental and physical state using the feature quantity (feature value) X extracted from the biological waveform data within each analysis window.

[0057] The estimation unit 16 includes a first estimation unit 24 and a second estimation unit 26, each having an analysis window with a different time length. The first estimation unit 24 has an analysis window 42 with a time length of time ta. The second estimation unit 26 has an analysis window 44 with a time length of time tb, where time tb of the analysis window 44 is longer than time ta of the analysis window 42 (tb>ta, see FIG. 2B ).

[0058] In the estimation unit 16, the first estimation unit 24 and the second estimation unit 26 each analyze the biological waveform data within the analysis windows 42, 44 at a preset time interval. As a result, the first estimation unit 24 calculates an estimated value Ya, and the second estimation unit 26 calculates an estimated value Yb.

[0059] The output unit 18 outputs an estimated value Ys that enables estimation of the occupant's physical and mental state in accordance with the estimated values ​​Ya and Yb. At this time, the output unit 18 outputs the estimated value Ys in accordance with the determination results of the grip determination unit 20 and the appropriateness determination unit 22.

[0060] The output unit 18 may be, for example, a display such as an LCD. The output unit 18 may also be an indicator that changes its lighting or flashing state depending on the output estimated value Ys, or a speaker that issues a warning to the occupant when the estimated value Ys reaches a value indicating a predetermined mental and physical state. The display or indicator that constitutes the output unit 18 may be disposed, for example, on the instrument panel of the vehicle, and the estimated value Ys or information indicating the estimated value Ys (such as a message indicating the mental and physical state of the occupant) may be displayed on a display installed on the instrument panel of the vehicle.

[0061] Next, the operation of this embodiment will be described. The estimation device 10 of this embodiment is installed in a vehicle. The estimation device 10 includes sensor electrodes 30L and 30R, which are arranged on a steering wheel 32. As a result, the estimation device 10 estimates the mental and physical state of a vehicle occupant (driver) who performs driving operations as a subject, and who grips the steering wheel 32 to contact the sensor electrodes 30L and 30R while the vehicle is running.

[0062] For example, the estimation device 10 starts operation when an ignition switch (not shown) of the vehicle is turned on to enable the vehicle to run, and ends operation when the ignition switch is turned off to stop the vehicle from running. Note that the start / end of the operation of the estimation device 10 is not limited to this, and the estimation device 10 may operate at any timing according to the operation of the occupant, etc.

[0063] When the operation of the estimation device 10 is started, the measurement unit 12 measures the potential of each of the sensor electrodes 30L, 30R and outputs an electrocardiogram waveform (electrocardiogram waveform data) corresponding to the occupant's heart rate fluctuations, etc. The measurement unit 12 also outputs a signal corresponding to the potential of the sensor electrodes 30L, 30R to the grip determination unit 20.

[0064] The waveform generating unit 14 generates and outputs bio-waveform data used to estimate the mental and physical state of the occupant from the electrocardiogram waveform input from the measuring unit 12. The grip determining unit 20 determines whether the occupant is gripping the steering wheel 32 with both hands based on the potential and change in potential of each of the sensor electrodes 30L, 30R (determines whether the occupant is touching the steering wheel 32).

[0065] The biowaveform data is input to each of the first estimation unit 24 and the second estimation unit 26 of the estimation unit 16, and is also input to the appropriateness determination unit 22. The appropriateness determination unit 22 determines whether or not there are any breaks in the biowaveform data. At this time, when data indicating changes in RRI are used as the biowaveform data, the appropriateness determination unit 22 determines whether or not there are any breaks in the biowaveform data, i.e., the electrocardiogram waveform, based on whether or not the RRI is too long, for example.

[0066] The first estimator 24 and the second estimator 26 use analysis windows 42 and 44, respectively, to extract biological waveform data having time widths of times ta and tb at predetermined time intervals, and perform analysis processing such as frequency analysis to calculate feature quantities (feature values) Xa, Xb, and Xc. Furthermore, the first estimator 24 and the second estimator 26 use an estimation model 40 to calculate estimated values ​​Ya and Yb from the feature quantities Xa, Xb, and Xc calculated by each of them, and output the estimated values ​​Ya and Yb to the output unit 18.

[0067] The output unit 18 receives the estimated values ​​Ya and Yb, and sets and outputs an estimated value Ys that enables the occupant's mental and physical state to be estimated from the estimated values ​​Ya and Yb. At this time, the output unit 18 compares the estimated value Ys with a preset reference value and outputs the result, thereby enabling the occupant's mental and physical state to be estimated. The output unit 18 may also evaluate the occupant's mental and physical state from the difference between the reference value and the estimated value Ys, and output, for example, the degree of drowsiness as not drowsy (weak), drowsy, or very drowsy.

[0068] Here, when the estimated value Ya is input, if the estimated value Yb is not input, the output unit 18 outputs the estimated value Ya as the estimated value Ys. Furthermore, when the estimated value Yb is input, the output unit 18 outputs the estimated value Yb, whose time length tb is longer than the time ta, as the estimated value Ys, regardless of whether the estimated value Ya is input. In other words, the output unit 18 outputs the estimated value Yb in preference to the estimated value Ya.

[0069] Fig. 4(A) is a diagram showing an outline of the extraction of biowaveform data by the first estimating unit 24, and Fig. 4(B) is a diagram showing an outline of the extraction of biowaveform data by the second estimating unit 26. Fig. 5 is a diagram showing an outline of the estimated value Ys output from the output unit 18. In Figs. 4(A), 4(B), and 5, the horizontal axis represents time. In Figs. 4(A) and 4(B), the vertical axis represents the change in the biowaveform indicated by the biowaveform data, and if the biowaveform indicates an RRI change, the vertical axis represents the amount of change in the RRI change. In Fig. 5, the vertical axis represents the estimated value.

[0070] As shown in Figures 4A and 4B, the first estimating unit 24 and the second estimating unit 26 extract biological waveform data at time intervals shorter than times ta and tb, respectively, and calculate estimated values ​​Ya and Yb. ０ By starting the estimation process from time T ０ , T １ , T ２ , T ３ , T ４ , T ６ , T ７, ..., analysis using analysis windows 42A, 42B, 42C, 42D, 42F, 42G, ... is started. ３ The estimated value Ya １ , Ya ２ , Ya ３ , Ya ４ , Ya ５ , Ya ６ , Ya ７ , ... are calculated in order and output.

[0071] In contrast, the second estimation unit 26 estimates the time T ０ If the process is started at time T ５ The calculation of the estimated value Yb is not performed until the analysis windows 44A, 44B, and 44C reach the time T ０ The estimation process is started sequentially from the analysis window 44D, which starts at time T ６ The estimated value Yb １ , the estimated value Yb using the analysis window 44E ２ , the estimated value Yb using the analysis window 44F ３ , the estimated value Yb using the analysis window 44G ４ are calculated in order and output.

[0072] As shown in FIG. 5, the output unit 18 outputs the estimated value Yb at a time T ３ , T ４ , T ５ In the estimated value Ya １ , Ya ２ , Ya ３ The output unit 18 outputs the estimated value Ys. ６ , T ７ , T ８ , T ９ In this case, the estimated value Ya ４ , Ya ５ , Ya ６ , Ya ７ , ... are input, and the estimated value Yb １ , Yb ２ , Yb ３ , Yb ４ , ... are input. As a result, the output unit 18 outputs the time T ５Hereafter, the estimated value Yb １ , Yb ２ , Yb ３ , Yb ４ , ... are output as the estimated value Ys.

[0073] Here, the estimated value Yb (Yb １ , Yb ２ The time tb of the analysis window 44 applied to the estimate Ya (Ya １ , Ya ２ The analysis window 44 having a time length of time tb is longer than the time ta of the analysis window 42 applied to the vehicle occupants (...) (tb>ta). Therefore, it can be said that the estimated value Yb has higher estimation accuracy than the estimated value Ya. As a result, in the estimation device 10, by using the analysis window 44 having a time length of time tb, the accuracy of estimating the physical and mental state of the occupant is improved compared to when only the analysis window 42 having a time length of time ta is used.

[0074] Furthermore, in the estimation device 10, during the time (period) when the estimated value Yb is not calculated using the analysis window 44, the estimated value Ya is calculated using the analysis window 42, and this estimated value Ya can be output as the estimated value Ys. In the estimation device 10, by using the analysis window 42 whose time length is set to time ta, which is shorter than time tb, in addition to the analysis window 44 of time tb, the time during which the estimated value Ys is not output despite the device being in operation can be shortened.

[0075] In the estimation device 10, the grip determination unit 20 determines whether the occupant is gripping the steering wheel 32 with both hands, and the determination result is input to the output unit 18. When the determination result of the grip determination unit 20 and the time information are input to the output unit 18, the output unit 18 stops outputting at least one of the estimated value Ya and the estimated value Yb as the estimated value Ys.

[0076] The electrocardiogram waveform as biometric information can be measured when both hands of the occupant contact the sensor electrodes 30 (30L, 30R), and if at least one hand moves away from the sensor electrode 30, an appropriate electrocardiogram waveform cannot be obtained, reducing the accuracy of estimating the physical and mental state from the electrocardiogram waveform.

[0077] When the grip determination unit 20 determines that the occupant's hand has been removed from the sensor electrode 30, the output unit 18 stops outputting the estimated values ​​Ya and Yb estimated from the biological waveform data at the time corresponding to the time information as the estimated value Ys. This prevents the estimation device 10 from outputting an inappropriate estimated value Ys, thereby preventing a decrease in the accuracy of estimating the physical and mental state.

[0078] Furthermore, if the biowaveform data used to estimate the mental and physical state is not continuous (partially interrupted), it becomes difficult to estimate the mental and physical state appropriately in the estimation device 10. In the estimation device 10, the appropriateness determination unit 22 determines whether or not there is an interruption in the biowaveform data used to estimate the mental and physical state, i.e., whether or not the biowaveform data is appropriate for estimating the mental and physical state, and the determination result is input to the output unit 18.

[0079] When the appropriateness determination unit 22 determines that there is a break in the biowaveform data, the output unit 18 stops outputting the estimated values ​​Ya and Yb estimated from the biowaveform data at the time corresponding to the time information as the estimated value Ys. This prevents the estimation device 10 from outputting an inappropriate estimated value Ys, thereby preventing a decrease in the accuracy of estimating the physical and mental state.

[0080] Furthermore, when it becomes possible to estimate an appropriate mental and physical state, the estimation device 10 starts (resumes) outputting the estimate Ys. At this time, the estimation device 10 uses an analysis window 42 having a time length ta shorter than the time tb in addition to the analysis window 44 having a time length tb, thereby shortening the time during which the output of the estimate Ys is stopped.

[0081] In this way, the estimation device 10 can effectively estimate and output the mental and physical state of the occupant by using analysis windows 42, 44 of different time lengths. In this case, the estimation device 10 includes the analysis window 44 of time tb, which is longer than time ta, thereby improving the accuracy of estimating the mental and physical state of the occupant compared to when only the analysis window 42 of time ta is used. Furthermore, the estimation device 10 uses the estimation model 40, making it easier to estimate the mental and physical state of the occupant and improving the accuracy of estimating the mental and physical state.

[0082] Furthermore, the estimation device 10 outputs the estimated value Yb, which is the estimation result of the analysis window 44, in preference to the estimated value Ya, which is the estimation result of the analysis window 42, thereby improving the accuracy of estimating the mental and physical state of the occupant. Moreover, when the estimated value Yb is not output from the second estimator 26, the estimation device 10 outputs the estimated value Ya output from the first estimator 24, thereby reducing the time during which the estimated value Ys of the mental and physical state of the occupant is not output.

[0083] In the embodiment described above, the estimated value Yb of the second estimator 26 is output preferentially. However, the estimated value of the first estimator and the estimated value of the second estimator may be averaged and output as an estimated value of the occupant's mental and physical state. This improves the accuracy of estimating the occupant's mental and physical state even if the estimated value includes an estimated value with a short analysis window time length.

[0084] Furthermore, the estimated value of the occupant's mental and physical state is not limited to the average of the estimates of the first and second estimators. For example, the estimates of the first and second estimators may be weighted according to the time length of the analysis window, and the average or weighted average of the weighted estimates may be output. In this case, for example, the accuracy of estimating the occupant's mental and physical state can be further improved by increasing the weighting coefficient of the estimate using a shorter analysis window compared to the estimate using a shorter analysis window.

[0085] In the embodiment described above, electrocardiogram data is used to estimate the physical and mental state. However, pulse wave data indicating heart rate fluctuations may be used as biological information. In this case, an estimation model may be set using feature quantities obtained from the pulse wave data.

[0086] In this embodiment, the sensor electrodes 30L, 30R are arranged on the steering wheel 32. However, it is also possible to arrange one of the pair of electrodes on the steering body and the other on a seat cushion that supports the buttocks and thighs of the occupant in the seat on which the occupant sits.

[0087] The disclosure of Japanese Patent Application No. 2024-050275, filed on March 26, 2024, is incorporated herein by reference in its entirety. All documents, patent applications, and technical standards mentioned herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. A mental and physical state estimation device comprising: a measurement unit that measures biometric information indicating the heart rate or heart rate fluctuations of an occupant seated in the driver's seat using a pair of electrodes arranged on a steering body and outputs a biometric waveform corresponding to the biometric information; an estimation unit that uses a plurality of analysis windows with different time lengths and estimates the mental and physical state of the occupant from feature quantities obtained by analyzing the biometric waveforms within the analysis windows; and an output unit that outputs the estimation results of the estimation unit.

2. The mental and physical state estimation device according to claim 1, wherein the output unit outputs an estimation result in which an analysis window set to a long time is used as the analysis window.

3. The mental and physical state estimation device of claim 1, wherein the estimation unit uses an estimation model in which an estimated value corresponding to the occupant's mental and physical state is calculated from one or more feature quantities obtained by analyzing the biological waveform, and calculates the estimated value from the feature quantities obtained from the biological waveform in each of the analysis windows.

4. The mental and physical state estimation device according to claim 3, wherein the estimation unit includes: a first estimation unit that uses a first analysis window having a time length of a first hour as the analysis window and calculates a first estimated value as the estimated value from the feature quantity obtained from the biological waveform within the first analysis window; and a second estimation unit that uses a second analysis window having a time length of a second hour longer than the first time as the analysis window and calculates a second estimated value as the estimated value from the feature quantity obtained from the biological waveform within the second analysis window; and wherein the output unit outputs an estimated value corresponding to the first estimated value and the second estimated value as an estimated value indicating the mental and physical state of the occupant.

5. The mental and physical state estimation device according to claim 4, wherein the output unit outputs the second estimated value as an estimated value indicating the mental and physical state of the occupant in preference to the first estimated value.

6. The mental and physical state estimation device described in claim 4, wherein the output unit outputs the second estimated value when the second estimated value is output from the second estimation unit, and outputs the first estimated value output from the first estimation unit when the second estimated value is not output from the second estimation unit.

7. A mental and physical state estimation device as described in claim 1, further comprising a grip determination unit, wherein the measurement unit detects a change in potential of each of the pair of electrodes, the grip determination unit determines whether the occupant is gripping the steering wheel with both hands based on the change in potential detected by the measurement unit, and the output unit outputs an estimated result of the mental and physical state of the occupant based on the determination result of the grip determination unit.

8. A mental and physical state estimation device as described in claim 1, including an appropriateness determination unit that determines the periodicity of the biological waveform and determines whether the biological information has been detected appropriately, and the output unit that outputs an estimation result of the occupant's mental and physical state based on the determination result of the appropriateness determination unit.

Images