Driver State Estimation Device

Abstract

To detect as early as possible an abnormality sign of a driver by minimizing computing quantity when estimating the abnormality sign of the driver driving a mobile body.SOLUTION: A driver state estimating device is equipped with a detecting portion that detects an abnormality sign of a driver from a behavior in a pitch direction and a behavior in a roll direction of a head portion of the driver. The detecting portion computes periodic feature quantities with respect to time-sequence data indicating the behavior in the pitch direction and the behavior in the roll direction of the driver, and computes feature vectors with respect to the respective computed periodic feature quantities. With respect to the computed feature vector in the pitch direction and the computed feature vector in the roll direction, considering at least an average value of the entire feature vectors, an average value of the feature vector in the pitch direction, and an average value of the feature vector in the roll direction, presence / absence of the abnormality sign of the driver is detected.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The technology disclosed herein belongs to the technical field related to a driver state estimation device that estimates the state of a driver who drives a mobile object. [Background technology]

[0002] Recently, the development of automated driving systems has been promoted nationwide. The applicant of this application believes that, at present, there are two main directions for automated driving systems.

[0003] The first direction is a system in which the car takes the initiative and transports passengers to their destination without the need for driver operation, known as fully automated driving. On the other hand, the second direction is an automated driving system that assumes that a human will still be driving the car, for example, for the enjoyment of driving.

[0004] In the second direction of automated driving systems, it is assumed that the vehicle will automatically take over from the driver and continue driving automatically when, for example, the driver develops an illness or other condition that makes normal driving difficult. For this reason, from the perspective of improving the driver's survival rate and ensuring the safety of those around the vehicle, it is extremely important to be able to detect driver abnormalities, particularly functional disorders or illnesses, as early and accurately as possible.

[0005] Patent Document 1 discloses a state detection device that detects a driver's inability to drive based on the amount of shaking of the driver's head, or detects a driver's inability to drive based on the degree to which the driver's eyes are white. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent No. 6361312 [Non-patent literature]

[0007] [Non-Patent Document 1] T. Nakamura, et al., “Multiscale Analysis of Intensive Longitudinal Biomedical Signals and its Clinical Applications”, Proceedings of the IEEE, Institute of Electrical and Electronics Engineers, 2016, vol.104, pp.242-261 [Non-Patent Document 2] Mizuta et al., "Fractal Analysis of Center of Gravity Sway," Equilibrium Research, Japan Society of Equilibrium Medicine, 2016, Vol. 75(3), pp. 154-161 Summary of the Invention [Problem to be solved by the invention]

[0008] As shown in Patent Documents 1 and 2, a technique for determining whether a driver is unable to drive or has an abnormality based on the movement of the driver's head and the state of the driver's eyes is already known. However, Patent Document 1 can detect a decrease in concentration and physical fatigue, but cannot predict in advance the illness that the driver is suffering from. Patent Document 2 detects a state in which the driver becomes unable to drive after the illness has appeared. When an abnormality occurs in the driver, in order to perform an emergency stop or the like more safely, it is preferable to quickly detect the signs before the driver becomes unable to drive.

[0009] In order to estimate the driver's abnormal sign, the inventors of the present application have focused on the periodicity of the driver's head behavior that is observed just before the driver becomes unable to drive. Specifically, the inventors of the present application have proposed estimating the driver's abnormal sign by obtaining a time-series variation pattern for the regularity of the driver's head behavior. However, the amount of calculation required to obtain a time-series variation pattern for the regularity of the driver's head behavior is enormous. For this reason, it is difficult to complete the calculation within the moving body, and there is a risk that it may become difficult to detect the driver's abnormal sign early.

[0010] The technology disclosed here has been developed in consideration of these points, and its purpose is to minimize the amount of calculations required when estimating abnormal signs in the driver of a moving vehicle, and to detect abnormal signs in the driver as early as possible. [Means for solving the problem]

[0011] In order to solve the above-mentioned problems, the technology disclosed herein is directed to a driver state estimation device that estimates the state of a driver who drives a moving object, and includes a head behavior detection unit that detects behavior in a pitch direction and a roll direction of the head of the driver using a sensor, and a detection unit that detects an abnormality sign of the driver from each head behavior detected by the head behavior detection unit, and the detection unit detects the driver's abnormality sign in the pitch direction of the driver. Head Behavior and roll direction Head A periodic feature amount is calculated for each of the time series data indicating the behavior, a feature vector is calculated for each of the calculated periodic feature amounts, and an average value of all of the feature vectors in the pitch direction and the roll direction is calculated for the calculated feature vector in the pitch direction and the calculated feature vector in the roll direction. is equal to or smaller than a first predetermined value set to a level at which the driver's head behavior can be determined to be regular, and a second condition is satisfied, the difference between the average value of the pitch direction feature vector and the average value of the roll direction feature vector being within a predetermined range set to be larger than the difference between the average value of the pitch direction feature vector and the average value of the roll direction feature vector in a normal state and smaller than the difference between the average value of the pitch direction feature vector and the average value of the roll direction feature vector in a disease state. The idea was to have it like this.

[0012] The inventors of the present application first calculated periodic feature amounts for time-series data on the behavior of the driver's head in the pitch direction and the roll direction, and calculated the time-series fluctuation pattern of the calculated periodic feature amount using a dimension reduction method.The inventors of the present application then performed an inverse analysis to reproduce the periodic feature amount from the time-series fluctuation pattern of the calculation result, thereby calculating the elements taken into consideration when calculating the time-series fluctuation pattern of the periodic feature amount.As a result, the inventors of the present application were able to determine that, of the multiple parameters related to the pitch direction feature vector, which is a feature vector in the pitch direction of the head, and the roll direction feature vector, which is a feature vector in the roll direction, Pitch Direction It was found that the average value of all feature vectors in the roll direction, the average value of feature vectors in the pitch direction, and the average value of feature vectors in the roll direction are important factors.

[0013] Therefore, in the technology disclosed herein, the detection unit detects the presence or absence of a driver's abnormal sign by considering at least the average value of all the feature vectors in the pitch direction and the roll direction, the average value of the feature vectors in the pitch direction, and the average value of the feature vectors in the roll direction. This limits the elements used when detecting a driver's abnormal sign based on the time-series variation pattern of the periodic feature amount, thereby reducing the amount of calculation. As a result, it becomes possible to detect a driver's abnormal sign as early as possible.

[0015] As a result of further research, the inventors of the present application found that by detecting signs of abnormality using the average value of all feature vectors in the pitch and roll directions and the difference between the average value of feature vectors in the pitch direction and the average value of feature vectors in the roll direction, it is possible to accurately reproduce the detection results obtained by using the time-series variation pattern of periodic features. This makes it possible to effectively reduce the amount of calculation and detect signs of abnormality in the driver as early as possible.

[0016] In the driver state estimating device, the detection unit In addition to the first and second conditions, The average value of the pitch direction feature vector is The size was set to a level that would allow the driver's head movement to be judged as regular. Less than or equal to the second predetermined value And,The average of the feature vectors in the roll direction is The size was set to a level that would allow the driver's head movement to be judged as regular. When the value is equal to or less than a third predetermined value, it is estimated that the driver is in an abnormality prediction state. 、 The following configuration is also possible.

[0017] According to this configuration, it is possible to more accurately detect a driver's abnormality sign, which makes it possible to detect the driver's abnormality sign as early as possible. Effect of the Invention

[0018] As described above, the technology disclosed herein makes it possible to minimize the amount of calculation required when estimating abnormal signs in a driver operating a mobile vehicle, and to detect abnormal signs in the driver as early as possible. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is a graph illustrating the fluctuations in driving ability levels as a driver develops a disease state. [Diagram 2] FIG. 2 shows the relationship between head behavior homeostasis and disease, where (a) shows the normal state, (b) shows the critical deceleration state, and (c) shows the disease state. [Diagram 3] FIG. 3 is a diagram showing head behavior and periodic feature amounts of head behavior. [Figure 4] FIG. 4 is a diagram showing two-dimensional data indicating a time-series fluctuation pattern of the periodic feature amount of head behavior. [Diagram 5] FIG. 5 is a diagram illustrating an analysis technique used when examining a detection method of the driver state estimating device according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating each element used when calculating a time-series fluctuation pattern of a periodic feature amount. [Figure 7] FIG. 7 is a graph showing the degree of reproduction of the periodic feature amount when the number of elements is changed. [Figure 8] FIG. 8 is a configuration diagram of an in-vehicle system including a driver state estimating device. [Figure 9] FIG. 9 is a part of a flowchart showing a process of estimating a driver's state based on a head behavior. [Figure 10] FIG. 10 shows the remainder of the flowchart illustrating the process of estimating the driver's state based on the head behavior. [Figure 11] FIG. 11 is a graph showing memory consumption when detection of a driver's abnormality sign is performed by the dimensionality reduction method and when detection is performed by the driver state estimating device according to this embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Exemplary embodiments will now be described in detail with reference to the drawings.

[0021] FIG. 1 is a conceptual diagram showing the positioning of the technology according to the present disclosure. The state change that causes the driver to become unable to drive is summarized into three patterns. Case A is a pattern in which some of the functions of perception, judgment, and movement decline, Case B is a pattern in which the overall function declines gradually, and Case C is a pattern in which consciousness is suddenly lost. Of these, in the cases of Cases A and B, as shown in FIG. 1, the driver's driving ability level gradually declines after the onset of the disease, and eventually leads to a state in which the driver is unable to drive. Therefore, if this state of decline in driving ability can be detected, it is possible to detect the signs of the driver's inability to drive. If the signs of inability to drive can be detected, then, for example, the driver's intention can be confirmed and emergency measures such as pulling the vehicle to the shoulder by automatic driving control can be taken. Even in the case of Case C, if the signs before loss of consciousness can be detected, emergency measures can be taken.

[0022] The technology according to the present disclosure focuses on the human homeostasis maintenance function and detects a sign of inability to drive (hereinafter referred to as an abnormality sign) from the behavior of the driver's head.

[0023] Figure 2 shows the relationship between homeostasis of head behavior and disease (see Non-Patent Documents 1 and 2). Each graph in Figure 2 shows head movement as seen from above. (a) in Figure 2 shows the normal state in which homeostasis is maintained, (b) in Figure 2 shows the state of critical deceleration between normal and disease, and (c) in Figure 2 shows the diseased state. The graphs in Figures 2(a) and (c) are taken from Non-Patent Document 1.

[0024] Humans have a function called homeostasis, which tries to keep a constant state against disturbances. Head behavior homeostasis refers to the tendency to maintain the head posture while driving. As shown in Fig. 2(a), in a normal state, the head moves irregularly to maintain homeostasis. On the other hand, as shown in Fig. 2(c), in an ill state, the head movement becomes smaller and the behavior becomes stable. And, as shown in Fig. 2(b), in the critical deceleration state during the transition from a normal state to an ill state, the head is thought to move with periodicity (autocorrelation) (see Non-Patent Document 2).

[0025] The inventors of the present application have focused on the above-mentioned findings and have considered that if the state transition to the critical deceleration state can be captured from the driver's head behavior, it is possible to detect signs of abnormality. They have also considered that the periodic feature amount of the head behavior can be used to detect signs of abnormality in the driver.

[0026] In order to obtain head behavior when abnormal signs occur, the inventors of the present application conducted an experiment in which a subject with a disease of Case C was actually made to drive a vehicle using a driving simulator. In the experiment, the pitch angle (angle in the forward / backward direction) and roll angle (angle in the left / right direction) of the head were obtained by a sensor as data indicating the subject's head behavior. In the experiment, the subject's brain waves were also obtained to determine whether or not epilepsy actually occurred.

[0027] The upper diagram in Figure 3 shows the results of actual head movement. The normal zone, premonition zone, and seizure zone shown in Figure 3 were determined from the results of EEG analysis. The inventors of the present application conducted an experiment and found that, as shown in FIG. 3, in a normal state, the head moves randomly and slightly, in a premonition state, minute vibrations occur, and in a seizure state, the head moves randomly and significantly. In particular, in a premonition state, although there are occasional large movements, periodic vibrations continue in general. In order to detect this periodic change, the inventors of the present application obtained an autocorrelation index (scaling exponent α) for the time series data of the head pitch angle and roll angle by DFA (Detrended Fluctuation Analysis). DFA is a method for investigating scaling by removing components that change very slowly, that is, so-called trends. The autocorrelation index (scaling exponent α) is an example of a feature that indicates the periodicity of data.

[0028] The bottom graph in Figure 3 shows the autocorrelation index of the head pitch angle and roll angle. The smaller the autocorrelation index value, the stronger the autocorrelation, and the larger the value, the weaker the autocorrelation.

[0029] As shown in the lower diagram of Figure 3, in the warning zone, the autocorrelation index tends to decrease and changes regularly. This experimental result is consistent with the above-mentioned findings that in a normal state, the head moves irregularly, and in a critical deceleration state, the head moves periodically. On the other hand, as shown in the lower diagram of Figure 3, even in a normal zone, there are cases where the autocorrelation index is at the same level as in the warning zone.

[0030] Therefore, the inventors of the present application focused on the time series fluctuation pattern of the autocorrelation index. In order to obtain the time series fluctuation pattern, the inventors of the present application calculated a feature vector that indicates the regularity of the time series data of the autocorrelation index and performed pattern classification. First, the inventors of the present application performed a nonlinear dimensionality reduction method (UMAP: Uniform Manifold Approximation and Projection) on the calculated feature vector as a pattern classification method.

[0031] Figure 6 shows a time series fluctuation pattern compressed into two dimensions by UMAP. As shown in Figure 6, when signs of anomalies appear, peculiar changes are observed in the time series fluctuation pattern. In other words, it was found that signs of anomalies can be detected by compressing the dimensions of the feature vectors of autocorrelation indices using UMAP.

[0032] However, the inventors of the present application have found that the method using UMAP requires a huge amount of calculations, which results in a huge amount of memory usage, and while the calculations can be performed sufficiently using a general personal computer, it is difficult to perform the calculations using a calculation device mounted on a moving object such as an automobile.

[0033] Therefore, the inventors of the present application considered replacing the pattern classification method from UMAP with another method. Specifically, as shown in Fig. 5, an inverse analysis was performed on the two-dimensional data obtained by UMAP, and elements necessary for obtaining the results of the autocorrelation index as shown in Fig. 3 were extracted. Here, elements showing classification rules were extracted by performing an inverse analysis using the random forest method.

[0034] As a result of the inverse analysis carried out by the present inventors, the elements are as follows, as shown in FIG. -Average value of all feature vectors Average of pitch direction feature vectors Average value of feature vectors in the roll direction The difference between the average feature vector in the pitch direction and the average feature vector in the roll direction - Gradient of feature vector in pitch direction - Gradient of feature vector in roll direction The difference between the slope of the feature vector in the pitch direction and the slope of the feature vector in the roll direction The difference between the slope of the entire feature vector and the slope of the feature vector in the pitch direction The difference between the inclination of the entire feature vector and the inclination of the feature vector in the roll direction Standard deviation of feature vectors in the pitch direction Standard deviation of roll direction feature vectors The difference between the standard deviation of the feature vector in the pitch direction and the standard deviation of the feature vector in the roll direction It was found that the time series changes in the autocorrelation index can be explained from the two-dimensional data of UMAP using these 12 elements. Each feature vector shown in Fig. 6 is calculated by dividing the time series data of the periodic feature amount of the pitch angle and roll angle in chronological order. The vertical axis of Fig. 6 is the autocorrelation index calculated by DFA as mentioned above, and represents the amount of the feature vector.

[0035] Furthermore, as the inventors of this application continued their analysis, they found that the time series changes in the autocorrelation index could be almost entirely explained by only a specific number of elements, without having to consider all 12 elements. Figure 7 shows the results of the inventors' analysis. The explanatory index on the vertical axis can be regarded as the degree of reproduction of the time series changes in the autocorrelation index, and a value of 1 means that the time series changes in the autocorrelation index can be perfectly reproduced.

[0036] The rightmost element (1) is the case where only the "difference between the average value of the feature vectors in the pitch direction and the average value of the feature vectors in the roll direction" is taken into consideration, and the element (2) is the case where only the "average value of all feature vectors" is taken into consideration. As shown in Fig. 7, when only the difference between the average value of the feature vectors in the pitch direction and the average value of the feature vectors in the roll direction is taken into consideration, the explanation index is 0, but when only the average value of all feature vectors is taken into consideration, the explanation index increases slightly. This is because the difference between the feature vectors alone only indicates the relative positional relationship between the feature vectors in the pitch direction and the feature vectors in the roll direction, and does not reflect the absolute value of each feature vector. On the other hand, when the average value of all feature vectors is taken into consideration, the explanation index is thought to increase because the absolute value of each feature vector is reflected.

[0037] The two elements in Fig. 7 are the case where "the difference between the average of the feature vectors in the pitch direction and the average of the feature vectors in the roll direction" is taken into consideration in addition to "the average of all feature vectors." As shown in Fig. 7, in this case, it can be seen that the explanation index exceeds 0.9. This is because information indicating the relative positional relationship between the feature vectors in the pitch direction and the feature vectors in the roll direction is added to information regarding the absolute value of each feature vector.

[0038] The four elements in Fig. 7 are the case where the "average value of feature vectors in the pitch direction" and the "average value of feature vectors in the roll direction" are taken into consideration in addition to the "average value of all feature vectors" and the "difference between the average value of feature vectors in the pitch direction and the average value of feature vectors in the roll direction". As shown in Fig. 7, in this case, the explanation is almost 1, and it can be seen that the explanation index is about the same as when all 12 elements are taken into consideration. This is because the absolute value of only the feature vectors in the pitch direction and the absolute value of only the feature vectors in the pitch direction are taken into consideration.

[0039] Although not shown, when only the "tilt of the feature vector in the pitch direction" or the "standard deviation of the feature vector in the roll direction" is considered as one element among the 12 elements described above, the explanatory index is considered to be 0. This is because the tilt and standard deviation only indicate the relative positional relationship of each feature vector, and the absolute value of the feature vector is not considered. In addition, when considering two elements, even if the "tilt of the feature vector in the pitch direction" or the "standard deviation of the feature vector in the roll direction" is considered in addition to the "average value of the entire feature vector", the explanatory index is considered to be lower than when the "average value of the entire feature vector" and the "difference between the average value of the feature vector in the pitch direction and the average value of the feature vector in the roll direction" are adopted as two elements. This is because, while the "difference between the average value of the feature vector in the pitch direction and the average value of the feature vector in the roll direction" can explain the relative positional relationship of the entire feature vector in the pitch direction and the entire feature vector in the roll direction by combining it with the "average value of the entire feature vector", the tilt and standard deviation cannot express the relative positional relationship of the entire feature vector in the pitch direction and the entire feature vector in the roll direction even when combined with the "average value of the entire feature vector".

[0040] From these analysis results, the driver's abnormal sign is detected by considering at least the average value of the entire feature vector, the average value of the feature vector in the pitch direction, and the average value of the feature vector in the roll direction for the feature vector calculated from the periodic feature amount. Specifically, it was found that the time series variation pattern of the autocorrelation index can be almost explained by two elements, "the average value of the entire feature vector" and "the difference between the average value of the feature vector in the pitch direction and the average value of the feature vector in the roll direction", and preferably, four elements including "the average value of the feature vector in the pitch direction" and "the average value of the feature vector in the roll direction". Therefore, the inventors of the present application detect the driver's abnormal sign based on the four elements, "the average value of the entire feature vector", "the difference between the average value of the feature vector in the pitch direction and the average value of the feature vector in the roll direction", "the average value of the feature vector in the pitch direction", and "the average value of the feature vector in the roll direction".

[0041] Fig. 8 is a block diagram showing an example of the configuration of an in-vehicle system including a driver state estimation device according to this embodiment. In the in-vehicle system of Fig. 8, a camera 10, a speaker 11, an information presentation unit 12, a switch 13, and a microphone 14 are mounted inside the vehicle. The information processing device 20 is composed of, for example, a single IC chip having a processor and a memory, or multiple IC chips having a processor and a memory. A vehicle stop control unit 40 receives an instruction from the information processing device 20 and performs control to automatically move the vehicle to the shoulder of the road and stop it.

[0042] The camera 10 is installed, for example, on the inside of the windshield, and captures the situation inside the vehicle including the driver. The image captured by the camera 10 is transmitted to the information processing device 20, for example, via an in-vehicle network. The camera 10 is a high-performance camera capable of measuring minute head movements of the driver. Instead of the camera 10, the head behavior may be measured by a sensor worn by the driver.

[0043] In the information processing device 20, the head behavior detection unit 21 detects the behavior of the driver's head from the image captured by the camera 10. For example, the head of the driver is recognized from the image, and the tilt angle of the head, for example, the pitch angle and roll angle, is obtained. The processing in the head behavior detection unit 21 can be realized by existing image processing technology. The processing by the head behavior detection unit 21 makes it possible to obtain time-series data of the head behavior as shown in FIG. 4. The time-series data of the head behavior is sent to a detection unit 30 that detects signs of abnormality in the driver.

[0044] The detection unit 30 includes a periodic feature amount calculation unit 31, a feature vector calculation unit 32, an abnormality determination unit 33, and an abnormality determination database 34. The periodic feature amount calculation unit 31 calculates a periodic feature amount from the time series data of head behavior obtained by the head behavior detection unit 21. Specifically, for example, an autocorrelation index (scaling exponent α) is found as the periodic feature amount by DFA (Detrended Fluctuation Analysis). The periodic feature amount calculation unit 31 can obtain time series data of the periodic feature amount as shown in the middle diagram in FIG. 5.

[0045] The feature vector calculation unit 32 extracts the time series data of the periodic feature obtained by the periodic feature calculation unit 31 in chronological order, and calculates a feature vector from the extracted time series data.

[0046] The abnormality determination unit 33 determines four elements, namely, "average value of all feature vectors," "difference between the average value of feature vectors in the pitch direction and the average value of feature vectors in the roll direction," "average value of feature vectors in the pitch direction," and "average value of feature vectors in the roll direction," from the data obtained by the feature vector calculation unit 32. The abnormality determination unit 33 compares the four elements thus determined with predetermined values ​​stored in the abnormality determination database 34, and determines whether or not a sign of abnormality has occurred in the driver.

[0047] The detection unit 30 accumulates time-series variation patterns including times when the driver is in an abnormality precursor state, and corrects data relating to specific patterns stored in the abnormality determination database 34 based on the accumulated time-series variation patterns.

[0048] When the abnormality determination unit 33 of the detection unit 30 determines that an abnormality symptom has occurred, the detection unit 30 outputs to the interrogation unit 22 a request to interrogate the driver.

[0049] The interrogation unit 22 interrogates the driver when receiving a request to interrogate the driver from the detection unit 30. This interrogation is for confirming the driver's intention as to whether or not it is acceptable to make the vehicle automatically drive to an emergency evacuation. The interrogation is performed, for example, by voice through the speaker 11 or by display through the information presentation unit 12 such as a monitor.

[0050] The response detection unit 23 detects the driver's response to the question posed by the questioning unit 22. The driver responds, for example, by operating the switch 13 or by speaking via the microphone 14. When the driver's intention is confirmed or when there is no response from the driver, the information processing device 20 instructs the vehicle stop control unit 40 to automatically move the vehicle to the shoulder of the road and stop it.

[0051] The driver state estimating device according to the present embodiment includes at least a head behavior detecting unit 21 and a detecting unit 30 in an information processing device 20. The driver state estimating device according to the present disclosure may also include a camera 10.

[0052] 9 and 10 are flowcharts showing the processing operations for detecting an abnormality sign by the driver abnormality inferring device.

[0053] First, in step S1, head behavior detection unit 21 recognizes the driver's head from the image captured by camera 10, and calculates the inclination angles, here the pitch angle and roll angle, of the recognized head. This calculation is performed, for example, every 100 ms.

[0054] Next, in step S2, the detection unit 30 determines whether or not a prescribed number of pieces of head tilt angle data have been accumulated. If the result is YES, that is, the detection unit 30 has accumulated the prescribed number of pieces of head tilt angle data, the detection unit 30 proceeds to step S3. On the other hand, if the result is NO, that is, the detection unit 30 has not accumulated the prescribed number of pieces of head tilt angle data, the detection unit 30 returns to step S1.

[0055] In step S3, the detection unit 30 calculates a periodic feature amount for the time series data of the head tilt angle. The detection unit 30 obtains an autocorrelation index (scaling exponent α) as the periodic feature amount by, for example, DFA. The calculation of the periodic feature amount is performed, for example, every 100 ms, using, for example, 256 pieces of tilt angle data while shifting the time range of the target tilt angle data.

[0056] Next, in step S4, the detection unit 30 judges whether or not the data of the periodicity feature amount has been accumulated to a prescribed number or more. If the result is YES, that is, that the data has been accumulated to a prescribed number or more, the detection unit 30 proceeds to step S5. On the other hand, if the result is NO, that is, that the data has not been accumulated to a prescribed number or more, the detection unit 30 returns to step S1.

[0057] In step S5, the detection unit 30 calculates a feature vector for the periodic feature amount. The feature vector is calculated for, for example, 256 pieces of periodic feature amount data, for example, every 100 ms, while shifting the time range of the target periodic feature amount data.

[0058] Next, in step S6, the detection unit 30 acquires various predetermined values ​​stored in the abnormality determination database .

[0059] Then, in step S7, the detection unit 30 compares the value of each element obtained in step S5 with each of the predetermined values ​​acquired in step S6.

[0060] The determination by the detection unit 30 is shown in Fig. 10. In step S71, the detection unit 30 determines whether or not the average value of all the feature vectors is equal to or less than a first predetermined value. If the result is YES, that is, the average value of all the feature vectors is equal to or less than the first predetermined value, the detection unit 30 proceeds to step S72. On the other hand, if the result is NO, that is, the average value of all the feature vectors is greater than the first predetermined value, the detection unit 30 returns to step S1.

[0061] In step S72, the detection unit 30 determines whether the average value of the feature vectors in the pitch direction is equal to or less than a second predetermined value. If the average value of the feature vectors in the pitch direction is equal to or less than the second predetermined value (YES), the detection unit 30 proceeds to step S73. On the other hand, if the average value of the feature vectors in the pitch direction is greater than the second predetermined value (NO), the detection unit 30 returns to step S1.

[0062] In step S73, the detection unit 30 determines whether the average value of the feature vectors in the roll direction is equal to or less than a third predetermined value. If the average value of the feature vectors in the roll direction is equal to or less than the third predetermined value (YES), the detection unit 30 proceeds to step S74. On the other hand, if the average value of the feature vectors in the roll direction is greater than the third predetermined value (NO), the detection unit 30 returns to step S1.

[0063] In step S74, the detection unit 30 determines whether the difference between the average value of the feature vectors in the pitch direction and the average value of the feature vectors in the roll direction is within a predetermined range. If the difference is within the predetermined range (YES), the detection unit 30 returns and proceeds to step S8. On the other hand, if the difference is outside the predetermined range (NO), the detection unit 30 returns to step S1.

[0064] In step S8, the detection unit 30 outputs a request to inquire of the driver to the inquiry unit 22. After step S8, the process ends.

[0065] The first, second and third predetermined values ​​are set to a value large enough to determine that the driver's head behavior is regular. The fact that each average value of the feature vector is smaller than each predetermined value indicates that the autocorrelation index is small, and therefore means that the head behavior is regular. Based on the results of conventional research, the predetermined range is set to be larger than the difference between the average value of the feature vector in the pitch direction and the average value of the feature vector in the roll direction in a normal state, and smaller than the difference between the average value of the feature vector in the pitch direction and the average value of the feature vector in the roll direction in a diseased state. For example, in the results of conventional research (see Non-Patent Document 2), it is said that the periodicity of the head sway of a healthy person is about 6% smaller in the pitch direction than in the roll direction, and the periodicity of the head sway of a person with cerebellar ataxia is about 11% larger in the pitch direction than in the roll direction.

[0066] In the above-mentioned flowchart, the order of steps S71 to S74 may be changed as appropriate. Also, steps S71 to S74 may be processed simultaneously.

[0067] Fig. 11 shows the memory consumption for analysis using UMAP and analysis using the method of this embodiment. As shown in Fig. 11, when UMAP is used, the memory required, which was about 100 MB, is reduced by an order of 10 to the power of 7, to about 10. Therefore, it was found that this embodiment significantly reduces memory consumption, making it possible to perform calculations even with a calculation device mounted on a moving object.

[0068] Therefore, in this embodiment, there is provided head behavior detection unit 21 which detects the behavior of the driver's head in the pitch direction and the roll direction using a sensor, and detection unit 30 which detects signs of an abnormality in the driver from each head behavior detected by head behavior detection unit 21, and detection unit 30 calculates periodic feature amounts for time series data indicating the behavior of the driver in the pitch direction and the roll direction, calculates feature vectors for each calculated periodic feature amount, and detects the presence or absence of signs of an abnormality in the driver by taking into consideration at least the average value of all feature vectors, the average value of feature vectors in the pitch direction, and the average value of feature vectors in the roll direction for the calculated pitch direction feature vector and the calculated roll direction feature vector.

[0069] In particular, in this embodiment, the detection unit 30 is configured to estimate that the driver is in a pre-abnormal state when the average value of all the feature vectors is equal to or less than a first predetermined value, the difference between the average value of the feature vectors in the pitch direction and the average value of the feature vectors in the roll direction is within a predetermined range, the average value of the feature vectors in the pitch direction is equal to or less than a second predetermined value, and the average value of the feature vectors in the roll direction is equal to or less than a third predetermined value.

[0070] This limits the elements to be used when detecting a driver's abnormality sign based on the time-series variation pattern of the periodic feature, thereby reducing the amount of calculations, and as a result, it becomes possible to detect a driver's abnormality sign as early as possible.

[0071] (Other embodiments) The technology disclosed herein is not limited to the above-described embodiment, and may be substituted without departing from the spirit and scope of the claims.

[0072] For example, in the above embodiment, the autocorrelation index (scaling exponent α) is calculated as the periodic feature amount by DFA for the time-series data of head behavior, but the periodic feature amount may be calculated by a method other than DFA.

[0073] In the above-described embodiment, the driver's abnormal sign is detected by taking into consideration four elements, namely, "average value of all feature vectors," "difference between the average value of feature vectors in the pitch direction and the average value of feature vectors in the roll direction," "average value of feature vectors in the pitch direction," and "average value of feature vectors in the roll direction." This is not limiting, and it is sufficient to take into consideration at least two elements, namely, "average value of all feature vectors," and "difference between the average value of feature vectors in the pitch direction and the average value of feature vectors in the roll direction." In addition, a configuration may be adopted in which the driver's abnormal sign is detected by three elements, namely, "average value of all feature vectors," "average value of feature vectors in the pitch direction," and "average value of feature vectors in the roll direction."

[0074] Furthermore, in the above-described embodiment, an automobile is exemplified as a moving body, but the driver state estimating device 100 described above may be adopted in moving bodies other than an automobile, such as a train.

[0075] The above-described embodiments are merely illustrative and should not be construed as limiting the scope of the present disclosure. The scope of the present disclosure is defined by the claims, and all modifications and variations within the scope of the claims are within the scope of the present disclosure. [Industrial Applicability]

[0076] The technology disclosed herein is useful as a driver state estimation device that estimates the state of a driver who drives a mobile object. [Explanation of symbols]

[0077] 10 Camera (sensor) 21 Head behavior detection unit 30 Detection unit 100 Driver state estimation device

Claims

1. A driver state estimation device that estimates a state of a driver who drives a moving object, comprising: a head behavior detection unit that detects a behavior of the driver's head in a pitch direction and a behavior of the driver's head in a roll direction using a sensor; a detection unit that detects an abnormality sign of the driver from each head behavior detected by the head behavior detection unit, The detection unit is calculating a periodicity feature amount for each of time series data indicating a head behavior in a pitch direction and a head behavior in a roll direction of the driver; A feature vector is calculated for each calculated periodic feature amount, The driver is presumed to be in an abnormality precursor state when the calculated pitch direction feature vector and the calculated roll direction feature vector satisfy a first condition that an average value of all of the pitch direction and roll direction feature vectors is equal to or less than a first predetermined value set to a magnitude that allows the driver's head behavior to be determined to be regular, and when the difference between the average value of the pitch direction feature vector and the average value of the roll direction feature vector is within a predetermined range that is set to be larger than the difference between the average value of the pitch direction feature vector and the average value of the roll direction feature vector in a normal state and smaller than the difference between the average value of the pitch direction feature vector and the average value of the roll direction feature vector in a diseased state. A driver state estimating device comprising:

2. 2. The driver state estimating device according to claim 1, The detection unit is In addition to the first and second conditions, a driver state estimation device which estimates that the driver is in an abnormality precursor state when an average value of the feature vectors in the pitch direction is equal to or less than a second predetermined value set to a magnitude sufficient to determine that the driver's head behavior is regular, and when an average value of the feature vectors in the roll direction is equal to or less than a third predetermined value set to a magnitude sufficient to determine that the driver's head behavior is regular.

3. A driver state estimation device that estimates a state of a driver who drives a moving object, comprising: a head behavior detection unit that detects a behavior of the driver's head in a pitch direction and a behavior of the driver's head in a roll direction using a sensor; a detection unit that detects an abnormality sign of the driver from each head behavior detected by the head behavior detection unit, The detection unit is calculating a periodicity feature amount for each of time series data indicating a head behavior in a pitch direction and a head behavior in a roll direction of the driver; A feature vector is calculated for each calculated periodic feature amount, a driver state estimation device which estimates that the driver is in an abnormality precursor state when, for a calculated pitch direction feature vector and a calculated roll direction feature vector, an average value of all of the pitch direction and roll direction feature vectors is equal to or less than a first predetermined value set to a magnitude that allows the driver's head behavior to be determined to be regular, the average value of the pitch direction feature vectors is equal to or less than a second predetermined value set to a magnitude that allows the driver's head behavior to be regular, and the average value of the roll direction feature vectors is equal to or less than a third predetermined value set to a magnitude that allows the driver's head behavior to be regular.

Images