Driver's fatigue estimation system and driver's fatigue estimation method

Abstract

To provide a driver's fatigue estimation system and a driver's fatigue estimation method capable of safely measuring without a fear of the violation of privacy and without causing an obstruction during driving, and capable of estimating a fatigue state with high accuracy.SOLUTION: Seating pressure measurement means 11 is provided to measure a seating pressure distribution of a driver seated on a driver seat of a vehicle pressing a seating surface of the driver seat by right and left ischial bones. Estimation means 12 is provided to perform topological data analysis, time-series analysis of center-of-gravity coordinates obtained from the seating pressure distribution, and time frequency analysis of the center-of-gravity with respect to the seating pressure distribution measured by the seating pressure measurement means 11, and estimate a fatigue state of the driver on the basis of the analysis results.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a driver fatigue estimation system and a driver fatigue estimation method. [Background technology]

[0002] It has become clear that driver fatigue is one of the causes of automobile accidents, and early detection of driver fatigue is important from the perspective of safe driving. Driving fatigue is extremely dangerous because it leads to a decline in driving ability and reaction speed, making it easier for the driver to become inattentive to the road ahead (distracted driving), and leading to an increased accident rate.

[0003] Conventional methods for objectively estimating driver fatigue include, for example, a method of estimating fatigue by photographing the face of the driver while driving and detecting the driver's eye closure (blinking) and yawning through image analysis (see, for example, Patent Documents 1 and 2, or Non-Patent Document 1), and a method of measuring the driver's accelerated pulse wave data using a pulse meter or electrocardiograph and estimating fatigue based on the ratio (LF / HF value) of the low frequency component (LF value) to the high frequency component (HF value) in the frequency domain of the pulse wave data (see, for example, Patent Documents 3 or 4). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2016-524939 [Patent Document 2] Patent Publication No. 2021-034035 [Patent Document 3] Patent No. 5491749 [Patent Document 4] Japanese Patent Application Publication No. 2017-063966 [Non-patent literature]

[0005] [Non-Patent Document 1] "Microsoft researchers use visual AI to make India's roads safer," [online], November 7, 2017, [Retrieved February 21, 2024], Internet <URL: https: / / news.microsoft.com / en-in / features / microsoft-hams-maruti-idtr-visual-ai-india-roads-driving / > Summary of the Invention [Problem to be solved by the invention]

[0006] However, the fatigue estimation methods using image analysis described in Patent Documents 1 and 2 and Non-Patent Document 1, etc., have the problem that the accuracy of detecting blinks and yawns changes depending on the brightness of the driver's seat and the intensity of light falling on the driver, resulting in a decrease in the accuracy of fatigue estimation. Also, since the driver's face is photographed, there is a risk of violating privacy, making it difficult to adopt.

[0007] Furthermore, in the methods described in Patent Documents 3 and 4, which estimate fatigue by measuring fluctuations related to the autonomic nervous system, such as heart rate, the heart rate of the driver is measured while the driver is driving, which increases the influence of body movement noise, reducing the signal-to-noise ratio and decreasing the accuracy of fatigue estimation. Also, since sensors and measuring devices must be attached to the driver, this interferes with driving and can be dangerous.

[0008] The present invention has been made in light of these issues, and aims to provide a driver fatigue estimation system and a driver fatigue estimation method that are free from the risk of privacy invasion, enable safe measurements without getting in the way while driving, and are capable of estimating the fatigue state with high accuracy. [Means for solving the problem]

[0009] When sitting in a seat, if a person is not tired, they can press the seat surface evenly with both ischial bones, but as fatigue accumulates, it becomes difficult to maintain the balance of the body's center of gravity, so the center of gravity shifts to one of the ischial bones, causing the pelvis to become distorted and the body's center of gravity to shift. Based on this knowledge, the inventors discovered that by monitoring the balance between the left and right ischial bones in real time, it is possible to estimate fatigue associated with sitting, and this led to the present invention.

[0010] In other words, the driver fatigue estimation system of the present invention is characterized by having a seat pressure measurement means that measures the seat pressure distribution where the left and right ischial bones of a driver seated in the driver's seat of a vehicle press against the seat surface of the driver's seat, and an estimation means that performs topological data analysis on the seat pressure distribution measured by the seat pressure measurement means, time series analysis of the center of gravity coordinates calculated from the seat pressure distribution, and time frequency analysis of the center of gravity coordinates, and estimates the driver's fatigue state based on the results of these analyses.

[0011] The driver fatigue estimation method of the present invention is characterized by comprising a seat pressure measurement step of measuring the seat pressure distribution where the left and right ischial bones of a driver seated in the driver's seat of a vehicle press against the seat surface of the driver's seat, and an estimation step of performing topological data analysis on the measured seat pressure distribution, time series analysis of the center of gravity coordinates calculated from the seat pressure distribution, and time frequency analysis of the center of gravity coordinates, and estimating the driver's fatigue state based on the results of these analyses.

[0012] The driver fatigue estimation method according to the present invention can be suitably implemented by the driver fatigue estimation system according to the present invention. The driver fatigue estimation system and driver fatigue estimation method according to the present invention measure the force with which the driver's left and right ischial bones press against the seat surface of the driver's seat, i.e., seat pressure, which is less susceptible to the influence of ambient conditions such as light shining on the driver, and less susceptible to the influence of body movement noise caused by driving compared to fluctuations related to the autonomic nervous system such as heart rate. This makes it possible to suppress a decrease in the signal-to-noise ratio of the measured seat pressure, enabling a highly accurate estimation of the fatigue state.

[0013] Furthermore, the driver fatigue estimation system and driver fatigue estimation method according to the present invention do not photograph the driver's face, so the driver cannot be identified and there is no risk of privacy being invaded. Furthermore, to measure seat pressure, a sensor or measuring device for seat pressure measurement means can be attached to the seat surface or inside the seat, and there is no need to attach it directly to the driver, so measurements can be made safely and non-invasively without getting in the way of driving.

[0014] The driver fatigue estimation system and driver fatigue estimation method according to the present invention detect variations in seat pressure distribution at predetermined time intervals (for example, every one second) and perform various analyses, thereby estimating the driver's accumulated fatigue and state of fatigue in near real time. Furthermore, differential analysis can detect variations in seat pressure according to individual differences between drivers. This makes it possible to reduce the influence of disturbances such as body movement noise, thereby improving the accuracy of fatigue estimation.

[0015] In the driver fatigue estimation system and driver fatigue estimation method according to the present invention, the seat pressure measuring means for measuring the force of the driver's left and right ischial bones pressing against the seat surface of the driver's seat, i.e., the seat pressure distribution, may be any device capable of measuring seat pressure, such as a commercially available pressure sensor or a sheet-like pressure sensor. The sheet-like pressure sensor may be any device capable of measuring seat pressure distribution, such as a sheet-like sensor with multiple pressure sensors installed at predetermined intervals in the front, back, left, and right directions, a capacitance-type sheet-like pressure sensor with an insulator sandwiched between multiple vertically arranged electrodes and multiple horizontally arranged electrodes, or a sheet-like pressure sensor with a strain gauge formed on the surface of a diaphragm (separating membrane) that converts resistance changes caused by deformation (physical strain) due to pressure into an electrical signal for detection. The seat pressure measuring means is preferably built into the seat of the driver's seat or attached by being laid on the seat surface.

[0016] In the driver fatigue estimation system according to the present invention, the estimation means may construct a simplicial complex from the seat pressure distribution at each time point using the topological data analysis, determine the number of holes in the first-order homology of the simplicial complex, and estimate the driver's fatigue state based on the determined number of holes. In the driver fatigue estimation method according to the present invention, the estimation step may construct a simplicial complex from the seat pressure distribution at each time point using the topological data analysis, determine the number of holes in the first-order homology of the simplicial complex, and estimate the driver's fatigue state based on the determined number of holes. In this case, when the driver becomes fatigued, the center of gravity shifts to one of the left and right ischial tuberosities, causing the number of holes in the first-order homology of the simplicial complex constructed from the seat pressure distribution to change, for example, from two to one. Therefore, the driver's fatigue state can be estimated by detecting the change in the number of holes.

[0017] In topological data analysis, it is preferable to determine the number of holes in the first-order homology from the measured seating pressure distribution at predetermined time intervals. Furthermore, the point cloud data for constructing the simplicial complex is preferably the coordinates of points measuring the seating pressure distribution, including points with seating pressures equal to or greater than a predetermined seating pressure, points with seating pressures equal to or less than a predetermined seating pressure, or points with seating pressures within a predetermined range. Furthermore, the simplicial complex may be any complex, such as a Check complex, an alpha complex, or a Lips complex. In topological data analysis, any method, such as Mapper or persistent homology, may be used for the analysis.

[0018] In the driver fatigue estimation system according to the present invention, the estimation means may use the time series analysis to approximate a distribution range of the center of gravity coordinates for each time period within a predetermined time range when plotted on planar coordinates, with an ellipse, and estimate the driver's fatigue state based on the shape of the ellipse. Alternatively, the estimation means may calculate a circularity for the distribution range of the center of gravity coordinates for each time period within a predetermined time range when plotted on planar coordinates, and estimate the driver's fatigue state based on the circularity. In the driver fatigue estimation method according to the present invention, the estimation step may use the time series analysis to approximate a distribution range of the center of gravity coordinates for each time period within a predetermined time range when plotted on planar coordinates, with an ellipse, and estimate the driver's fatigue state based on the shape of the ellipse. Alternatively, the estimation step may calculate a circularity for the distribution range of the center of gravity coordinates for each time period within a predetermined time range when plotted on planar coordinates, and estimate the driver's fatigue state based on the circularity. In this case, when approximating the center of gravity coordinates with an ellipse, the driver's fatigue state can be estimated based on, for example, the lengths of the major and minor axes of the ellipse and the direction of inclination of the ellipse.

[0019] In the driver fatigue estimation system according to the present invention, the estimation means may perform a short-time Fourier transform (STFT) or a discrete wavelet transform (DWT) on the seat pressure data of the center of gravity coordinate at each time within a predetermined time range using the time-frequency analysis, and estimate the driver's fatigue state based on the transform results. In the driver fatigue estimation method according to the present invention, the estimation step may perform a short-time Fourier transform or a discrete wavelet transform on the seat pressure data of the center of gravity coordinate at each time within a predetermined time range using the time-frequency analysis, and estimate the driver's fatigue state based on the transform results. In this case, when the driver becomes fatigued, the center of gravity rests on one of the ischial tuberosities, causing distortion of the pelvis and shifting the center of gravity of the body. By detecting this center of gravity shift from changes in a predetermined frequency component, it is possible to detect changes in the driver's posture and estimate fatigue.

[0020] In the driver fatigue estimation system according to the present invention, the estimation means preferably estimates the driver's fatigue state based on changes over time in the analysis results. In the driver fatigue estimation method according to the present invention, the estimation step preferably estimates the driver's fatigue state based on changes over time in the analysis results. In this case, when performing time series analysis or time-frequency analysis, for example, if the time range over which the distribution of each centroid coordinate and the STFT or DWT transformation results are obtained is Δt, the distribution may be obtained every Δt and the driver's fatigue state may be estimated based on the changes over time. Alternatively, the distribution of each centroid coordinate and the STFT or DWT transformation results for the most recent Δt may be obtained in real time and the driver's fatigue state may be estimated based on the changes over time. More specifically, for example, the centroid coordinates may be obtained every second and the distribution of each centroid coordinate and the STFT or DWT transformation results may be obtained every 10 minutes. Alternatively, the distribution of each centroid coordinate and the STFT or DWT transformation results for the most recent 10 minutes may be obtained in real time.

[0021] Furthermore, the driver fatigue estimation system according to the present invention may include an output means for outputting an estimation result to the driver when the estimation means estimates that the driver is fatigued. The driver fatigue estimation method according to the present invention may include an output step for outputting an estimation result to the driver when the estimation step estimates that the driver is fatigued. In this case, presenting the estimation result to the driver or issuing a warning to the driver depending on the estimated fatigue state can be useful for managing the driver's fatigue while driving and can reduce the risk of accidents.

[0022] In the driver fatigue estimation system according to the present invention, the estimation means may be configured to first perform the topological data analysis, and if the topological data analysis does not result in an estimation of fatigue, perform the time series analysis, and if the time series analysis does not result in an estimation of fatigue, perform the time frequency analysis, thereby estimating the driver's fatigue state, and repeatedly estimate the driver's fatigue state at predetermined time intervals. In the driver fatigue estimation method according to the present invention, the estimation step may be configured to first perform the topological data analysis, and if the topological data analysis does not result in an estimation of fatigue, perform the time series analysis, and if the time series analysis does not result in an estimation of fatigue, perform the time frequency analysis, thereby estimating the driver's fatigue state, and repeatedly estimate the driver's fatigue state at predetermined time intervals. In this case, the driver's fatigue state can be estimated more accurately. [Effects of the Invention]

[0023] According to the present invention, it is possible to provide a driver fatigue estimation system and a driver fatigue estimation method that can estimate a driver's fatigue state with high accuracy, without violating privacy and allowing safe measurements without getting in the way while driving. [Brief explanation of the drawings]

[0024] [Figure 1] 1 is a perspective view showing a driver fatigue estimation system according to an embodiment of the present invention; [Figure 2] (a) A two-dimensional heat map showing the results of measurement of seat pressure distribution, (b) a three-dimensional heat map, (c) a two-dimensional heat map showing the results of measurement of back pressure distribution, and (d) a three-dimensional heat map, of the driver fatigue estimation system shown in Figure 1, after a predetermined time has elapsed since the start of driving. [Figure 3]This is a plan view illustrating the x-coordinate (coordinate in the left-right direction of the driver) xi, the y-coordinate (coordinate in the front-rear direction of the driver) yj (i, j are integers from 1 to n) of the position of each sensor of the seat pressure measuring means consisting of a sheet-shaped pressure sensor having n vertical x n horizontal sensors of the driver fatigue estimation system shown in Figure 1, and the pressure pij of the sensor at (xi, yj). [Figure 4] Figure 1 shows a heat map of seat pressure distribution at the start of driving, (b) 10 minutes, (c) 20 minutes, (d) 30 minutes, (e) 40 minutes, (f) 50 minutes, and (g) 60 minutes after starting driving, for a male driver in his 40s, using the driver fatigue estimation system. (h) A graph showing the time-dependent changes in the X and Y coordinates of the center of gravity coordinate G calculated every second. (i) A graph plotting the time-dependent changes in the center of gravity coordinate G in (h) on a plane coordinate system. [Figure 5] Figure 1 shows a heat map of seat pressure distribution at the start of driving, (b) 10 minutes after starting driving, (c) 20 minutes after starting driving, (d) 30 minutes after starting driving, (e) 40 minutes after starting driving, (f) 50 minutes after starting driving, and (g) 60 minutes after starting driving, for a female driver in her 20s. (h) A graph of the time changes in the X and Y coordinates of the center of gravity coordinate G calculated every second. (i) A graph plotting the time changes in the center of gravity coordinate G in (h) on a plane coordinate system. [Figure 6] 2 is a graph showing the distribution when each center of gravity coordinate of the driver fatigue estimation system shown in FIG. 1 is plotted on a plane coordinate system, and a method of approximating the distribution range with an ellipse. [Figure 7] (a) is a graph showing the distribution of pressure centroid coordinates for every 10 minutes from the start of driving when the driver is a male in his 40s, and (b) is a graph showing the distribution of pressure centroid coordinates for every 10 minutes from the start of driving when the driver is a female in her 20s, of the driver fatigue estimation system shown in Figure 1. [Figure 8] 3 is a flowchart showing the flow of a method for estimating driver fatigue according to an embodiment of the present invention. [Figure 9]The driver fatigue estimation system shown in Figure 1 starts 40 minutes after the start of driving. (a) 0 seconds (starting point), (b) 30 seconds, (c) 60 seconds, (d) 90 seconds, (e) 120 seconds, (f) 150 seconds, and (g) 180 seconds after the start of driving are shown as two-dimensional heat maps of seat pressure distribution, (h) a graph of the time changes in the X and Y coordinates of the center of gravity calculated every second, and (i) a graph in which the time changes in the center of gravity coordinates of (h) are plotted on a plane coordinate system. [Figure 10] The driver fatigue estimation system shown in Figure 1 starts 80 minutes after the start of driving. (a) 0 seconds (starting point), (b) 30 seconds, (c) 60 seconds, (d) 90 seconds, (e) 120 seconds, (f) 150 seconds, and (g) 180 seconds after the start of driving are shown as two-dimensional heat maps of seat pressure distribution, (h) a graph of the time changes in the X and Y coordinates of the center of gravity calculated every second, and (i) a graph in which the time changes in the center of gravity coordinates of (h) are plotted on a plane coordinate system. [Figure 11] The driver fatigue estimation system shown in Figure 1 starts at 120 minutes after the start of driving. (a) 0 seconds (starting point), (b) 30 seconds, (c) 60 seconds, (d) 90 seconds, (e) 120 seconds, (f) 150 seconds, and (g) 180 seconds after the start of driving are shown as two-dimensional heat maps of seat pressure distribution, (h) a graph of the time changes in the X and Y coordinates of the center of gravity calculated every second, and (i) a graph in which the time changes in the center of gravity coordinates of (h) are plotted on a plane coordinate system. [Figure 12] The driver fatigue estimation system shown in Figure 1 starts at 160 minutes after the start of driving. (a) 0 seconds (starting point), (b) 30 seconds, (c) 60 seconds, (d) 90 seconds, (e) 120 seconds, (f) 150 seconds, and (g) 180 seconds after the start of driving are shown as a two-dimensional heat map of the seat pressure distribution. (h) A graph of the time change in the X and Y coordinates of the center of gravity calculated every second is shown. (i) A graph in which the time change in the center of gravity coordinate of (h) is plotted on a plane coordinate system. [Figure 13]The driver fatigue estimation system shown in Figure 1 starts at 200 minutes after the start of driving. (a) 0 seconds (starting point), (b) 30 seconds, (c) 60 seconds, (d) 90 seconds, (e) 120 seconds, (f) 150 seconds, and (g) 180 seconds after that starting point are shown as two-dimensional heat maps of seat pressure distribution, (h) a graph of the time changes in the X and Y coordinates of the center of gravity calculated every second, and (i) a graph in which the time changes in the center of gravity coordinates of (h) are plotted on a plane coordinate system. [Figure 14] The driver fatigue estimation system shown in Figure 1 starts at 240 minutes after the start of driving. (a) 0 seconds (starting point), (b) 30 seconds, (c) 60 seconds, (d) 90 seconds, (e) 120 seconds, (f) 150 seconds, and (g) 180 seconds after the start of driving are shown as two-dimensional heat maps of seat pressure distribution, (h) a graph of the time changes in the X and Y coordinates of the center of gravity calculated every second, and (i) a graph in which the time changes in the center of gravity coordinates of (h) are plotted on a plane coordinate system. [Figure 15] The analysis results of the driver fatigue estimation system shown in Figure 1 using the seat pressure distribution shown in Figure 14 are shown below: (a) seat pressure distribution shown in Figure 14(a), (b) map of a simplicial complex obtained by topological data analysis, (c) a graph plotting the time change in the center of gravity coordinate shown in Figure 14(h) on plane coordinates, which is used in time series analysis, and (d) power spectrum obtained by time-frequency analysis. [Figure 16] Figure 1 shows the analysis results of the driver fatigue estimation system using the seat pressure distribution immediately after starting driving: (a) seat pressure distribution 0 seconds after starting driving (baseline time), (b) map of simplicial complexes obtained by topological data analysis, (c) a graph plotting the time change in center of gravity coordinates on plane coordinates used in time series analysis, and (d) power spectrum obtained by time-frequency analysis. DETAILED DESCRIPTION OF THE INVENTION

[0025] Hereinafter, embodiments of the present invention will be described with reference to the drawings and examples. 1 to 16 show a driver fatigue estimation system and a driver fatigue estimation method according to an embodiment of the present invention. As shown in FIG. 1, a driver fatigue estimation system 10 includes a seat pressure measuring means 11 and an estimation means 12.

[0026] The seat pressure measuring means 11 is made up of a sheet-like pressure sensor and is attached to the seat surface of the driver's seat so that it can measure the seat pressure distribution of the driver seated in the driver's seat. In one specific example, the seat pressure measuring means 11 is made up of a commercially available sheet-like pressure sensor that has a strain gauge formed on the surface of a diaphragm (separating membrane) and is configured to convert and detect resistance changes caused by deformation (physical strain) due to pressure into electrical signals.

[0027] The estimation means 12 is made up of a computer and is connected to the seat pressure measurement means 11. The estimation means 12 is configured to be able to receive data on seat pressure distribution measured by the seat pressure measurement means 11. The estimation means 12 is configured to estimate the driver's fatigue state from the seat pressure distribution measured by the seat pressure measurement means 11. The estimation means 12 is configured to perform topological data analysis (TDA) on the seat pressure distribution, phase-synchronization analysis (PSA) on the barycentric coordinates obtained from the seat pressure distribution, and time-frequency analysis (TFA) on the barycentric coordinates, and to estimate the driver's fatigue state based on the results of these analyses. Note that in the specific example shown in FIG. 1, the estimation means 12 is connected to the seat pressure measurement means 11 by wire, but it may also be connected wirelessly.

[0028] [Topological Data Analysis] When the estimation means 12 estimates the driver's fatigue state by topological data analysis, first, a simplicial complex is constructed from the seat pressure distribution at each time. At this time, the point cloud data for constructing the simplicial complex are coordinates of points at which the seat pressure distribution is measured, that is, points with a seat pressure equal to or greater than a predetermined seat pressure, points with a seat pressure equal to or less than a predetermined seat pressure, or points with a seat pressure within a predetermined range. The simplicial complex may be, for example, a Check complex, an alpha complex, or a Lips complex.

[0029] Next, the number of holes in the first-order homology of the constructed simplicial complex is calculated. In this way, by calculating the number of holes in the first-order homology from the measured seat pressure distribution at predetermined time intervals, changes in the number of holes can be detected. For example, when a driver becomes fatigued, the center of gravity falls on either the left or right ischium, causing the number of holes in the first-order homology of the simplicial complex constructed from the seat pressure distribution to change from two to one. Therefore, by detecting changes in the number of holes, the driver's fatigue state can be estimated as "fatigued."

[0030] In topological data analysis, any method such as Mapper or persistent homology may be used, and more specifically, the analysis may be performed using Ripser, GUDHI, or the like, which are publicly available Python libraries. In topological data analysis, by appropriately selecting parameters such as the radius at each point of the simplicial complex, the shape of the point cloud data of seat pressure distribution can be accurately reflected, and the driver's fatigue state can be estimated with high accuracy.

[0031] [Time series analysis of centroid coordinates] When the estimation means 12 estimates the driver's fatigue state through time-series analysis, it first calculates the center of gravity coordinate of the seat pressure distribution at each time point. The method for calculating the center of gravity coordinate is described below with a specific example. Specifically, a subject (driver) was asked to drive in a driving simulator for one hour, and the seat pressure distribution during that time was measured. Figures 2(a) and 2(b), and 2(c) and 2(d) show two-dimensional and three-dimensional heat maps, respectively, showing the seat pressure distribution and back pressure distribution after a predetermined time has elapsed since the start of driving. Note that in Figure 2, a "seat sensor (seat pressure distribution measuring device)" manufactured by XSENSOR was used as the seat pressure measurement means 11. Figure 2(a) shows the pressure distribution when the seat is viewed from below. The x-coordinate in Figure 2(a) is the coordinate in the driver's front-to-back direction, with the larger number indicating the buttocks. The y-coordinate in Figure 2(a) is the coordinate in the driver's left-to-right direction, with the center approximately corresponding to the driver's midline. Figure 2(c) shows the pressure distribution on the back. The x-coordinate in Figure 2(c) is the left-right direction, and the y-coordinate is the vertical direction. Figures 2(b) and (d) are three-dimensional representations of the pressure distribution in Figures 2(a) and (c), respectively, where the higher the peak, the higher the pressure. Figures 2(b) and (d) show that the driver's pressure distribution is biased to the right.

[0032] Using the data of the seat pressure distribution thus obtained, the barycentric coordinates of the pressure at each time from the start of driving are calculated. The barycentric coordinates are calculated, for example, as follows. As shown in FIG. 3, the seat pressure measuring means 11 is a sheet-like pressure sensor having n vertical × n horizontal sensors, and the x coordinates of the positions of the sensors (coordinates in the left-right direction of the driver) are calculated as x i , y coordinate (the driver's front-to-rear coordinate) j (i and j are integers from 1 to n), (x i ,y j ) sensor pressure p ij Then, the pressure center of gravity coordinate G(X, Y) is expressed by equation (1). Note that the two circles in Figure 3 indicate the approximate positions of the left and right ischial bones when the driver is seated.

[0033]

number

[0034] The time changes in the calculated center of gravity coordinates are shown in Figures 4 and 5. Figure 4 shows the results when the driver was a man in his 40s, and Figure 5 shows the results when the driver was a woman in her 20s. (a) to (g) in Figures 4 and 5 are heat maps of the seat pressure distribution at the start of driving and every 10 minutes after the start of driving. (h) shows the time changes in the X and Y coordinates of the center of gravity coordinate G calculated every second using equation (1), and (i) is a plot of the time changes in the center of gravity coordinate G in (h) on a plane coordinate system. The seat pressure values ​​used in Figures 4 and 5 were measured using an SR Soft Vision (registered trademark) manufactured by Sumitomo Riko Co., Ltd.

[0035] As shown in Figure 4(i), it can be seen that the pressure center of gravity coordinate of a man in his 40s changes significantly back and forth while driving. Furthermore, as shown in Figure 5(i), it can be seen that the pressure center of gravity coordinate of a woman in her 20s also changes significantly back and forth while driving, but the range of change in the center of gravity coordinate is smaller than that of the man in his 40s shown in Figure 4(i). The estimation means 12 may be configured to transmit such time-dependent changes in the center of gravity coordinate to a computer or smartphone in real time for monitoring.

[0036] Next, based on the barycentric coordinate of the seating pressure distribution for each time in a specified time range, the time change of the barycentric coordinate is quantified. As a method of quantification, a method of approximating the distribution range when the barycentric coordinate at each time is plotted on a plane coordinate with an ellipse, or a method of finding the circularity of the distribution range, is used. In the method of approximating the distribution range when the barycentric coordinate at each time is plotted on a plane coordinate with an ellipse and finding the approximate ellipse, as shown in Figure 6, first, the distance between the barycentric coordinate point at each time and the line y = -xb is found, and the maximum of the found distances is taken as L. max , the minimum value is L min Then, L=L max -L minThen, calculate the length L of the major axis of the ellipse. Also, calculate the distance between each centroid coordinate point and the line y = xb, and calculate the maximum value of the calculated distances as S. max , the minimum value is S min Then, S=S max -S min Then, find the length S of the minor axis of the ellipse, where b>n.

[0037] Using the major axis L and minor axis S of the ellipse thus obtained, for example, Qe=Log 10 Find the value of (L / S). When the value of Qe is zero, L=S, and it becomes a circle. Also, when the value of Qe is positive, it becomes an ellipse that slopes upward to the right on the plane coordinate system, and the larger the value of Qe, the more elongated the ellipse becomes. Also, when the value of Qe is negative, it becomes an ellipse that slopes downward to the right on the plane coordinate system, and the smaller the value of Qe, the more elongated the ellipse becomes. In this way, Qe=Log 10 Using the value of (L / S), the distribution (change over time) of the barycentric coordinates can be quantified using the sign and numerical value of the Qe value, with the reference value of the bias of the barycentric coordinates of pressure set to Qe=0.

[0038] In the method of quantifying the distribution range when the coordinates of the center of gravity at each time are plotted on a plane coordinate system using circularity, the area of ​​the distribution range is S P , the perimeter of the distribution range is L P Then, 4πS P / L P 2 If the distribution range is circular, the circularity can be calculated by S P =πr 2 , L P = 2πr (where r is the radius of the distribution range), the circularity is 1. When quantifying, the distribution range of each centroid coordinate is traced by image analysis to determine the area S of the distribution range. P and perimeter L P can be calculated, and the circularity can be calculated.

[0039] Next, using the Qe value and circularity, which quantify the time change in the center of gravity coordinate, for example, the relationship between the value and the driver's fatigue state is determined in advance, and the Qe value, circularity, etc. are determined in real time while driving, thereby estimating the driver's accumulated fatigue state and the driver's fatigue state in real time. In a more specific example, the center of gravity coordinate of the pressure distribution is determined at one-second intervals, and a quantified value is determined based on the distribution of the determined center of gravity coordinates at 10-minute intervals or for the most recent 10 minutes, thereby making it possible to estimate the driver's fatigue state while driving in almost real time.

[0040] For example, Figures 7(a) and (b) show the distribution of barycentric coordinates of pressure for each driver shown in Figures 4 and 5 for every 10 minutes from the start of driving. In Figure 7(a), no significant changes are observed in the distribution of barycentric coordinates while driving. However, in Figure 7(b), the distribution of barycentric coordinates is almost circular for the first 10 minutes after the start of driving, but after 60 minutes (the 10 minutes from 50 to 60 minutes after the start of driving), the distribution of barycentric coordinates changes to an ellipse, clearly indicating the influence of fatigue. In this case, for example, by calculating the value of Qe using the method shown in Figure 6, it is possible to estimate the driver's fatigue state while driving based on the distribution of barycentric coordinates for the 10 minutes.

[0041] [Time-frequency analysis of barycentric coordinates] When the estimation means 12 estimates the driver's fatigue state using time-frequency analysis, it first obtains the coordinates of the center of gravity of the seat pressure distribution at each time, similar to the time series analysis. Next, it performs a short-time Fourier transform (STFT) or a discrete wavelet transform (DWT) on the seat pressure data of the coordinates of the center of gravity of the seat pressure distribution at each time within a predetermined time range. Furthermore, in the case of the short-time Fourier transform, a power spectrum is obtained as a feature, and in the case of the discrete wavelet transform, an energy distribution at a predetermined frequency is obtained, and the time change of the feature is obtained.

[0042] For example, if the driver becomes fatigued, the center of gravity will be placed on one of the ischial bones, causing the pelvis to distort and the center of gravity of the body to shift. Therefore, by detecting this center of gravity shift from changes in feature values ​​at a predetermined frequency, it is possible to detect changes in the driver's posture and estimate fatigue. Furthermore, for example, by calculating the barycentric coordinates of the pressure distribution at one-second intervals and performing STFT or DWT based on the distribution of the barycentric coordinates over a recent predetermined time range, it is possible to estimate the driver's fatigue state in almost real time while driving.

[0043] The driver fatigue estimation method according to the embodiment of the present invention can be suitably implemented by a driver fatigue estimation system 10. The driver fatigue estimation system 10 and the driver fatigue estimation method measure the force with which the driver's left and right ischial bones press against the seat surface of the driver's seat, i.e., seat pressure, which is less susceptible to the influence of ambient conditions such as light shining on the driver, and less susceptible to the influence of body movement noise caused by driving compared to fluctuations related to the autonomic nervous system such as heart rate. This makes it possible to suppress a decrease in the signal-to-noise ratio of the measured seat pressure, enabling highly accurate estimation of the fatigue state.

[0044] Furthermore, because the driver's face is not photographed in the driver's fatigue estimation system 10 and the driver's fatigue estimation method, the driver cannot be identified and there is no risk of privacy being infringed. Furthermore, to measure seat pressure, a sensor or seat pressure measurement means 11 can be attached to the seat surface or inside the seat, and there is no need to attach it directly to the driver, so measurements can be made safely and non-invasively without getting in the way of driving.

[0045] The driver fatigue estimation system 10 and the driver fatigue estimation method detect variations in seat pressure distribution at predetermined time intervals (for example, every one second) and perform various analyses, thereby estimating the driver's accumulated fatigue and state of fatigue in near real time. Furthermore, differential analysis can detect variations in seat pressure according to individual differences between drivers. This makes it possible to reduce the influence of disturbances such as body movement noise, thereby improving the accuracy of fatigue estimation.

[0046] The driver fatigue estimation system 10 may also have an output means for outputting the estimation result to the driver when the estimation means 12 estimates that the driver is fatigued. Furthermore, the estimation means 12 may be configured to compare the quantified value (number of circles, Qe value, circularity, feature amount, etc.) with the fatigue state, set a threshold value in advance when the value reaches a level that interferes with safe driving, and issue an alarm or the like to the driver when the threshold value is exceeded. In these cases, presenting the estimation result to the driver or issuing a warning or the like to the driver depending on the estimated fatigue state can be useful for managing the driver's fatigue while driving and can reduce the risk of accidents.

[0047] Furthermore, the driver fatigue estimation system 10 may estimate fatigue using the driver fatigue estimation method according to the embodiment of the present invention shown in Fig. 8, as follows. That is, the estimation means 12 measures the seat pressure distribution while driving for a predetermined time (e.g., 60 minutes in Fig. 8) (seat pressure measurement step; step 21), and then first performs topological data analysis (TDA) on the measurement data (TDA step; step 22). If the topological data analysis does not predict fatigue (F; false), it performs time series analysis (PSA) (PSA step; step 23). If the time series analysis does not predict fatigue (F; false), it performs time frequency analysis (TFA) (TFA step; step 24), thereby estimating the driver's fatigue state. If either analysis predicts fatigue (T; true) (fatigue determination step; step 25), the output means 12 outputs a result indicating fatigue. Furthermore, if neither analysis determines that the driver is fatigued (F; false) (no fatigue determination step; step 26), the output means 12 outputs that the driver is not fatigued, or steps 22 to 24 are repeated at predetermined time intervals while continuously measuring the seat pressure distribution (seat pressure measurement step), thereby enabling a more accurate estimation of the driver's fatigue state. [Example]

[0048] One subject (driver) drove in a driving simulator for four hours, and the seat pressure distribution was measured using the seat pressure measurement means 11 immediately after the start of driving and every 40 minutes after the start of driving. Measurement results 40, 80, 120, 160, 200, and 240 minutes after the start of driving are shown in Figures 9 to 14, respectively. (a) to (g) in each figure show two-dimensional heat maps of the seat pressure distribution from the start of each measurement, starting at (a) 0 seconds (starting point), (b) 30 seconds, (c) 60 seconds, (d) 90 seconds, (e) 120 seconds, (f) 150 seconds, and (g) 180 seconds. In each figure, (h) shows a graph of the time change in the X and Y coordinates of the center of gravity calculated every second, and (i) shows a graph in which the time change in the center of gravity coordinates in (h) is plotted on a plane coordinate system. Note that a "seat sensor (seat pressure distribution measuring device)" manufactured by XSENSOR is used as the seat pressure measuring means 11. From the results of measuring the seat pressure distribution, it was confirmed that the seat pressure distribution changes over time, particularly in Figures 9, 11, 13, and 14.

[0049] Next, using the seat pressure distribution shown in FIG. 14, which is considered to represent a "fatigue" state due to changes in the seat pressure distribution among FIGS. 9 to 14, the estimation unit 12 performed topological data analysis, time series analysis, and time-frequency analysis. The analysis results are shown in FIG. 15. FIG. 15(a) shows the seat pressure distribution (measurement data) shown in FIG. 14(a), FIG. 15(b) shows a map of a simplicial complex obtained by topological data analysis of the measurement data, FIG. 15(c) shows a graph plotting the time change in the center of gravity coordinate shown in FIG. 14(h) on a plane coordinate system, which is used in the time-series analysis, and FIG. 15(d) shows a power spectrum obtained by time-frequency analysis. For comparison, the estimation unit 12 also performed each analysis using the seat pressure distribution of the same driver in a "non-fatigue" state, that is, from immediately after the start of driving (0 seconds) until 180 seconds have elapsed. The analysis results are shown in FIG. 16, as in FIG. 15.

[0050] In topological data analysis, a simplicial complex was constructed using a Lips complex. Figure 15(b) shows one circle, while Figure 16(b) shows two. In time series analysis, the L / S value when approximated to an ellipse was 7.786 in Figure 15(c) and 0.494 in Figure 16(c). In time-frequency analysis, the power spectrum of the short-time Fourier transform (TSFT) showed a peak frequency of 1.00 Hz, a power of 52.89, and a time T of 18.6 seconds in Figure 15(d), while the peak frequency was 1.00 Hz, a power of 44.03, and a time T of 21.4 seconds in Figure 16(d).

[0051] The results of Figures 15 and 16 confirm that in both analyses, quantified values ​​such as the number of circles, L / S value, and power spectrum power differ significantly depending on whether or not the driver is fatigued. Therefore, it can be said that by setting appropriate standards and thresholds for these values, it is possible to accurately estimate whether or not the driver is fatigued.

[0052] Using the seat pressure distribution shown in Fig. 14, the estimation means 12 performed a wavelet transform through time-frequency analysis, and it was confirmed that the frequency of minute shaking was 0.9 Hz to 1 Hz, and the frequency of weight shift was 0.85 Hz to 1.25 Hz. Therefore, it can be said that by detecting changes in these frequency components, it is possible to detect changes in the driver's posture and estimate fatigue. [Explanation of symbols]

[0053] 10 Driver fatigue estimation system 11. Seat pressure measurement means 12 Estimation means

Claims

1. a seat pressure measuring means for measuring a seat pressure distribution in which the left and right ischial bones of a driver seated in the driver's seat of a vehicle press against the seat surface of the driver's seat; an estimation means for performing a topological data analysis, a time series analysis of a center of gravity coordinate calculated from the seat pressure distribution, and a time frequency analysis of the center of gravity coordinate on the seat pressure distribution measured by the seat pressure measurement means, and estimating a fatigue state of the driver based on the results of these analyses; A driver fatigue estimation system comprising:

2. The driver fatigue estimation system according to claim 1, characterized in that the estimation means constructs a simplicial complex from the seat pressure distribution at each time using the topological data analysis, calculates the number of holes in the first-order homology of the simplicial complex, and estimates the driver's fatigue state based on the calculated number of holes.

3. 3. The system for estimating driver fatigue according to claim 2, wherein said estimating means estimates the driver's fatigue state as being fatigued when the number of holes is one.

4. The driver fatigue estimation system according to claim 1, characterized in that the estimation means uses the time series analysis to approximate the distribution range when the center of gravity coordinates for each time period within a predetermined time range are plotted on plane coordinates, and estimates the driver's fatigue state based on the shape of the ellipse.

5. The driver fatigue estimation system according to claim 1, characterized in that the estimation means calculates a circularity for a distribution range when the center of gravity coordinates for each time in a predetermined time range are plotted on a plane coordinate system using the time series analysis, and estimates the driver's fatigue state based on the circularity.

6. The driver fatigue estimation system according to claim 1, characterized in that the estimation means performs a short-time Fourier transform (STFT) or a discrete wavelet transform (DWT) on the seat pressure data of the center of gravity coordinates at each time within a predetermined time range using the time-frequency analysis, and estimates the driver's fatigue state based on the transformation results.

7. 2. The system for estimating driver fatigue according to claim 1, wherein the estimation means estimates the driver's fatigue state based on a change over time in the analysis result.

8. 2. The system for estimating driver fatigue according to claim 1, further comprising output means for outputting an estimation result to said driver when said estimating means estimates that said driver is fatigued.

9. The driver fatigue estimation system according to any one of claims 1 to 8, characterized in that the estimation means is configured to first perform the topological data analysis, and if the topological data analysis does not result in an estimation of fatigue, perform the time series analysis, and if the time series analysis does not result in an estimation of fatigue, perform the time frequency analysis, thereby estimating the driver's fatigue state, and to repeatedly estimate the driver's fatigue state at predetermined time intervals.

10. a seating pressure measuring step of measuring a seating pressure distribution in which the left and right ischial bones of a driver seated in a driver's seat of a vehicle press against a seat surface of the driver's seat; a step of performing a topological data analysis (TDA) on the measured seating pressure distribution, a time series analysis of the center of gravity coordinates obtained from the seating pressure distribution, and a time frequency analysis of the center of gravity coordinates, and estimating the fatigue state of the driver based on the results of these analyses; A method for estimating driver fatigue, comprising:

11. 11. The method for estimating driver fatigue according to claim 10, wherein the estimation step first performs the topological data analysis, and if the topological data analysis does not indicate that the driver is fatigued, performs the time series analysis, and if the time series analysis does not indicate that the driver is fatigued, performs the time frequency analysis, thereby estimating the driver's fatigue state, and repeatedly estimating the driver's fatigue state at predetermined time intervals.

Images