Method and device for hands-free motion detection
The method uses signal preprocessing, autoregressive models, and Fourier transformations with linear discriminant analysis to detect freehand driving independently of vehicle-specific data, improving safety by ensuring driver engagement.
Patent Information
- Application Number
- DE102010033744
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2010-08-07
- Publication Date
- 2024-12-05
- Estimated Expiration
- 2030-08-07
AI Technical Summary
Existing methods for detecting freehand driving in vehicles rely on vehicle-specific parameters such as speed and engine speed, requiring specific data for each vehicle series, which complicates the detection process.
A method involving signal preprocessing, feature extraction using autoregressive models and Fourier transformations, and classification with linear discriminant analysis to determine whether a driver is hands-free, independent of vehicle-specific data.
Provides a reliable and simplified method for detecting freehand driving, reducing reliance on vehicle-specific parameters and enhancing safety by ensuring driver engagement.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[0001] The invention relates to a method for detecting a hands-free driving of a motor vehicle according to the preamble of claim 1 and a corresponding device according to the preamble of claim 7.
[0002] Assistance systems in modern vehicles, such as adaptive cruise control, cruise control, or lane keeping systems, are increasingly simplifying or relieving drivers of more and more tasks. These assistance systems aim to make road traffic safer. In particular, they are intended to prevent accidents caused by driver inattention. However, there is a risk that drivers will rely too heavily on these assistance systems. Since these systems do not offer complete safety, care must be taken to ensure that the driver retains control of the vehicle. This includes systems that detect and monitor driver activity, as an active driver is a prerequisite for the functionality of some driver assistance systems.One way to determine driver activity is to determine whether the driver is in a hands-free state, since this can be interpreted as a condition in which the driver's ability to act is limited. For example, in driver assistance systems with lane guidance support, it is necessary to continuously monitor the contact state of the driver's hands with the steering wheel to avoid a driving situation in which the driver is not in contact with the steering wheel, i.e., a hands-free state.
[0003] For example, DE 10 2005 056 438 A1 discloses a method for determining hands-free driving. The method measures accelerations at the steering wheel, and uses the measured accelerations to determine a transfer function, which is then compared with a reference transfer function. The reference transfer function represents the system behavior when the steering wheel is not gripped. If the measured transfer function matches the reference transfer function, the driver has not gripped the steering wheel, i.e., hands-free driving is occurring. However, the transfer functions, including the reference transfer function, are in particular a function of the engine speed and the vehicle speed. Consequently, the vehicle speed and engine speed range must be covered by a sufficient number of predetermined and stored reference functions in order to implement the method.
[0004] Furthermore, the document DE 10 2007 039 332 A1 describes a method for detecting the contact state of at least one hand of the driver with the steering wheel of the vehicle, wherein the steering movement of the steering wheel is detected by means of a torque sensor and an angle sensor, the free steering movement of the steering wheel is modeled taking non-linear friction effects into account using experimental measurement data, the driver steering torque is determined as a state by means of a state observer, and that the state observer is used to estimate the driver steering torque and the contact state between the driver's hands and the steering wheel is estimated by means of the estimated driver steering torque.
[0005] Furthermore, DE 10 2008 042 277 A1 discloses a method and device for determining a frictional connection and / or a force transmission between a controlling unit, in particular a person and a mounted shaft or axle, wherein a sensor is designed to detect at least one variable which can be determined for the frictional connection and / or the force transmission and in doing so carries out more than one measurement at different times within a time interval, characterized in that a pattern which enables the determination is evaluated from the more than one measurement.
[0006] A disadvantage of the known methods is that they depend in various ways on vehicle-specific parameters such as vehicle speed, engine speed or experimental measurement data, etc., so that specific data must be determined at least for each model series of a motor vehicle.
[0007] The invention is therefore based on the object of creating a simplified and reliable method and a corresponding device for detecting a hands-free journey, which is independent of vehicle-specific data.
[0008] This object is achieved by a method for detecting a hands-free movement having the features of claim 1 and by a corresponding device having the features of claim 7. Preferred embodiments of the invention are the subject of the dependent claims.
[0009] The method according to the invention for detecting a hands-free driving of a driver of a motor vehicle comprises the following steps: - Determining the time course of the steering torque signal applied to the steering wheel of the motor vehicle, - Extracting features from the time course of the steering torque signal, - Summarizing the extracted features in a feature vector, - Classification of the feature vector in a classifier into a class concerning freehand driving and a class concerning hands-on driving.
[0010] The steering torque signal undergoes preprocessing adapted to the subsequent feature extraction. In other words, the signal preprocessing must consider the manner in which the subsequent feature extraction will be performed. For subsequent processing in the frequency domain, it is disadvantageous if the signal is subjected to prior smoothing, such as using a moving-average filter, because this filter would remove frequency information. On the other hand, smoothing using such a filter is advantageous for subsequent processing in the time domain.
[0011] The steering torque signal undergoes trend component elimination during signal preprocessing. This may be necessary, for example, if the steering torque signal is superimposed with another signal not caused by the driver. This additional signal must be removed using appropriate measures to obtain the driver's direct steering torque.
[0012] As already mentioned above, in order to process the steering torque signal in the time domain, it is subjected to smoothing in the signal preprocessing and / or in order to process the steering torque signal in the frequency domain, it is subjected to convolution with a window function in the signal preprocessing, whereby a Hamming window, for example, can be used as the window function.
[0013] Preferably, for feature extraction in the time domain, the smoothed steering torque signal at a time t is compared with an estimate of the steering torque signal using an autoregressive model, wherein the estimate is determined as a weighted linear combination of a predetermined number p of past steering torque signals tp, p=1, ..., k, and the coefficients of the autoregressive model for a predetermined number p of time intervals are used as features to form the feature vector. The motivation for using an autoregressive model is that the steering torque signal exhibits a high degree of similarity in local environments, i.e., there is a high correlation in the neighborhood. For any existing steering torque value at time t i It can be observed that the successor value at time t i+1with a high probability it will barely differ from the previous value. With a somewhat lower probability, it will change slightly, and it becomes increasingly unlikely that the value will deviate significantly from the previous value. This means that the steering torque values exhibit a predictable structure when the driver has their hands on the steering wheel and a rather unpredictable one when the driver is driving hands-free. A sequence of such measurement events can be generated using the autoregressive model, or AR model. The AR model can be used to forecast steering torque measurement events beyond the observation period and thus provides coefficients as features that encode information about the steering torque.
[0014] Further preferably, for feature extraction in the frequency domain, the steering torque signal convolved with the window function is subjected to a discrete Fourier transformation, and the resulting Fourier components represent features for forming the feature vector. The analysis of the steering torque signal using discrete Fourier analysis is based on the oscillation of the steering torque, with the frequency components being obtained by the Fourier transformation of the steering torque. Placing the hands on the steering wheel has a dampening effect on the frequency components of the steering torque signal. Therefore, deflections in the high frequency band of the steering torque can be observed when the driver is driving hands-free, and deflections in the low frequency band of the steering torque can be seen when the driver places their hands on the steering wheel. Therefore, the Fourier components of the steering torque signal can be used as features for forming feature vectors.
[0015] Further preferably, for feature extraction in the time domain for an electromechanical steering system that can be subjected to a controller torque, the cross-correlation of the controller torque and the driver's steering torque is calculated, and the determined cross-correlations are used as features to form the feature vector. From the cross-correlation coefficients, it can be concluded whether or not a hands-free driving situation is present.
[0016] For the subsequent classification of the feature vector(s), the respective feature space is divided into two classes using Fisher's linear discriminant analysis. In other words, linear discriminant analysis is used to find a discriminant function that divides the respective feature space into two classes, so that the membership of a feature vector in one of the two classes can be easily determined.
[0017] To determine the discriminant function that divides the respective feature space of the classifier into two classes, an estimation, i.e., a calculation, of the discriminant function is performed using a training set of feature vectors whose class membership is known. In other words, with regard to the question of whether or not a hands-free ride occurred, training drives are conducted to determine feature vectors for which it is known whether the driver has gripped the steering wheel.
[0018] A device according to the invention, which is designed to carry out the method described in the preceding, comprises - a sensor for determining the steering torque applied to the steering wheel, - a device for preprocessing the steering torque signal, - a device for extracting features from the preprocessed steering torque signal and for generating feature vectors, and - a device for classifying the feature vectors, whereby the classification is carried out by a linear discriminant analysis.
[0019] Further preferably, the device comprises a device for determining the discriminant function of the linear discriminant analysis from a training set of feature vectors of known class membership.
[0020] A preferred embodiment of the invention is explained below with reference to the drawings. Fig. 1 an example of a time course of a steering torque exerted by a driver, Fig. 2 the basic structure of a hands-on system, Fig. 3a - 3e show signal preprocessing and feature extraction using the example of a discrete Fourier transform in the frequency domain, Fig. 4 the modelling of the discriminant function of a linear discriminant analysis according to Fisher, Fig. 5 an example of hands-on detection.
[0021] Fig. Figure 1 shows the time course of a steering torque detected by a torque sensor on an electromechanical steering system. If the electric motor of the steering system does not provide assistance, the steering torque applied by the driver is measured. However, if electric motor steering assistance is provided, for example, because a lane keeping system intervenes to keep the vehicle in the lane, the driver's steering actions may be distorted. This can be seen in Fig. 1, that the steering torque signal fluctuates around the zero position depending on the driver intervention, with the steering torque exhibiting large and small oscillations.
[0022] For a hands-free driving detection system, also known as a hands-on system, the following signals are generally relevant, i.e., they can be used to detect hands-free driving: steering torque, steering wheel angle, controller torque (torque of the electric motor of the electromechanical steering system), vehicle speed, and lane data (curvature of the lane center, lane width). It is obvious that, of the available signals mentioned, the driver's steering torque contains the most information regarding the hands-free driving problem, thus forming the basis for hands-on detection.
[0023] Fig. Figure 2 shows the basic structure of the hands-free driving or hands-on detection system according to the invention. The steering torque signal 1, determined by a steering torque sensor (not shown), is fed to a hands-free driving detection system 2, which has three processing stages: preprocessing 3 of the signal, subsequent feature extraction 4 from the preprocessed signal, and classification 5 of the extracted features. The result 6 provides the logical statement as to whether or not hands-free driving is occurring, i.e., whether the driver has their hands on the steering wheel.
[0024] Important features for hands-free detection are found in the steering torque oscillations, as placing the hands on the steering wheel has a dampening effect on the frequency components of the steering torque signal. Therefore, peaks in the high frequency band of the steering torque can be observed when the driver is driving hands-free, and peaks in the low frequency band of the steering torque can be seen when the driver places their hands on the steering wheel. Therefore, the frequencies are examined and analyzed in the frequency domain, so that the steering torque signal is subjected to a Fourier transformation.
[0025] Further features can be obtained by formalizing measurement events of the steering torque signal in a model and estimating the corresponding model parameters. The approach adopted here is based on the following considerations: The steering torque signal is highly similar in local environments, ie, there is a high correlation in the neighborhood. For any given steering torque value at time t i It can be observed that the successor value at time t i+1with a high probability it will hardly differ from the previous value. With a somewhat lower probability it will change slightly, and it becomes increasingly unlikely that the value will deviate very significantly from the previous value. This means that the steering torque values have a predictable structure when the driver has their hands on the steering wheel and a rather unpredictable one when the driver is driving hands-free. One mechanism that could generate such a sequence of measurement events is the autoregressive process (AR model). The AR model can be used to forecast steering torque measurement events beyond the observation period and thus provides coefficients that encode information about the steering torque.
[0026] The cross-correlation between controller and steering torque represents another option for calculating features for hands-free detection. First, both signals are appropriately scaled. If the steering torque signal correlates with the controller torque signal, it can be assumed that the vehicle is being driven hands-free. If, in contrast, no correlation can be determined between the two signals, the vehicle is being driven with the hands clasped.
[0027] The obtained features can be used individually or in combination.
[0028] Fig. Figure 3 now shows the preprocessing and feature extraction of a steering torque signal in five steps a to e using a feature extraction in the frequency domain.
[0029] In this case, Fig. 3a shows the determined steering torque signal 7, which is subject to a trend. The trend can have many causes, for example, a drift of the steering torque sensor or an overlay with a controller torque of the torque-generating controller of the electromechanical steering. By generating a trend line 8, for example, using a least-squares method, the trend can be removed from the input signal 7, resulting in the steering torque signal 9 without a trend, as described in part b) of the Fig. 3 is shown.
[0030] In order to be able to process the signal in the frequency domain, the steering torque signal 9 without trend is converted into a periodic form in the usual way, ie periodized. This can be done, for example, by convolving the steering torque signal 9 without trend with a window function, where the window function used here is the value specified in part c) of the Fig. 3 is used. The result of the convolution is shown in Fig. 3d) as the steering torque signal 10 convolved with a Hamming window without trend.
[0031] This steering torque signal 10 without trend, folded with a Hamming window, is then transformed into the frequency domain using a discrete Fourier transformation, resulting in Fig. 3e) symbolically represented Fourier coefficients, which are used to form a feature vector, where Fig. 3e) the frequency in Hz is plotted against the amplitude.
[0032] For the sake of completeness, the discrete Fourier transform for N input values x(0), x(1), ..., x(N-1) is given below, which is defined as: y(k)=1N∑n=0N−1x(n)exp(−j2πNkN), k=0, 1 …, N−1 and j=−1
[0033] The Fig. 3e) The Fourier coefficients shown as examples can be used in whole or in part to form a feature vector for subsequent classification. However, if all coefficients are used, the feature vector will have an unfavorable number of dimensions for further processing. In order to keep the dimension of the feature vectors within reasonable limits, the coefficients are summarized or selected, whereby care must be taken when selecting the coefficients to reduce the redundancy present in the coefficients. Since, when detecting a hands-free movement, the presence of low frequencies indicates a hands-on movement, it is sufficient to form a feature vector only from the frequencies that lie below a predetermined cutoff frequency of, for example, 6 Hz. However, the feature vector can also be formed from the high frequencies indicating a hands-free movement.
[0034] Similarly, features can also be separated from the time domain of the steering torque signal, which can be used to form feature vectors for detecting hands-free driving.
[0035] Another possible approach is based on the following considerations: The steering torque signal is highly similar in local environments, ie, there is a high correlation in the neighborhood. Suppose we have any value of the steering torque at sampling time t i and we look at the next sampling time t i+1This value will, with a high probability, hardly differ from the previous value. With a somewhat lower probability, it will change slightly, and it becomes increasingly unlikely that the value will deviate very significantly from the previous value. Applied to the hands-on problem, this means that the steering torque values exhibit a predictable structure when the driver has their hands on the steering wheel and a rather unpredictable one when the driver drives hands-free. This internal dependency structure of a series of measurement events of the steering torque signal must be formalized in a model, and the corresponding model parameters must be estimated. The resulting coefficients encode required information about the steering torque and are suitable as features for a feature vector. One mechanism that can generate such a sequence of measurement events is the p-th order autoregressive process (AR model).First, let I(t) be a time-discrete stochastic signal, here a sequence of steering torque signals, where t = p + 1,..., n are equidistant points in time. Furthermore, it is assumed that the steering torque sequence I(t) is stationary, i.e., it is time-shift invariant, and its autocorrelation sequence r(k) of the form exists. r(k)=E(l(t)l(t−k)) where k = 1,... ,p is the time delay and E(X) is the expected value of a stochastic process X. The current steering torque value I(t) is calculated in a defined time window as the sum l(t)=l^(t)+η(t) where [l̂(t) is the estimate of I(t) and η(t) denotes the estimation error. The estimated value l̂(f) is determined as a weighted linear combination based on p past values, ie l^(t)=∑k=1pa(k)l(t−k)
[0036] Instead of the sum in the above equation, a vector matrix notation is used, where a→(p) the column vector with elements a(I),... ,a(p) and l→p(t−1) is the column vector with elements l(t-1),... l(tp). Consequently, a→T(p) the transposed vector of a→(p), a row vector. Using this vector-matrix notation, the linear combination can be written as l^(t)=a→T(p)⋅l→p(t−1) where the vector a→T(p) is the parameter vector of the autoregressive model, which can be optimally estimated if the total squared estimation error is minimized. The components of the vector can be used as features of a feature vector to determine a freehand trip.
[0037] In order to be able to decide whether hands-free driving is taking place based on the feature vectors, a classifier is required that decides whether a given feature vector belongs to a “hand-off” class, i.e. hands-free driving, or to a “hand-on” class, i.e. the driver has grasped the steering wheel. This is done by setting up a parametric model that can unambiguously assign a feature vector to one of the two classes. The class membership of the feature vectors available for developing the model is known because these feature vectors are formed through suitable training drives. They can therefore be used to monitor the classifier’s learning process. The goal is therefore to define a model for the classifier and to estimate its model parameters through a suitable training phase.
[0038] Fig. Figure 4 shows an example of the spatial distribution of two-dimensional feature vectors in an abstract feature space with values between -5 and 5 for the y-axis and values between -4 and 4 for the y-axis, where the signs “+” represent a hands-off situation and the signs “ * " represent a hands-on situation. It is clear that this exemplary training room cannot be completely separated linearly.
[0039] To ensure online application in motor vehicles, assuming reasonable computing capacity, a classification method is used that separates the two classes linearly, and it is assumed that a suitable discriminant function is available or known. This allows the parameters of this discrimination function to be estimated from the available training set.
[0040] Therefore, a classification problem for two classes is considered, where w1 is the first class and W2the second class. The Fig. 4 The linear discriminant function denoted by LD is a linear combination of the components of a feature vector x→ and can be used as g(x→)=ω→Tx→+ω0=ω0+∑i=1dωixi=∑i=0dωixi be written, where ω→ the weight vector, d the dimension of the feature vector x→ and ω0 is a threshold value.
[0041] Using Fischer's linear discriminant method, the linear discriminant function can be determined from a training set with known class membership of the feature vectors for both feature vectors in the frequency domain and those in the time domain.
[0042] Fig.Figure 5 shows the result of hands-on detection using the discrete Fourier transform. It shows the steering torque 1 and the hands-on situations 13 determined using the method. The value "1" indicates that the driver has grasped the steering wheel, while the value "0" means hands-free driving. Above this, the actual hands-free driving function is indicated, with the value "4" indicating hands-free driving and the value "5" indicating hands-on driving.
[0043] Detectable deviations of the hands-on detection from the actual function 14 arise from the feature space, which cannot be completely linearly separated. However, this is only of minor importance for the method, since a reaction of a lane keeping system due to hands-on driving may only occur after a predetermined period of hands-on driving in order to prevent system oscillation. Therefore, the classification results are preferably subjected to temporal smoothing, and plausibility rules can also be introduced, for example, by logically linking the steering torque input signal with the controller torque. Furthermore, the classifications can be combined using the Fourier transformation, the autoregressive model, and the cross-correlation between steering torque and controller torque, which further increases the determination of the probability of hands-on driving. List of reference symbols 1 steering torque 2 Hands-free travel detection 3 Preprocessing 4 Feature extraction 5 Classification 6 results 7 Steering torque signal with trend 8 Trend line 9 Steering torque signal without trend 10 Hamming windows 11 Steering torque signal without trend folded with Hamming window 12 Fourier components 13 Hands-On 14 labeled hands-on functions LD Linear Discriminant Function
Claims
[1] Method for detecting a hands-free driving of a driver of a motor vehicle, comprising the following steps: Determining a time profile of a steering torque signal (1, 7) applied to the steering wheel of the motor vehicle, Extracting (4) features characterizing the hands-free driving from the time course of the steering torque signal (1, 7), Summarizing the extracted features in a feature vector, Classification (5) of the feature vector in a classifier into a class concerning the freehand driving and into a class concerning the hands-on driving, characterized byin that the steering torque signal (1, 7) is subjected to signal pre-processing (3) which is adapted to a later feature extraction (4), wherein the steering torque signal (1, 7) is subjected to an elimination of trend components (8) in the signal pre-processing (3), wherein in order to process the steering torque signal (1, 7) in the time domain, this is subjected to a smoothing in the signal pre-processing (3) and in order to process the steering torque signal (1, 7) in the frequency domain, this is subjected to a convolution with a window function (10) in the signal pre-processing (3). [2] Method according to claim 1, characterized bythat for feature extraction (4) in the time domain, a smoothed steering torque signal at a time t is compared with an estimate of the steering torque signal by means of an autoregressive model, wherein the estimate is determined as a weighted linear combination of a predetermined number of past steering torque signals and coefficients of the autoregressive model for a predetermined number p of previous time intervals are used as features to form the feature vector. [3] Method according to claim 1, characterized by that for feature extraction (4) in the time domain in an electromechanical steering system which can be subjected to a controller torque, the cross-correlation of the controller torque and a steering torque of the driver is determined and the determined cross-correlations are used as features for forming the feature vector. [4] Method according to claim 1, characterized bythat for feature extraction (4) in the frequency domain, the steering torque signal convolved with the window function (10) is subjected to a discrete Fourier transformation and the resulting Fourier components (12) are used as features to form the feature vector. [5] Method according to one of the preceding claims, characterized by that in order to classify the feature vector(s), a respective feature space is divided into two classes by a linear discriminant analysis. [6] Method according to claim 5, characterized by that a discriminant function (LD) for subdividing the respective feature space of the classifier is estimated from a training set of feature vectors whose class membership is known. [7] Device designed to carry out the method according to one of the preceding claims with a sensor for determining the steering torque exerted on the steering wheel (1), a device for preprocessing the steering torque signal (1, 7), a device for extracting features from the preprocessed steering torque signal and for generating feature vectors, a device for classifying the feature vectors, whereby the classification is carried out by a linear discriminant analysis, characterized by a device for pre-processing (3) of the steering torque signal (1, 7) adapted to the later feature extraction (4), wherein trend components (8) are eliminated from the steering torque signal (1, 7) by the pre-processing (3), wherein for processing the steering torque signal (1, 7) in the time domain, the latter is subjected to smoothing in the signal pre-processing (3) and for processing the steering torque signal in the frequency domain, the latter is subjected to convolution with a window function (10) in the signal pre-processing (3). [8] Device according to claim 7, characterized bythat the device comprises a device for determining a discriminant function (LD) of the linear discriminant analysis from a training set of feature vectors of known class membership.
Citation Information
Patent Citations
Procedure and device for monitoring the steering wheel
DE102005056438A1
Driver`s hand contact condition detecting method for use in vehicle, involves evaluating driver steering moment using condition observer, and detecting contact condition between hands and handle by hand-off detector using evaluated moment
DE102007039332A1
Method for identification of missing driver activity at steering wheel of motor vehicle, involves determining hand moment of driver exerted at steering wheel
DE102008021150A1
Method for determining frictional connection or force transfer, particularly controlling units, particularly for person, involves detecting frictional connection or force transfer of defined size by sensor
DE102008042277A1
motor vehicle steering device
DE19806458B4
