A driver hands-off detection method, system, device, and medium

Abstract

The application discloses a driver hand-off detection method, system, device and medium, and relates to the technical field of automobile driving assistance. The method comprises the following steps: acquiring a target feature sequence of a vehicle, wherein the target feature sequence comprises a plurality of feature data sets with time steps in chronological order, each feature data set of each time step comprises basic feature data and extended feature data; inputting the target feature sequence into a hand-off probability prediction model to obtain an initial hand-off probability, wherein the hand-off probability prediction model is obtained by training a to-be-trained network by using samples, and the samples comprise a historical feature sequence and a hand-off probability label, wherein the historical feature sequence comprises historical basic feature data and historical extended feature data; and comparing the initial hand-off probability with a preset hand-off probability threshold value, and determining whether the driver is in a hand-off state according to a comparison result.

Description

Technical Field

[0001] This application relates to the field of automotive driver assistance technology, and in particular to a method, system, device, and medium for detecting driver hands-off operation. Background Technology

[0002] Currently, with the rapid development of automotive intelligent technology, Advanced Driver Assistance Systems (ADAS) are widely used in various vehicle models, among which driver status monitoring has become a key link in ensuring driving safety. Driver hands-off detection, as one of the core functions of driver status monitoring, aims to determine in real time whether the driver's hands have left the steering wheel, thereby issuing warnings to the driver or triggering safety intervention measures when necessary, ensuring driving safety in human-machine co-driving mode. Current mainstream hands-off detection solutions mainly rely on the hand torque signal collected by the steering wheel torque sensor, and determine whether the driver is holding the steering wheel based on a preset threshold.

[0003] However, traditional judgment methods based on fixed thresholds are difficult to adapt to complex and ever-changing driving environments, and are prone to false alarms or missed alarms under different road conditions and driving conditions. Summary of the Invention

[0004] The summary section introduces a series of simplified concepts, which will be further explained in detail in the detailed description section. This summary section is not intended to limit the key and essential technical features of the claimed technical solution, nor is it intended to determine the scope of protection of the claimed technical solution.

[0005] In a first aspect, embodiments of this application provide a method for detecting driver hands-off operation, the method comprising: Obtain the target feature sequence of the vehicle. The target feature sequence includes multiple feature datasets with time steps in sequence. Each feature dataset of time step includes basic feature data and extended feature data. The basic feature data includes the vehicle's motion parameters, steering wheel angle, and driver's hand torque within the corresponding time step. The extended feature data includes data on the degree of change of the hand torque parameter within the corresponding time step, as well as data on the motion characteristics of the steering wheel angle within the time step. The target feature sequence is input into the release probability prediction model to obtain the initial release probability. The release probability prediction model is trained using samples to train the network to be trained. The samples include: historical feature sequences and release probability labels. The historical feature sequences include historical basic feature data and historical extended feature data. The initial probability of disengagement is compared with a preset probability threshold for disengagement, and the driver is determined to be in a disengagement state based on the comparison result.

[0006] In one embodiment of the present invention, the release probability prediction model includes a fully connected network model and a long short-term memory network model. The step of inputting the target feature sequence into the release probability prediction model to obtain the initial release probability includes: The fully connected layer of the fully connected network model performs a nonlinear transformation on the target feature sequence at each time step and performs dimensionality reduction on the nonlinearly transformed target feature sequence to obtain the first feature sequence. The long short-term memory layer based on the long short-term memory network model performs time-series modeling on the first feature sequence, and then performs regularization on the time-series modeled first feature sequence to obtain the second feature sequence. The second feature sequence is dimensionally compressed based on the fully connected layer of the fully connected network model to obtain the initial release probability.

[0007] In one embodiment of the present invention, after inputting the target feature sequence into the release probability prediction model to obtain the initial release probability, the process includes: The current driving scenario of the vehicle is determined based on lateral acceleration and steering wheel angle. The current driving scenario of the vehicle includes a curve scenario and a straight road scenario. When the current driving scenario is the curve scenario, the product of the preset curve calibration factor and the initial release probability is used as the correction result of the initial release probability. When the current driving scenario is the straight road scenario, the product of the preset straight road calibration factor and the initial release probability is used as the correction result of the initial release probability, and the curve calibration factor is greater than the straight road calibration factor.

[0008] In one embodiment of the present invention, the hand-off probability threshold includes a straight-line hand-off probability threshold and a curve hand-off probability threshold, and determining whether the driver is in a hand-off state based on the comparison result includes: If the current driving scenario is the curve scenario and the initial probability of losing control is greater than the preset curve loss probability threshold, then the driver is determined to be in a hands-free state. If the current driving scenario is a straight road scenario and the initial hand-off probability is greater than the preset straight road hand-off probability threshold, it is determined that the driver is in a hand-off state, and the curve hand-off probability threshold is less than the straight road hand-off probability threshold.

[0009] In one embodiment of the present invention, comparing the initial sell probability with a preset sell probability threshold includes: Obtain multiple initial sell probabilities within a preset time period; The average of multiple initial sell probabilities is compared with a preset sell probability threshold.

[0010] In one embodiment of the present invention, obtaining the target feature sequence of the vehicle includes: The basic feature data and the extended feature data are concatenated to obtain the feature vector; The target feature sequence is obtained based on the feature vectors from multiple consecutive time steps.

[0011] In one embodiment of the present invention, the basic feature data includes vehicle speed, lateral acceleration, longitudinal acceleration, yaw rate, pitch rate, steering wheel angle, and driver's hand torque, and the extended feature data includes the mean hand torque, hand torque variance, hand torque skewness, hand torque kurtosis, hand torque standard deviation, hand torque range, steering wheel angle change rate, and steering wheel angle variance.

[0012] Secondly, this application proposes a driver hands-off detection system, the system comprising: a feature acquisition module, a probability determination module, and a probability comparison module; The feature acquisition module is configured to: acquire a target feature sequence of the vehicle, the target feature sequence including multiple feature datasets of time steps with a time sequence, each feature dataset of time step including basic feature data and extended feature data, the basic feature data including the vehicle's motion parameters, steering wheel angle and driver's hand torque in the corresponding time step, the extended feature data including data on the degree of change of hand torque parameters in the corresponding time step, and data on the motion characteristics of the steering wheel angle in the time step; The probability determination module is configured to: input the target feature sequence into the release probability prediction model to obtain the initial release probability. The release probability prediction model is obtained by training the network to be trained using samples. The samples include: historical feature sequences and release probability labels. The historical feature sequences include historical basic feature data and historical extended feature data. The probability comparison module is configured to compare the initial release probability with a preset release probability threshold, and determine whether the driver is in a release state based on the comparison result.

[0013] Thirdly, an electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program stored in the memory to implement the steps of a driver hands-off detection method as described in any of the first aspects above.

[0014] Fourthly, this application also proposes a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of a driver hands-off detection method according to any one of the first aspects.

[0015] In summary, the driver hands-off detection method of this application acquires a target feature sequence composed of basic feature data and extended feature data, and integrates instantaneous sensor information at a single moment with statistical feature information reflecting the degree of change in hand torque and the characteristics of steering wheel movement, thereby providing a more comprehensive data foundation for subsequent hands-off judgment. Inputting the target feature sequence into a hands-off probability prediction model effectively overcomes the misjudgment problem caused by traditional methods relying solely on threshold judgments at a single moment. Finally, by comparing the initial hands-off probability with a preset hands-off probability threshold, the driver's hands-off state can be clearly determined.

[0016] The driver hands-off detection method proposed in this application, along with other advantages, objectives, and features of this application, will be partly apparent from the following description and partly understood by those skilled in the art through study and practice of this application. Attached Figure Description

[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit this specification. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart illustrating a method for detecting driver hands-off operation, as provided in an embodiment of this application. Figure 2 This is a schematic diagram of a driver hands-off detection system provided in an embodiment of this application; Figure 3 This is a schematic diagram of a driver hands-off detection electronic device provided in an embodiment of this application. Detailed Implementation

[0018] To better understand the technical solutions provided in the embodiments of this specification, the technical solutions of the embodiments of this specification will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this specification and the specific features in the embodiments are detailed descriptions of the technical solutions of the embodiments of this specification, rather than limitations on the technical solutions of this specification. In the absence of conflict, the embodiments of this specification and the technical features in the embodiments can be combined with each other.

[0019] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The term "two or more" includes two or more cases.

[0020] Please see Figure 1 This is a flowchart illustrating a method for detecting driver hands-off operation provided in an embodiment of this application, specifically including: S110. Obtain the target feature sequence of the vehicle. The target feature sequence includes multiple feature datasets with time steps in sequence. Each feature dataset of time step includes basic feature data and extended feature data. The basic feature data includes the vehicle's motion parameters, steering wheel angle, and driver's hand torque within the corresponding time step. The extended feature data includes data on the degree of change of the hand torque parameter within the corresponding time step, as well as data on the motion characteristics of the steering wheel angle within the time step. For example, by acquiring a target feature sequence containing feature datasets of multiple consecutive time steps, basic input data is provided for subsequent driver hands-off state recognition. Each time step's feature dataset consists of basic feature data and extended feature data. The basic feature data covers the vehicle's motion parameters, steering wheel angle, and driver's hand torque within the corresponding time step. This data directly reflects the vehicle's operating state at the current moment and the driver's basic steering wheel operation information. The extended feature data further calculates the degree of change based on the driver's hand torque parameters within the corresponding time step and extracts its motion characteristics based on the steering wheel angle. This supplements the basic features with information on the dynamic evolution of the driver's operating behavior and detailed features of steering wheel control. Through this feature construction method that integrates instantaneous state information with historical trends, the target feature sequence can not only characterize the vehicle and driver state at each independent time point but also depict the fluctuation pattern of hand torque over time and the dynamic pattern of steering wheel rotation. This provides a composite feature input with both spatial and temporal dimensions for the hands-off probability prediction model, thus avoiding the limitations of a single feature and allowing the hands-off probability prediction model to judge the driver's operating state based on multi-dimensional parameters.

[0021] S120. Input the target feature sequence into the release probability prediction model to obtain the initial release probability. The release probability prediction model is obtained by training the network to be trained using samples. The samples include: historical feature sequences and release probability labels. The historical feature sequences include historical basic feature data and historical extended feature data. For example, after constructing the target feature sequence, the target feature sequence is input into a pre-trained release probability prediction model to obtain the corresponding initial release probability. This release probability prediction model is obtained by training the network to be trained with a large amount of historical sample data. Each training sample contains a historical feature sequence and its corresponding real release probability label, so that the release probability prediction model can learn the inherent mapping relationship between the feature sequence and the driver's release state during the training process. Through this data-driven modeling approach, the release probability prediction model can automatically extract deep features related to the release state from the input target feature sequence and output the probability value of the driver being in the release state at the current moment based on the learned mapping rules, thereby transforming multi-dimensional and time-series feature data into a quantifiable probability index.

[0022] S130. The initial hand-drop probability is compared with a preset hand-drop probability threshold, and the driver is determined to be in a hand-drop state based on the comparison result.

[0023] For example, the initial release probability output by the release probability prediction model is compared with a preset release probability threshold. Based on the comparison result of whether the initial release probability exceeds the preset release probability threshold, it is directly determined whether the current driver is in a state of release or not, thus completing the final decision of detection.

[0024] In summary, the driver hands-off detection method proposed in this application obtains a target feature sequence composed of basic feature data and extended feature data, and integrates instantaneous sensor information at a single moment with statistical feature information reflecting the degree of change in hand torque and the characteristics of steering wheel movement, thereby providing a more comprehensive data foundation for subsequent hands-off judgment. Inputting the target feature sequence into the hands-off probability prediction model effectively overcomes the misjudgment problem caused by traditional methods relying solely on threshold judgments at a single moment. Finally, by comparing the initial hands-off probability with a preset hands-off probability threshold, the driver's hands-off state can be clearly determined.

[0025] In some examples, the release probability prediction model includes a fully connected network model and a long short-term memory network model. The step of inputting the target feature sequence into the release probability prediction model to obtain the initial release probability includes: The fully connected layer of the fully connected network model performs a nonlinear transformation on the target feature sequence at each time step and performs dimensionality reduction on the nonlinearly transformed target feature sequence to obtain the first feature sequence. The long short-term memory layer based on the long short-term memory network model performs time-series modeling on the first feature sequence, and then performs regularization on the time-series modeled first feature sequence to obtain the second feature sequence. The second feature sequence is dimensionally compressed based on the fully connected layer of the fully connected network model to obtain the initial release probability.

[0026] For example, the drop probability prediction model adopts an architecture that combines a fully connected network model and a long short-term memory network model. When the target feature sequence is input into the drop probability prediction model, the fully connected layer in the fully connected network model first performs time-step nonlinear transformation processing on the input target feature sequence. Specifically, for the target feature sequence corresponding to each time step, the fully connected layer maps it to a 128-dimensional high-order space through linear transformation. The mapped target feature sequence is then subjected to batch normalization, ReLU-based nonlinear activation and sparsification, and Dropout-based regularization. Building upon this, the fully connected layer further reduces the dimensionality of the regularized target feature sequence through linear transformation, reducing the 128-dimensional feature sequence to 64 dimensions. This removes redundant information and retains core feature representations, ultimately generating a first feature sequence with uniform dimensionality and greater discriminative power at each time step. This first feature sequence preserves the integrity of key information from the original input while optimizing the feature space structure, laying the data foundation for subsequent temporal feature extraction. Subsequently, the first feature sequence undergoes batch normalization, ReLU-based nonlinear activation and sparsification, and Dropout-based regularization to prevent overfitting.

[0027] Further, after obtaining the first feature sequence, the Long Short-Term Memory (LSTM) layer in the Long Short-Term Memory (LSTM) network model performs temporal modeling on the first feature sequence, outputting the 50-dimensional hidden state of the last time step. This LSM layer processes the feature data of each time step in the first feature sequence sequentially along the time dimension. Through its internal forget gate, input gate, and output gate mechanisms, it dynamically selects to retain historical information, absorb current input, and output the updated hidden state. This allows it to capture the changing trend of the driver's hand torque over time, the dynamic law of steering wheel rotation, and the evolution pattern of vehicle motion state within the entire time window, achieving deep modeling of the temporal dependencies in the feature sequence. After completing the temporal modeling, the features obtained from the modeling are regularized based on Dropout to suppress model overfitting and enhance generalization ability, ultimately outputting a second feature sequence that encodes the temporal information within the entire 500ms time window. The feature sequence is then input into the fully connected layer of the fully connected network model for dimensionality compression. A linear transformation maps it to a low-dimensional space, specifically: the 50-dimensional temporal features are compressed to 32 dimensions. The compressed 32-dimensional feature sequence is then subjected to ReLU-based nonlinear activation and sparsification, followed by Dropout-based regularization to introduce nonlinearity and prevent overfitting. Finally, the regularized feature sequence is mapped to 2 dimensions to obtain a two-dimensional probability vector [P(not released), P(released)]. This two-dimensional probability vector is then converted into a probability distribution to output the initial release probability corresponding to the release state, thus transforming the multi-time-step composite features into a quantifiable probability index. The inputs and outputs of each layer in the release probability prediction model are shown in Table 1.

[0028]

[0029] Table 1 By integrating the spatial feature extraction capabilities of FCN (Fully Connected Network) with the temporal modeling capabilities of LSTM (Long Short-Term Memory), the drop-hand probability prediction model can not only uncover the correlations between features within a single time step (spatial features) but also capture the changing trends of features across consecutive time steps (temporal features). Compared to a single network model, this provides a more comprehensive understanding of driving operation features and significantly improves the accuracy of the initial drop-hand probability prediction. The model employs a step-by-step process of dimensionality upscaling, dimensionality reduction, and compression of the target feature sequence, continuously removing redundant information while extracting effective features. This reduces the computational load of the drop-hand probability prediction model and avoids inference delays caused by high-dimensional data. Regularization effectively prevents overfitting during training, allowing the model to maintain stable prediction capabilities even when faced with unseen driving scenario data, thus improving its generalization ability.

[0030] In some examples, after inputting the target feature sequence into the release probability prediction model to obtain the initial release probability, the process includes: The current driving scenario of the vehicle is determined based on lateral acceleration and steering wheel angle. The current driving scenario of the vehicle includes a curve scenario and a straight road scenario. When the current driving scenario is the curve scenario, the product of the preset curve calibration factor and the initial release probability is used as the correction result of the initial release probability. When the current driving scenario is the straight road scenario, the product of the preset straight road calibration factor and the initial release probability is used as the correction result of the initial release probability, and the curve calibration factor is greater than the straight road calibration factor.

[0031] For example, after obtaining the initial release probability output by the release probability prediction model, the current driving scenario is further determined based on the vehicle's real-time dynamic parameters as the basis for subsequent probability correction. Specifically, the lateral acceleration and steering wheel angle of the vehicle at the current moment are obtained. The absolute value of the lateral acceleration is compared with a preset acceleration threshold, and the absolute value of the steering wheel angle is compared with a preset angle threshold. If the absolute value of the lateral acceleration is greater than the acceleration threshold and the absolute value of the steering wheel angle is greater than the angle threshold, the current driving scenario of the vehicle is determined to be a curve scenario. If the absolute value of the lateral acceleration is less than or equal to the acceleration threshold, or the absolute value of the steering wheel angle is less than or equal to the angle threshold, the current driving scenario of the vehicle is determined to be a straight road scenario. The acceleration threshold can be set to 0.3 m / s² for example. 2The angle threshold can be set to 15° for example. The above judgment rules enable accurate identification of driving scenarios and provide scenario basis for subsequent differentiated correction strategies.

[0032] After determining the current driving scenario, the initial hand-off probability is differentially corrected based on the scenario type to adapt to the detection sensitivity requirements of different driving scenarios. Specifically, when the current driving scenario is determined to be a curve scenario, a preset curve calibration factor is obtained, and the product of the initial hand-off probability and the preset curve calibration factor is used as the correction result; when the current driving scenario is determined to be a straight road scenario, a preset straight road calibration factor is obtained, and the product of the initial hand-off probability and the preset straight road calibration factor is used as the correction result. The preset curve calibration factor is greater than the preset straight road calibration factor; for example, the curve calibration factor can be set to 1.1, and the straight road calibration factor can be set to 0.9.

[0033] In curved driving scenarios, the steering wheel is constantly turning, requiring the driver to apply continuous torque to maintain the vehicle's trajectory. If the driver loses control under these conditions, the risk is higher. Therefore, a larger calibration factor is used to amplify the initial probability of loss of control, improving detection sensitivity and reducing false alarms. In straight driving scenarios, the vehicle's direction is relatively stable, and the steering wheel position is essentially fixed. Instantaneous fluctuations in hand torque or sensor noise are more likely to cause misjudgments. Therefore, a smaller calibration factor is used to attenuate the initial probability of loss of control, maintaining the conservatism of the detection decision and reducing false alarms. The correction result obtained after scene adaptive correction will serve as the direct basis for subsequent loss of control determination, thus achieving differentiated detection in different driving scenarios. The probability calibration step is implemented through simple numerical multiplication, with low computational cost and no increase in inference latency of the onboard system, meeting real-time requirements. As an optimization step after model prediction, this step does not require modification of the model itself; it can be adapted to different scenarios simply by adjusting parameters, improving the flexibility and debuggability of the entire detection method.

[0034] In some examples, the hand-off probability threshold includes a straight-line hand-off probability threshold and a curve hand-off probability threshold, and determining whether the driver is in a hand-off state based on the comparison result includes: If the current driving scenario is the curve scenario and the initial probability of losing control is greater than the preset curve loss probability threshold, then the driver is determined to be in a hands-free state. If the current driving scenario is a straight road scenario and the initial hand-off probability is greater than the preset straight road hand-off probability threshold, it is determined that the driver is in a hand-off state, and the curve hand-off probability threshold is less than the straight road hand-off probability threshold.

[0035] For example, after correcting the initial release probability based on the current driving scenario, a differentiated release probability threshold matched to the current driving scenario is used for the final state determination to achieve a scenario-adaptive decision-making mechanism. Specifically, corresponding release probability thresholds are pre-set for different driving scenarios, including a curve release probability threshold for curve scenarios and a straight release probability threshold for straight road scenarios, and the preset curve release probability threshold is smaller than the preset straight road release probability threshold. Based on the initial release probability after scene correction and the current driving scenario being determined, if the current driving scenario is determined to be a curve scenario, the initial release probability is compared with a preset curve release probability threshold. When the initial release probability is greater than the preset curve release probability threshold, the driver is determined to be in a release state; when the initial release probability is less than or equal to the preset curve release probability threshold, the driver is determined to be in a hold state. If the current driving scenario is determined to be a straight road scenario, the initial release probability is compared with a preset straight road release probability threshold. When the initial release probability is greater than the preset straight road release probability threshold, the driver is determined to be in a release state; when the initial release probability is less than or equal to the preset straight road release probability threshold, the driver is determined to be in a hold state. The preset curve release probability threshold is 0.45, and the preset straight road release probability threshold is 0.55.

[0036] Compared to traditional methods using fixed thresholds, this method significantly improves the accuracy of detection results in different scenarios. It features a lower threshold for hand-off probability in cornering, coupled with a larger calibration factor, enabling dual detection in cornering scenarios. This allows for more sensitive identification of hand-off behavior in corners, avoiding missed detections due to the high demands of steering wheel operation in cornering, thus improving cornering safety. Conversely, it has a higher threshold for hand-off probability in straight-line scenarios, coupled with a smaller calibration factor, enabling dual conservative detection in straight-line scenarios. This effectively filters false hand-off signals caused by slight driver corrections or minor sensor vibrations, reducing false alarms and improving the driving experience. The scenario-based thresholds are preset parameters that can be flexibly adjusted according to different vehicle models and driving conditions, allowing the detection method to adapt to various automotive application scenarios and improving its suitability for industrial applications. The threshold comparison logic is simple, and combined with probability calibration, it eliminates the need for complex calculations, ensuring real-time detection.

[0037] In some examples, comparing the initial sell probability with a preset sell probability threshold includes: Obtain multiple initial sell probabilities within a preset time period; The average of multiple initial sell probabilities is compared with a preset sell probability threshold.

[0038] For example, after obtaining the initial release probability output by the release probability prediction model, a time sliding window mechanism is further introduced to collect the initial release probabilities at multiple consecutive moments, i.e., to obtain multiple initial release probabilities corresponding to a preset time period. The preset time period can be set to include 10 consecutive time steps. Within this time period, the initial release probability calculated by the release probability prediction model at each time step is collected sequentially, thus forming a probability sequence composed of multiple instantaneous probabilities. By obtaining this probability sequence, the judgment of the driver's release state can be extended from isolated observation at a single moment to dynamic observation of changes over a continuous period of time, providing a data foundation for subsequent decision-making based on statistical characteristics and avoiding the instantaneous sensor noise or random fluctuation interference that may be introduced by relying solely on the probability at a single moment.

[0039] After obtaining multiple initial release probabilities within a preset time period, the average of these initial release probabilities is calculated and used as a comprehensive measure of the driver's tendency to release hands during that time period. Subsequently, the calculated average is compared with a preset release probability threshold: if the average is greater than the preset release probability threshold, the driver is determined to be in a hands-free state during that time period; if the average is less than or equal to the preset release probability threshold, the driver is determined to be in a hands-free state. By using an average for comparison, the probability sequence within the window is essentially smoothed, effectively reducing the impact of single-point abnormal probability values ​​caused by momentary sensor jitter, brief road impacts, or accidental driver actions on the final decision. This makes the determination of the hands-free state closer to the driver's actual operating intention, improving the accuracy of the detection logic.

[0040] Specifically, the following uses the scenario of driving on an elevated ramp as an example to illustrate the complete process of final state determination.

[0041] Scene setting: Vehicle speed: [40, 60] km / h, lateral acceleration: [-0.8, -1.6] m / s² 2， Steering wheel angle: [-10, -50]°, driver assistance system: activated. Driver state evolution and detection results are shown in Table 2.

[0042]

[0043] Table 2 Judgment Logic Explanation: t=0s to t=2s: Hand torque is within the normal range, the model outputs a high probability of not releasing the hand, and the temporal consistency verification passes, indicating a non-release state. t=4s: Hand torque decreases to 0.6 N·m, the model output probability shows a tendency to release the hand (0.65), but the temporal consistency verification fails (probability <0.45 in some frames and >0.45 in others within a 10-frame sliding window), the system remains in a non-release state, only triggering an internal warning. t=6s: Hand torque further decreases to 0.2 N·m, the model outputs a high probability of releasing the hand (0.88), and the temporal consistency verification passes (probability >0.5 for 10 consecutive frames, window mean 0.72 > curve release probability threshold 0.45), indicating a release state, triggering a warning. t=10s: Hand torque remains close to zero, the detection result continues to indicate a release state, triggering a warning.

[0044] The above examples demonstrate the core technical features: through a temporal consistency verification mechanism, the same trend is required for multiple consecutive frames (10 frames) before state switching is allowed, effectively avoiding misjudgments caused by instantaneous noise or sensor jitter; Kalman filtering is used to achieve a smooth transition of state switching, improving the user experience; and scene-adaptive thresholds (0.45 for the probability of dropping hands on a curve and 0.55 for the probability of dropping hands on a straight road) are used to achieve differentiated detection strategies for different scenarios.

[0045] In some examples, obtaining the target feature sequence of the vehicle includes: The basic feature data and the extended feature data are concatenated to obtain the feature vector; The target feature sequence is obtained based on the feature vectors from multiple consecutive time steps.

[0046] For example, when acquiring the target feature sequence of a vehicle, the basic feature data and extended feature data corresponding to each time step need to be concatenated to form a complete feature vector for that time step. Specifically, the basic feature data includes vehicle motion parameters, steering wheel angle, and driver hand torque collected within the corresponding time step. These data directly reflect the vehicle's operating state at the current moment and the driver's basic steering wheel operation information. The vehicle motion parameters may further include vehicle speed, lateral acceleration, longitudinal acceleration, yaw rate, and pitch rate. The extended feature data calculates the degree of change of the driver's hand torque within the corresponding time step and extracts its motion characteristics based on the steering wheel angle. The degree of change of the hand torque may include multiple time-domain statistical features calculated based on the hand torque within a preset time window, such as the hand torque mean, hand torque variance, hand torque skewness, hand torque kurtosis, hand torque standard deviation, and hand torque range. The motion characteristics of the steering wheel angle may include the steering wheel angle change rate and steering wheel angle variance. By concatenating the aforementioned basic feature data with the extended feature data end to end, a 15-dimensional feature vector is generated. This feature vector contains both the instantaneous state information of the current moment and the statistical distribution of hand torque in the time dimension as well as the dynamic change characteristics of the steering wheel angle, thus providing a composite feature representation with both spatial and temporal dimension information for subsequent hands-free state recognition.

[0047] Based on this, a target feature sequence is constructed using feature vectors from multiple consecutive time steps. Specifically, basic feature data of the vehicle at each time step is continuously collected at a preset sampling frequency (e.g., 20Hz), and extended feature data for each time step is calculated simultaneously. Then, a 15-dimensional feature vector for each time step is generated through the aforementioned concatenation method. Subsequently, starting from the current moment, feature vectors from 50 consecutive time steps are extracted and arranged in chronological order to form a 15×50 target feature sequence. Each column vector of this target feature sequence represents a 15-dimensional composite feature at a specific time step, and the entire sequence fully records the evolution of vehicle motion state, driver hand torque, and steering wheel angle over time within a preset time window (e.g., 500ms). This sequence construction method enables the subsequent hands-off probability prediction model to simultaneously acquire spatial feature information at each time point and the dynamic evolution of features over time, providing a data foundation for in-depth analysis of the intrinsic relationship between driver operation behavior and hands-off state.

[0048] The standardized process of single-step feature concatenation and multi-step sequence construction ensures the uniformity of the target feature sequence format, enabling the hands-off probability prediction model to uniformly process feature data collected in different driving scenarios and at different times, thus improving the standardization of the input to the hands-off probability prediction model. Concatenating basic and extended feature data into a single feature vector makes the feature information at each time step more concentrated, facilitating the extraction of spatial features within a single time step by the hands-off probability prediction model. Arranging the feature vectors of multiple consecutive time steps in chronological order constructs a target feature sequence with a time dimension, perfectly adapting to the temporal modeling requirements of LSTM networks, allowing the hands-off probability prediction model to effectively capture the feature change patterns during continuous driving. This construction process is a fixed, standardized operation that can be automatically implemented by the algorithm module of the vehicle system without manual intervention, ensuring the efficiency and accuracy of target feature sequence generation. The concatenation and arrangement operations are basic data processing operations with low computational load, which can be completed quickly within a 50ms sampling period, meeting the real-time requirements of the vehicle system.

[0049] In some examples, the basic feature data includes vehicle speed, lateral acceleration, longitudinal acceleration, yaw rate, pitch rate, steering wheel angle, and driver hand torque, while the extended feature data includes the mean hand torque, hand torque variance, hand torque skewness, hand torque kurtosis, hand torque standard deviation, hand torque range, steering wheel angle change rate, and steering wheel angle variance.

[0050] For example, the basic feature data includes real-time vehicle status parameters and driver operation parameters acquired directly by onboard sensors or via the CAN (Controller Area Network) bus within the corresponding time step. Specifically, these include vehicle speed, lateral acceleration, longitudinal acceleration, yaw rate, pitch rate, steering wheel angle, and driver hand torque. Vehicle speed reflects the overall speed of the vehicle; lateral and longitudinal acceleration characterize the dynamic forces acting on the vehicle in the lateral and forward directions, respectively; yaw rate describes the speed of rotation of the vehicle around its vertical axis; and pitch rate reflects the pitch motion trend of the vehicle around its lateral axis. These five parameters together constitute the set of vehicle motion parameters, used to characterize the vehicle's driving state and dynamic response at the current moment. The steering wheel angle directly records the angle of rotation of the steering wheel by the driver, reflecting their active control intention over the vehicle's trajectory. The driver hand torque is acquired in real time by a steering wheel torque sensor, directly quantifying the magnitude of the contact force between the driver's hand and the steering wheel. The basic feature data is shown in Table 3.

[0051]

[0052] Table 3 The extended feature data is a set of derived features calculated based on the driver's hand torque and steering wheel angle within a preset time period before the corresponding time step. It is used to characterize the dynamic evolution and detailed features of the driver's operating behavior. Specifically, based on the driver's hand torque values ​​at multiple consecutive time steps within a preset time window, statistical calculations are performed to obtain the mean hand torque, variance hand torque, skewness hand torque, kurtosis hand torque, standard deviation hand torque, and range hand torque. The mean hand torque reflects the average torque level applied by the driver's hands within the time window. The variance and standard deviation of the hand torque together characterize the fluctuation and dispersion of the hand torque signal. The skewness of the hand torque describes the symmetry of the hand torque distribution relative to the mean to determine the directionality of torque changes. The kurtosis of the hand torque characterizes the sharpness of the hand torque distribution to identify the presence of instantaneous impact torque. The range of the hand torque reflects the maximum range of hand torque variation within the window. Simultaneously, based on the steering wheel angle values ​​at multiple consecutive time steps within the preset time window, the rate of change of the steering wheel angle and the variance of the steering wheel angle are calculated. The rate of change of the steering wheel angle reflects the speed characteristics of the driver's steering wheel rotation, while the variance of the steering wheel angle characterizes the stability and fluctuation of the steering wheel rotation. The aforementioned eight-dimensional extended feature data and seven-dimensional basic feature data together constitute a 15-dimensional complete feature vector for each time step, so that the target feature sequence not only contains the instantaneous state information at the current moment, but also integrates the statistical distribution pattern and dynamic change pattern of the driver's operation behavior in the time dimension.

[0053] The basic feature data selects seven core parameters related to vehicle motion and driver operation, covering three dimensions: vehicle driving state, vehicle posture, and driver steering wheel operation, ensuring the comprehensiveness of the basic feature data. The extended feature data undergoes in-depth processing of the core judgment criteria for driver hands-off detection (hand torque, steering wheel angle), selecting statistical features that accurately reflect the changing patterns of these parameters. Single numerical parameters are transformed into regular feature parameters, allowing the hands-off probability prediction model to determine whether the driver has taken their hands off the wheel based on the trend of parameter changes, rather than on a single numerical value, significantly improving the effectiveness of the feature data. The combination of the seven basic feature data and the eight extended feature data, totaling 15 dimensions, ensures feature comprehensiveness while controlling the number of feature dimensions. This avoids the problems of excessive computation and overfitting in the hands-off probability prediction model caused by high-dimensional features, balancing feature effectiveness and model practicality.

[0054] like Figure 2 As shown, this application proposes a driver hands-off detection system, which includes: a feature acquisition module 21, a probability determination module 22, and a probability comparison module 23; The feature acquisition module 21 is configured to: acquire a target feature sequence of the vehicle, the target feature sequence including multiple feature datasets of time steps with a time sequence, each feature dataset of time step including basic feature data and extended feature data, the basic feature data including the vehicle's motion parameters, steering wheel angle and driver's hand torque in the corresponding time step, the extended feature data including the data on the degree of change of hand torque parameters in the corresponding time step, and the data on the motion characteristics of the steering wheel angle in the time step; The probability determination module 22 is configured to: input the target feature sequence into the release probability prediction model to obtain the initial release probability. The release probability prediction model is obtained by training the network to be trained using samples. The samples include: historical feature sequences and release probability labels. The historical feature sequences include historical basic feature data and historical extended feature data. The probability comparison module 23 is configured to compare the initial release probability with a preset release probability threshold, and determine whether the driver is in a release state based on the comparison result.

[0055] The effects of applying the aforementioned method in the above system can be found in the description of the aforementioned method embodiments, and will not be repeated here.

[0056] like Figure 3 As shown, this application embodiment also provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor. When the processor 320 executes the computer program 311, it implements the steps of any of the above-described methods for detecting driver hands-off operation.

[0057] Since the electronic device described in this embodiment is the device used to implement a driver hands-off detection device in the embodiments of this application, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this embodiment based on the method described in the embodiments of this application. Therefore, how the electronic device implements the method in the embodiments of this application will not be described in detail here. Any device used by those skilled in the art to implement the method in the embodiments of this application is within the scope of protection of this application.

[0058] In practical implementation, when the computer program 311 is executed by the processor, it can achieve the following: Figure 1 Any of the corresponding implementation methods in the embodiments.

[0059] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0060] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-readable program code.

[0061] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0062] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0063] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0064] This application also provides a computer program product, which includes computer software instructions that, when executed on a processing device, cause the processing device to execute the LDPC decoding method of a solid-state drive controller.

[0065] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0066] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0067] In the several embodiments provided in this application, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0068] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0069] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0070] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0071] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

[0072] Although preferred embodiments have been described in this specification, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this specification.

[0073] Obviously, those skilled in the art can make various modifications and variations to this specification without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims and their equivalents, this specification is also intended to include such modifications and variations.

Claims

1. A method for detecting driver hands-off driving, characterized in that, The method includes: Obtain the target feature sequence of the vehicle. The target feature sequence includes multiple feature datasets with time steps in sequence. Each feature dataset of time step includes basic feature data and extended feature data. The basic feature data includes the vehicle's motion parameters, steering wheel angle, and driver's hand torque within the corresponding time step. The extended feature data includes data on the degree of change of the hand torque parameter within the corresponding time step, as well as data on the motion characteristics of the steering wheel angle within the time step. The target feature sequence is input into the release probability prediction model to obtain the initial release probability. The release probability prediction model is trained using samples to train the network to be trained. The samples include: historical feature sequences and release probability labels. The historical feature sequences include historical basic feature data and historical extended feature data. The initial probability of disengagement is compared with a preset probability threshold for disengagement, and the driver is determined to be in a disengagement state based on the comparison result.

2. The method for detecting driver hands-off driving according to claim 1, characterized in that, The release probability prediction model includes a fully connected network model and a long short-term memory network model. The step of inputting the target feature sequence into the release probability prediction model to obtain the initial release probability includes: The fully connected layer of the fully connected network model performs a nonlinear transformation on the target feature sequence at each time step and performs dimensionality reduction on the nonlinearly transformed target feature sequence to obtain the first feature sequence. The long short-term memory layer based on the long short-term memory network model performs time-series modeling on the first feature sequence, and then performs regularization on the time-series modeled first feature sequence to obtain the second feature sequence. The second feature sequence is dimensionally compressed based on the fully connected layer of the fully connected network model to obtain the initial release probability.

3. The method for detecting driver hands-off driving according to claim 1, characterized in that, After inputting the target feature sequence into the release probability prediction model to obtain the initial release probability, the process includes: The current driving scenario of the vehicle is determined based on lateral acceleration and steering wheel angle. The current driving scenario of the vehicle includes a curve scenario and a straight road scenario. When the current driving scenario is the curve scenario, the product of the preset curve calibration factor and the initial release probability is used as the correction result of the initial release probability. When the current driving scenario is the straight road scenario, the product of the preset straight road calibration factor and the initial release probability is used as the correction result of the initial release probability, and the curve calibration factor is greater than the straight road calibration factor.

4. The driver hands-off detection method according to claim 3, characterized in that, The hand-drop probability threshold includes a straight-line hand-drop probability threshold and a curve hand-drop probability threshold. Determining whether the driver is in a hand-drop state based on the comparison result includes: If the current driving scenario is the curve scenario and the initial probability of losing control is greater than the preset curve loss probability threshold, then the driver is determined to be in a hands-free state. If the current driving scenario is a straight road scenario and the initial hand-off probability is greater than the preset straight road hand-off probability threshold, it is determined that the driver is in a hand-off state, and the curve hand-off probability threshold is less than the straight road hand-off probability threshold.

5. The method for detecting driver hands-off operation according to claim 1, characterized in that, The step of comparing the initial sell probability with a preset sell probability threshold includes: Obtain multiple initial sell probabilities within a preset time period; The average of multiple initial sell probabilities is compared with a preset sell probability threshold.

6. The method for detecting driver hands-off driving according to claim 1, characterized in that, The acquisition of the target feature sequence of the vehicle includes: The basic feature data and the extended feature data are concatenated to obtain the feature vector; The target feature sequence is obtained based on the feature vectors from multiple consecutive time steps.

7. The method for detecting driver hands-off operation according to claim 1, characterized in that, The basic feature data includes vehicle speed, lateral acceleration, longitudinal acceleration, yaw rate, pitch rate, steering wheel angle, and driver's hand torque. The extended feature data includes the mean hand torque, variance of hand torque, skewness of hand torque, kurtosis of hand torque, standard deviation of hand torque, range of hand torque, rate of change of steering wheel angle, and variance of steering wheel angle.

8. A driver hands-off detection system, characterized in that, The system includes: a feature acquisition module, a probability determination module, and a probability comparison module; The feature acquisition module is configured to: acquire a target feature sequence of the vehicle, the target feature sequence including multiple feature datasets of time steps with a time sequence, each feature dataset of time step including basic feature data and extended feature data, the basic feature data including the vehicle's motion parameters, steering wheel angle and driver's hand torque in the corresponding time step, the extended feature data including data on the degree of change of hand torque parameters in the corresponding time step, and data on the motion characteristics of the steering wheel angle in the time step; The probability determination module is configured to: input the target feature sequence into the release probability prediction model to obtain the initial release probability. The release probability prediction model is obtained by training the network to be trained using samples. The samples include: historical feature sequences and release probability labels. The historical feature sequences include historical basic feature data and historical extended feature data. The probability comparison module is configured to compare the initial release probability with a preset release probability threshold, and determine whether the driver is in a release state based on the comparison result.

9. An electronic device, comprising: The memory and processor are characterized in that the processor, when executing a computer program stored in the memory, implements the steps of a driver hands-off detection method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of a driver hands-off detection method as described in any one of claims 1-7.

Images