Steering wheel out-of-hand detection method, device and equipment and medium

Abstract

The embodiment of the invention provides a steering wheel out-of-hand detection method and device, equipment and a medium, and relates to the technical field of vehicles. The method comprises the steps that interaction state data from a steering wheel system are obtained, and the interaction state data at least comprise contact sensing data, torque sensing data and visual sensing data; based on the interaction state data, performing single-mode hand-free state judgment on each kind of interaction state data; performing fusion decision on the judgment result of each single-mode hand-free state to obtain a corresponding hand-free detection result; and in response to the hand release detection result, indicating a hand release state, and executing a preset grading response process. The method provided by the invention effectively solves the problem that the driving safety is affected by misjudgment easily caused by hand release detection in the related technology.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a method, device, equipment and medium for detecting steering wheel hands-off. Background Technology

[0002] With the widespread application of advanced driver assistance systems (ADAS) such as Lane Control (LCC) and Navigation Assist (NOA), in-vehicle systems need to continuously monitor whether the driver has their hands on the steering wheel to ensure that they can take over the vehicle in a timely manner when necessary. Hands-off detection is a key technology for ensuring the safe operation of driver assistance systems. The accuracy of its detection results directly affects whether the system can correctly trigger protective measures such as warnings, downgrades, or safe stopping, and is the foundation for achieving a safe closed loop for human-machine co-driving.

[0003] In related technologies, the solution for hands-off detection is usually a single sensor solution (such as capacitive sensor detection or visual detection). Such solutions are prone to failure in complex scenarios in real-world applications: for example, a heavy object hanging on the steering wheel may deceive both the capacitive and torque sensors, causing the system to misjudge the hands-off state, resulting in insufficient accuracy and robustness of hands-off detection, which directly affects the reliability of driver assistance functions and driving safety. Summary of the Invention

[0004] This application provides a method, apparatus, equipment, and medium for detecting steering wheel hands-off detection, in order to solve the problem that hands-off detection in related technologies is prone to misjudgment and affects driving safety.

[0005] Firstly, this application provides a method for detecting steering wheel removal from hands, the method comprising the following steps:

[0006] Acquire interaction status data from the steering wheel system, wherein the interaction status data includes at least contact sensor data, torque sensor data and visual sensor data;

[0007] Based on the interaction state data, a single-modal release state judgment is performed for each type of interaction state data;

[0008] The results of each single-modal release state judgment are fused to make a decision, and the corresponding release detection result is obtained.

[0009] In response to the release detection result indicating the release status, a preset hierarchical response procedure is executed.

[0010] In one embodiment of this disclosure, the results of each single-modal release state judgment are fused to obtain the corresponding release detection result, including: determining the existence of a torque value that meets a preset fixed feature condition based on the results of the single-modal release state judgment corresponding to the torque sensing data, and triggering anti-cheating verification; in the anti-cheating verification, the release detection result is obtained based on the results of the single-modal release state judgment corresponding to the visual sensing data.

[0011] In one embodiment of this disclosure, the results of each single-modal release state judgment are fused to obtain the corresponding release detection result, including: determining the weight coefficients corresponding to the single-modal release state judgment results of contact sensing, torque sensing and vision sensing respectively; performing weighted calculation on each single-modal release state judgment result based on the weight coefficients to obtain a fused judgment value; comparing the fused judgment value with a preset decision threshold, and obtaining the release detection result based on the comparison result.

[0012] In one embodiment of this disclosure, the weighting coefficients are dynamically calibrated based on the confidence level of each sensor data or the characteristics of the current driving scenario, and the sum of each weighting coefficient is 1.

[0013] In one embodiment of this disclosure, in response to the hands-free detection result indicating a hands-free state, a preset hierarchical response process is executed, including: in response to the hands-free detection result indicating a hands-free state, executing a hands-free alarm process including at least two levels of prompts; if no effective steering wheel interaction is detected after the hands-free alarm process is completed, vehicle safety parking control is triggered.

[0014] In one embodiment of this disclosure, based on interactive state data, a single-modal release state judgment is performed on each type of interactive state data, including: performing de-jitter processing on contact sensing data, and obtaining a first judgment result based on the processed contact sensing data and a preset first threshold; performing filtering processing on torque sensing data, and obtaining a second judgment result based on the filtered torque value and its statistical characteristics in a continuous time period and a preset second threshold; and performing hand region recognition processing on visual sensing data, and obtaining a third judgment result based on the processed visual sensing data and a preset third threshold.

[0015] In one embodiment of this disclosure, the preset second threshold includes a preset torque threshold and a preset variance threshold; based on the filtered torque value and its statistical characteristics in a continuous time period and the preset second threshold, a second judgment result is obtained, including: if the filtered torque value is less than the preset torque threshold and its variance in a continuous time period is less than the preset variance threshold, then the second judgment result is determined to be that there is a release state.

[0016] Secondly, embodiments of this disclosure provide a steering wheel hands-off detection device, which includes:

[0017] The acquisition module is used to acquire interaction status data from the steering wheel system, wherein the interaction status data includes at least contact sensor data, torque sensor data and visual sensor data.

[0018] The analysis module is used to perform single-modal release state judgment for each type of interaction state data based on the interaction state data.

[0019] The processing module is used to fuse the results of each single-modal release state judgment to obtain the corresponding release detection result;

[0020] The execution module is used to execute a preset hierarchical response process in response to the release detection result indicating the release status.

[0021] Optionally, the processing module is specifically used to determine, based on the result of the single-modal release state judgment corresponding to the torque sensing data, the existence of a torque value that satisfies the preset fixed characteristic conditions, and trigger anti-cheating verification; in the anti-cheating verification, based on the result of the single-modal release state judgment corresponding to the visual sensing data, the release detection result is obtained.

[0022] Optionally, the processing module is specifically used to: determine the weight coefficients corresponding to the results of the single-modal release state judgment of contact sensing, torque sensing and vision sensing; perform weighted calculation on the release state judgment results of each single-modal sensor based on the weight coefficients to obtain a fused judgment value; compare the fused judgment value with a preset decision threshold, and obtain the release detection result based on the comparison result.

[0023] Optionally, the processing module specifically includes dynamically calibrating the weight coefficients based on the confidence level of each sensor data or the characteristics of the current driving scenario, and the sum of each weight coefficient is 1.

[0024] Optionally, the execution module is specifically used to execute a hands-free alarm process that includes at least two levels of prompts in response to the hands-free detection result indicating a hands-free state; if no effective steering wheel interaction is detected after the hands-free alarm process is completed, the vehicle safety parking control is triggered.

[0025] Optionally, the analysis module is specifically used to: perform jitter removal processing on the contact sensing data, and obtain a first judgment result based on the processed contact sensing data and a preset first threshold; perform filtering processing on the torque sensing data, and obtain a second judgment result based on the filtered torque value and its statistical characteristics in a continuous time period and a preset second threshold; and perform hand region recognition processing on the visual sensing data, and obtain a third judgment result based on the processed visual sensing data and a preset third threshold.

[0026] Optionally, the analysis module is specifically used to determine that the second judgment result is that there is a release state when the preset second threshold includes a preset torque threshold and a preset variance threshold.

[0027] Thirdly, embodiments of this application provide a control device, including: a memory and a processor;

[0028] The memory stores instructions that the computer executes;

[0029] The processor executes computer execution instructions stored in memory, causing the processor to perform a steering wheel hands-off detection method as described in the first aspect of this disclosure.

[0030] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the steering wheel hands-off detection method as described in the first aspect of this disclosure.

[0031] Fifthly, embodiments of this disclosure also provide a computer program product comprising computer execution instructions, which, when executed by a processor, are used to implement the steering wheel hands-off detection method as described in the first aspect of this disclosure.

[0032] The steering wheel hands-off detection method, apparatus, device, and medium provided in this disclosure acquire multimodal interactive state data, including contact, torque, and vision data. First, each type of data undergoes independent single-modal hands-off state judgment. Then, multiple judgment results are integrated and fused for decision-making. Finally, a tiered response process is executed based on the decision results. This effectively solves the problem of frequent false detections and missed detections caused by the limitations of sensor principles and insufficient scene adaptability in single or simple dual-modal detection schemes. Furthermore, the pre-emptive single-modal judgment ensures the quality of basic data, and the multimodal fusion decision integrates multi-dimensional complementary information, effectively overcoming the perception failure of traditional schemes in low-torque, visually obstructed, or specific deceptive scenarios. Therefore, in complex and ever-changing real-world driving environments, it significantly improves the overall accuracy, robustness, and real-time performance of hands-off state determination, providing crucial technical support for the safe and reliable operation of advanced driver assistance systems. Attached Figure Description

[0033] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0034] Figure 1 This diagram illustrates an application scenario of the steering wheel hands-off detection method, apparatus, equipment, and medium provided in this disclosure.

[0035] Figure 2 This is a flowchart of a steering wheel hands-off detection method provided in one embodiment of the present disclosure;

[0036] Figure 3 A flowchart illustrating a steering wheel hands-off detection method provided in yet another embodiment of this disclosure;

[0037] Figure 4 A schematic diagram of the steering wheel hands-off detection device provided in yet another embodiment of this disclosure;

[0038] Figure 5 This is a schematic diagram of the structure of a control device provided in yet another embodiment of this disclosure.

[0039] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0040] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0041] Hands-off detection is a core technology for ensuring the safe operation of advanced driver assistance systems (ADAS). It must be able to accurately and in real time identify whether the driver is effectively controlling the steering wheel in the complex dynamic environment of a vehicle, so as to provide timely warnings or take over in emergencies. The essential difficulty of this task lies in the fact that there are many interference and deceptive factors in real-world application scenarios. For example, hanging objects on the steering wheel (such as the driver placing a mobile phone, tablet, or water bottle on the steering wheel) can simultaneously simulate hand contact and the application of torque. On the other hand, driving at a constant speed or when the driver's hand is loosely gripping the steering wheel results in a weak effective control signal. Relying solely on any single type of sensor cannot make a reliable judgment under all operating conditions.

[0042] In related technologies, hand-drop detection mainly relies on a single sensor or a simple combination of two sensors. However, these solutions are limited by the inherent physical limitations of the sensors and the single assumption of the application scenario, making it difficult to fundamentally address the aforementioned complex challenges. For example, while a dual-modal solution using capacitance and torque sensors improves reliability in some scenarios, it cannot handle the deceptive interference caused by fixed torque. Furthermore, adding a visual modality, if only involving simple result superposition or polling, cannot provide effective supplementation in situations of drastic changes in lighting or partial occlusion. This results in the inability to solve the problem of limited practicality in hand-drop detection.

[0043] The steering wheel hands-off detection method, apparatus, equipment, and medium provided in this application acquire multimodal raw data and perform targeted single-modal state judgments for each, providing high-quality, interference-free primary features for subsequent fusion. Thus, by introducing a dedicated fusion decision-making stage—which is not a simple voting or polling process but acts as an intelligent center—it can make comprehensive decisions based on the inherent logic of multiple criteria, thereby achieving precise adaptation to various complex scenarios, especially those where existing technologies are prone to failure. By linking the decision results with proactive safety processes, a complete safety closed loop of detection, judgment, and intervention is formed, effectively overcoming the reliability challenge of hands-off detection in real-world complex environments.

[0044] Figure 1 This is a schematic diagram illustrating the application scenarios of the steering wheel hands-off detection method, apparatus, equipment, and medium provided in this application, such as... Figure 1 As shown, during vehicle operation, the vehicle control system 100 automatically collects multimodal driver interaction data 110 and, upon detecting when the driver takes their hands off the wheel, controls the vehicle's driving status according to the corresponding control strategy 120.

[0045] It should be noted that, Figure 1 The scenario shown includes an in-vehicle control system, multimodal driver interaction data, and control strategies. Only one or a specific number of these are used as examples for illustration, but this disclosure is not limited to this. That is to say, the number of in-vehicle control systems, multimodal driver interaction data, and control strategies can be arbitrary.

[0046] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0047] Figure 2 This is a flowchart illustrating the steering wheel hands-off detection method provided in the embodiments of this application. The following is a summary of the process. Figure 2 The main procedures for the steering wheel hands-off detection method are explained below:

[0048] S201. Obtain interaction status data from the steering wheel system.

[0049] The interaction status data includes at least contact sensing data, torque sensing data, and visual sensing data.

[0050] Specifically, in this embodiment of the steering wheel hands-off detection method, the executing entity is an onboard domain controller or dedicated driver assistance computing unit capable of real-time data communication with the vehicle's steering wheel capacitive sensing system (HOD), electric power steering system (EPS), and driver monitoring system (DMS) to obtain the required interactive state data and perform processing and decision-making. For ease of explanation, it will be referred to as the system below.

[0051] During the activation of combined driver assistance functions, such as lane centering cruise or navigation-assisted driving mode, the system needs to continuously monitor the driver's status to ensure that the driver can take over at any time.

[0052] Therefore, the system needs to simultaneously collect raw data representing the interaction state between the driver and the steering wheel from multiple vehicle sensors with different physical principles, in order to form the perception basis for all subsequent analysis and judgment.

[0053] The interaction status data here is not a single signal, but a collection of at least three types of data, each reflecting the interaction from a unique dimension.

[0054] Contact sensing data typically comes from a capacitive sensor array integrated within the steering wheel rim. The principle is to detect local capacitance changes caused by human hand contact, thereby generating a discrete or continuous state signal characterizing contact or non-contact.

[0055] The torque sensing data comes from the torque sensor in the electric power steering system. It directly measures the steering torque applied by the driver to the steering wheel. This signal is a continuous quantity, and its magnitude and trend can indirectly reflect the driver's control intention and degree of intervention.

[0056] The visual sensing data comes from cameras installed in the cockpit that cover the driver and steering wheel area. It provides a video stream containing rich spatial and temporal information, and can directly observe the driver's hand position, posture and relative relationship with the steering wheel.

[0057] In practical implementation, the system can subscribe to and receive this data from the corresponding electronic control unit at fixed time intervals (e.g., 10 milliseconds or 100 milliseconds) through bus protocols such as the vehicle controller area network.

[0058] To ensure data is time-aligned for subsequent fusion, the system adds a precise timestamp to received data packets, or requests timestamps to be sent from the data source.

[0059] In some embodiments, in order to achieve more robust perception, the system may perform preliminary validity checks after acquiring raw data, such as checking whether the signal is within the physically possible range, whether it is continuously zero or exceeds the limit, and whether the corresponding sensor has reported a fault code, thereby providing quality indicators for subsequent processing.

[0060] S202. Based on the interaction state data, perform single-modal release state judgment for each type of interaction state data.

[0061] Specifically, the system needs to transform the acquired raw, noisy sensor data into intermediate state quantities with clear physical meaning and judgment direction.

[0062] At this point, the system needs to process and analyze each type of data independently in parallel, and make a preliminary judgment on whether a slip has occurred from the perspective of that modality based on the data characteristics and physical meaning of each data type.

[0063] This parallel processing structure ensures the independence of each modality judgment, avoids potential cross-contamination of errors in early fusion, and provides multi-dimensional and credible preliminary evidence for subsequent fusion decisions.

[0064] For contact sensor data, which may fluctuate due to electrical noise or transient interference, the system performs stability processing.

[0065] One implementation approach is to apply a time-based hysteresis filter or anti-jitter algorithm. For example, the system does not immediately change its state due to a single "contact-to-non-contact" transition. Instead, it requires the "non-contact" signal to be maintained for more than a preset time window (e.g., 200 milliseconds) before determining that the mode tends to be in the off-hand state. This effectively filters out instantaneous false triggers.

[0066] The analysis of torque sensing data is more complex, with the core issue being the distinction between torque actively applied by the driver and torque caused by environmental or vehicle dynamics. The system first uses filtering algorithms (such as low-pass filtering or moving average) to smooth the data and remove high-frequency noise.

[0067] Therefore, the system not only focuses on the instantaneous absolute value of the torque, but also on its statistical characteristics over a period of time, such as variance or standard deviation. A torque that is persistent but fluctuates very little is likely not from the driver's active control.

[0068] In practice, the system can calculate the variance of the filtered torque value in the most recent second. If the variance is lower than a threshold that indicates "no active operation", it can indicate the possibility of release even if the absolute value of the torque is not zero.

[0069] For visual sensing data, the system needs to extract semantic information from the video stream. This is typically achieved through a lightweight computer vision model or algorithm, such as a deep learning-based keypoint detection network or a traditional image processing workflow. The goal of this processing is to identify the bounding boxes or keypoints (such as fingertips and wrists) of the driver's hands in the video frames and determine the positional relationship between these points and the predefined steering wheel grip area in the image coordinate system.

[0070] If multiple consecutive frames detect that the hand keypoints are all outside the steering wheel area, a visual modality for determining whether the hand is off-hand is generated. This process may involve robust handling of changes in lighting and partial occlusion.

[0071] S203. The results of each single-modal release state judgment are fused to make a decision, and the corresponding release detection result is obtained.

[0072] Specifically, the system needs to construct a decision-making mechanism capable of evaluating the confidence level of each modality's judgment results and intelligently integrating them based on the scenario. The output of this fusion decision is a comprehensive and more robust final detection result.

[0073] In terms of implementation, the system takes the three single-modal judgment results (for example, each result can be a binary state "released / not released", or a probability value between 0 and 1) as input.

[0074] The decision-making mechanism can be designed as a rule-based expert system or a probability-based inference model.

[0075] For example, in a basic implementation, the system can assign a static or dynamic confidence weight to each modality, which can be adjusted based on sensor self-test status, historical accuracy, or the current driving scenario (such as straight lines, curves, or bumpy roads). The system can then perform a comprehensive evaluation based on the weighted results.

[0076] In some embodiments, a state machine or context-aware model may also be introduced into the system. For example, when the system suspects the existence of a water bottle-like deception through torque characteristic analysis (even if the torque value is not zero but lacks fluctuation), it can temporarily reduce the weight of the torque mode or trigger a special decision subprocess, and instead rely more on the output of the visual mode to make the final decision.

[0077] This enables the fusion decision-making process to identify which modal(s) provide the most reliable information at the current moment and in the current scenario, and to make a globally optimal judgment accordingly, thereby effectively dealing with complex situations where any single sensor fails or is deceived.

[0078] S204. In response to the release detection result indicating the release status, execute the preset graded response procedure.

[0079] Specifically, the system will follow a preset, progressively escalating intervention strategy, also known as a graded response process, to give the driver ample opportunity to react and take over, while ensuring safety, and to avoid abrupt exit of functions due to false alarms, which would affect the driving experience.

[0080] In practice, the tiered response process is typically designed with multiple progressively advancing stages. When the fusion decision output initially determines "hands off," the system does not immediately take strong intervention measures. Instead, it initiates the first-level response, such as displaying a soft visual cue icon on the dashboard or head-up display, or emitting a brief alert sound. The purpose of this stage is to non-intrusively alert the driver.

[0081] If the hands are out of reach for more than a short time threshold (e.g., 3 seconds), the system will escalate to a second-level response, strengthening the alerts. For example, the alert sounds become more frequent and urgent, and the color of the visual warning may change from yellow to red.

[0082] There may be even higher levels of response, such as providing tactile warnings through steering wheel vibration or seatbelt pretensioning.

[0083] Only if, after all levels of alerts, the system fails to detect a valid return of driver interaction (such as a torque input that meets the conditions or a visually confirmed return of the hand) within the preset maximum allowable hands-free time (e.g., a total of 15 seconds), will the system trigger the final safe stop control.

[0084] This control is not an emergency brake, but rather, taking into account the safety of the surrounding environment, it automatically activates the hazard warning lights and controls the vehicle to slow down smoothly, eventually coming to a safe stop within the lane or on the shoulder.

[0085] The parameters of the entire process, such as the duration of each level of reminder and the trigger threshold, can be calibrated according to regulatory requirements and vehicle model positioning.

[0086] The steering wheel hands-off detection method provided in this application acquires multimodal interaction state data, including contact, torque, and vision. It first performs independent single-modal hands-off state judgments on each type of data, then integrates and merges multiple judgment results for a comprehensive decision, and finally executes a tiered response process based on the decision results. This effectively solves the problem of frequent false detections and missed detections caused by the limitations of sensor principles and insufficient scene adaptability in single or simple dual-modal detection schemes. Furthermore, the pre-emptive single-modal judgment ensures the quality of basic data, and the multimodal fusion decision integrates multi-dimensional complementary information, effectively overcoming the perception failures of traditional schemes in low-torque, visually obstructed, or specific deceptive scenarios. Therefore, in complex and ever-changing real-world driving environments, it significantly improves the overall accuracy, robustness, and real-time performance of hands-off state determination, providing crucial technical support for the safe and reliable operation of advanced driver assistance systems.

[0087] Figure 3 Another embodiment of the steering wheel hands-off detection method provided in this disclosure, in Figure 2 Based on the illustrated embodiment, the following is combined with Figure 3 The implementation process of the steering wheel hands-off detection method is explained in detail, and it specifically includes the following steps:

[0088] S301. Obtain interaction status data from the steering wheel system.

[0089] The interaction status data includes at least contact sensing data, torque sensing data, and visual sensing data.

[0090] Specifically, this step is related to Figure 2 The corresponding steps in the illustrated embodiment are the same. The system will synchronously receive the capacitive sensing signal from HOD, the steering torque signal from EPS, and the video stream data from DMS camera, which will not be described in detail here.

[0091] S302. Perform jitter removal processing on the contact sensing data, and obtain a first judgment result based on the processed contact sensing data and a preset first threshold.

[0092] Specifically, the system will perform jitter removal processing on the original "released / not released" status signal from the HOD to eliminate rapid signal jumps caused by electrical noise, instantaneous contact or environmental interference, and ensure the stability of status determination.

[0093] A typical implementation approach is to use time-based hysteresis logic or a finite state machine. For example, the system does not immediately change its output state due to a single signal transition. Instead, it only determines the state as "disengaged" when the received "disengaged" signal persists for more than a preset stable time window (such as 300 milliseconds) (the first judgment result is "disengaged"), and vice versa.

[0094] The preset first threshold is the length of this stable time window, and its value can be calibrated through real vehicle testing to achieve a balance between response speed and anti-interference.

[0095] S303. Filter the torque sensing data and obtain a second judgment result based on the filtered torque value, its statistical characteristics in a continuous time period, and a preset second threshold.

[0096] Specifically, the system filters the raw hand torque signal provided by EPS, for example, by using Kalman filtering or low-pass filtering, to remove high-frequency noise and obtain the optimal estimated torque value.

[0097] Subsequently, the system will not only examine the instantaneous torque value after filtering, but also analyze its statistical characteristics over a continuous period (such as the most recent 1.5 seconds), mainly the variance.

[0098] The principle is that the torque exerted by the driver actively controlling the steering wheel usually has a certain degree of fluctuation, while the torque generated by the heavy object used to deceive the system (such as a water bottle) may exhibit a "fixed" characteristic with a high mean but low variance, or a "no input" characteristic with a low mean and low variance.

[0099] Furthermore, if the preset second threshold includes a preset torque threshold and a preset variance threshold, the method for determining the second judgment result can be: if the filtered torque value is less than the preset torque threshold, and its variance in a continuous time period is less than the preset variance threshold, then the second judgment result is determined to be that there is a release state.

[0100] Specifically, the core judgment logic here is: if the filtered torque value is less than the preset torque threshold (indicating that the input is very weak), and the variance in the same continuous time period is less than the preset variance threshold (indicating that there is a lack of fluctuation), then the system determines that the second judgment result is that there is a slip-out state.

[0101] This combination of conditions effectively distinguishes low-torque cruise (which may have minor corrections, but the variance will not be extremely low) from genuine off-hand or deceptive loads.

[0102] In some embodiments, when obtaining the second judgment result based on torque sensing data, there are further judgment logic branches. For example, if the filtered hand torque value is greater than a preset torque threshold, but its value remains stable within a narrow fixed range over a continuous period of time, then this state is identified as suspected cheating and needs to be transferred to the subsequent anti-cheating verification process for key confirmation, rather than being directly judged as not having released the hand.

[0103] S304. Perform hand region recognition processing on the visual sensing data, and obtain a third judgment result based on the processed visual sensing data and a preset third threshold.

[0104] Specifically, the system will perform frame-by-frame or frame-skipping analysis on the DMS video stream to determine the driver's hand movements.

[0105] Specifically, by running a computer vision model or algorithm (e.g., a hand keypoint detection model trained on an open-source deep learning framework), the system can identify the position of the driver's hand in the image and determine whether the identified hand keypoints (such as fingertips and the center of the palm) fall within the predefined image coordinate range of the steering wheel grip area.

[0106] Based on the processed results, such as calculating the overlap between the hand area and the steering wheel area and comparing it with a preset third threshold (such as an overlap ratio of 50%), if the overlap is below the threshold for multiple consecutive frames, the system can determine that the third judgment result is that the hand has been released.

[0107] S305. Based on the result of the single-mode release state judgment corresponding to the torque sensing data, determine that there is a torque value that meets the preset fixed characteristic conditions, and trigger the anti-cheating verification.

[0108] Specifically, in the process of obtaining the second judgment result, the system will analyze in parallel whether the torque data exhibits preset fixed characteristic conditions.

[0109] This condition specifically refers to the torque value being consistently within a narrow numerical range, with a potentially high mean (simulating hand grip force) but an extremely low variance, which is significantly different from the statistical characteristics of human dynamic control.

[0110] Once this pattern is identified, regardless of the preliminary conclusion of the second judgment result, the system will proactively trigger a dedicated anti-cheating verification process, which has a higher priority than the regular weighted fusion path.

[0111] S306. In the anti-cheating verification, the result of the hand release detection is obtained based on the result of the single-modal hand release state judgment corresponding to the visual sensing data.

[0112] Specifically, the system's decision-making logic changes during the triggered anti-cheating verification process.

[0113] At this point, the system will primarily or entirely rely on the third judgment result obtained based on visual sensing data to determine the final hand release detection result.

[0114] The principle is that visual information in this scenario provides direct evidence that cannot be easily forged by cheating methods (whether the hands are really on the steering wheel).

[0115] For example, even if the torque sensor shows a continuous force, but the vision does not detect the hand in the steering wheel area for several consecutive frames, the system will ultimately determine that the hand is out of control.

[0116] S307. Determine the weighting coefficients corresponding to the results of the single-mode release state judgment of contact sensing, torque sensing and vision sensing.

[0117] Specifically, for routine cases where anti-cheating verification is not triggered, the system adopts a weighted fusion strategy.

[0118] Therefore, the system needs to determine a weighting coefficient for the first (contact), second (torque), and third (visual) judgment results respectively.

[0119] Furthermore, the weighting coefficients are dynamically calibrated based on the confidence level of each sensor data or the characteristics of the current driving scenario, and the sum of each weighting coefficient is 1.

[0120] Specifically, in practical applications, these weighting coefficients are not fixed, but can be dynamically calibrated.

[0121] The calibration criteria may include: the real-time confidence level of each sensor data (such as sensor self-test status and signal quality indicators); and the characteristics of the current driving scenario (for example, when the vehicle is driving at a constant speed on a straight road, the normal torque variance may be very small, so the torque weight can be appropriately reduced and the visual weight increased; while the opposite is true in sharp curve scenarios).

[0122] The purpose of dynamic calibration is to enable the fusion strategy to adapt to changes in the environment, and the calibration must meet the constraint that the sum of all weight coefficients is 1, so as to ensure the consistency of the fusion calculation.

[0123] S308. The judgment results of each single-mode release state are weighted based on the weight coefficient to obtain the fusion judgment value.

[0124] Specifically, the system performs weighted calculations. The principle is to multiply the judgment results of each modality (for example, quantify "not released" as 1 and "released" as 0, or use probability values) by their corresponding dynamic weight coefficients and then sum them up, thereby unifying the multidimensional and heterogeneous judgment information into a single scalar value, which is convenient for subsequent decision-making.

[0125] S309. Compare the fused judgment value with the preset decision threshold, and obtain the off-hand detection result based on the comparison result.

[0126] Specifically, the system compares the calculated fusion judgment value with a preset decision threshold.

[0127] For example, the decision logic can be as follows: if the fusion judgment value is greater than or equal to the preset decision threshold, the final detection result of the release is determined to be not released; if the fusion judgment value is less than the preset decision threshold, the release is determined to be released.

[0128] This threshold determines the overall leniency or strictness of the system's judgment and can be obtained through calibration using a large amount of real vehicle data.

[0129] Steps S307 to S309 are optional steps that are parallel to steps S305 to S306. Those skilled in the art can choose to perform the corresponding steps according to the actual situation, and there are no restrictions here.

[0130] S310. In response to the hand-off detection result indicating a hand-off status, execute a hand-off alarm procedure including at least two levels of prompts.

[0131] Specifically, once the system obtains the final hand-drop detection result, it will initiate a tiered hand-drop alarm process.

[0132] At this point, the first-level warning might be a flashing visual icon on the dashboard and a soft warning sound, intended to alert the driver that their hands have left their hands.

[0133] If the hands-free state persists (e.g., 2-3 seconds), the alarm escalates to Level 2, which may include more frequent beeps, icon color changes, or slight vibrations in the steering wheel.

[0134] Each alarm level has its own duration and triggering conditions, forming a progressively escalating warning sequence that gives the driver ample reaction time.

[0135] In some embodiments, the time parameters in the aforementioned continuous time period or alarm process (such as descriptions like "within a certain continuous time period" or "within a certain time period after the last level alarm") are all calibrable time thresholds. For example, the time window for torque variance determination and the duration of each level of alarm can be specifically set and calibrated according to vehicle performance and regulatory requirements.

[0136] S311. If no effective interaction with the steering wheel is detected after the hands-off alarm process is completed, the vehicle safety stop control is triggered.

[0137] Specifically, the completion of the hands-free alarm process here refers to the system having executed all preset alarm actions (e.g., from level one to level three).

[0138] During this period, the system will continue to monitor. If the steering wheel is not detected to have resumed effective interaction after the hands-off alarm process is completed (for example, no torque fluctuation that meets the requirements and visual confirmation that the hands have not returned), the system will continue to monitor.

[0139] At this point, the system will trigger a safe parking control action.

[0140] This control is not emergency braking, but rather, based on a comprehensive perception of the safety of the surrounding environment, it automatically controls the vehicle to decelerate smoothly, activate hazard warning lights, and ultimately come to a stop within the lane or a safe area, in order to achieve the highest level of safety assurance.

[0141] The steering wheel hands-off detection method provided in this disclosure further refines the single-modal data processing method (such as de-jittering of contact signals and analysis of the combined value and variance of torque signals) on the basis of a multimodal fusion decision-making mechanism. It also introduces an anti-cheating verification mechanism for fixed torque deception scenarios and a dynamic weighted fusion strategy based on confidence level and scenario. Therefore, it not only possesses basic multi-source information fusion capabilities but also achieves accurate identification and special handling for specific high-risk scenarios such as water bottle-hanging cheating, low-torque misjudgment, and sensor reliability fluctuations under complex operating conditions, as well as adaptive decision-making capabilities for different driving environments. This maintains extremely high detection accuracy and system robustness in various edge and extreme cases, significantly improving the safety ceiling of advanced driver assistance functions.

[0142] Figure 4 This is a schematic diagram of the structure of a steering wheel hands-off detection device provided in one embodiment of this disclosure. Figure 4 As shown, the steering wheel hands-off detection device 400 includes:

[0143] The acquisition module 410 is used to acquire interaction status data from the steering wheel system, wherein the interaction status data includes at least contact sensor data, torque sensor data and visual sensor data.

[0144] Analysis module 420 is used to perform single-modal release state judgment for each type of interaction state data based on the interaction state data;

[0145] Processing module 430 is used to fuse the results of each single-modal release state judgment to obtain the corresponding release detection result;

[0146] The execution module 440 is used to execute a preset hierarchical response process in response to the release detection result indicating the release status.

[0147] Optionally, the processing module 430 is specifically used to determine, based on the result of the single-modal release state judgment corresponding to the torque sensing data, that there exists a torque value that satisfies the preset fixed characteristic conditions, and trigger anti-cheating verification; in the anti-cheating verification, based on the result of the single-modal release state judgment corresponding to the visual sensing data, the release detection result is obtained.

[0148] Optionally, the processing module 430 is specifically used to: determine the weight coefficients corresponding to the results of the single-modal release state judgment of the contact sensor, torque sensor and vision sensor; perform weighted calculation on the release state judgment results of each single-modal sensor based on the weight coefficients to obtain a fused judgment value; compare the fused judgment value with a preset decision threshold, and obtain the release detection result based on the comparison result.

[0149] Optionally, the processing module 430 specifically includes dynamically calibrating the weight coefficients based on the confidence level of each sensor data or the characteristics of the current driving scenario, and the sum of each weight coefficient is 1.

[0150] Optionally, the execution module 440 is specifically used to execute a hands-free alarm process including at least two levels of prompts in response to the hands-free detection result indicating a hands-free state; if no effective steering wheel interaction is detected after the hands-free alarm process is completed, the vehicle safety parking control is triggered.

[0151] Optionally, the analysis module 420 is specifically used to: perform jitter removal processing on the contact sensing data, and obtain a first judgment result based on the processed contact sensing data and a preset first threshold; perform filtering processing on the torque sensing data, and obtain a second judgment result based on the filtered torque value and its statistical characteristics in a continuous time period and a preset second threshold; and perform hand area recognition processing on the visual sensing data, and obtain a third judgment result based on the processed visual sensing data and a preset third threshold.

[0152] Optionally, the analysis module 420 is specifically used to determine that the second judgment result is that there is a release state when the preset second threshold includes a preset torque threshold and a preset variance threshold.

[0153] In this embodiment, the steering wheel hands-off detection device solves the problem of misjudgment and impact on driving safety in related technologies by combining various modules.

[0154] Figure 5 This is a schematic diagram of the structure of a control device provided in one embodiment of the present disclosure, as shown below. Figure 5 As shown, the control device 500 includes a memory 510 and a processor 520.

[0155] The memory 510 stores a computer program that can be executed by at least one processor 520. This computer program is executed by at least one processor 520 to enable the control device to implement the steering wheel hands-off detection method provided in any of the above embodiments.

[0156] The memory 510 and the processor 520 can be connected via a bus 530.

[0157] The relevant explanations can be understood by referring to the corresponding descriptions and effects in the method embodiments, and will not be repeated here.

[0158] One embodiment of this disclosure provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steering wheel hands-off detection method provided in any of the above embodiments.

[0159] The computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0160] One embodiment of this disclosure provides a computer program product comprising computer execution instructions that, when executed by a processor, are used to implement the steering wheel hands-off detection method provided in any of the above embodiments.

[0161] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0162] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0163] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0164] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for detecting steering wheel release from hands, characterized in that, Includes the following steps: Acquire interaction status data from the steering wheel system, wherein the interaction status data includes at least contact sensor data, torque sensor data, and visual sensor data; Based on the interaction state data, a single-modal release state judgment is performed for each type of interaction state data; The results of each single-modal release state judgment are fused to make a decision, and the corresponding release detection result is obtained. In response to the release detection result indicating a release state, a preset hierarchical response process is executed.

2. The method according to claim 1, characterized in that, The process of fusing the results of each single-modal release state judgment to obtain the corresponding release detection result includes: Based on the results of the single-mode release state judgment corresponding to the torque sensing data, it is determined that there is a torque value that meets the preset fixed characteristic conditions, and the anti-cheating verification is triggered. In the anti-cheating verification, the hand-drop detection result is obtained based on the result of the single-modal hand-drop state judgment corresponding to the visual sensing data.

3. The method according to claim 1, characterized in that, The process of fusing the results of each single-modal release state judgment to obtain the corresponding release detection result includes: Determine the weighting coefficients corresponding to the results of the single-modal release state judgment of contact sensing, torque sensing, and visual sensing; The weighted calculation of the judgment results of each single-mode release state is performed based on the weighting coefficients to obtain the fused judgment value; The fusion judgment value is compared with a preset decision threshold, and the hand-release detection result is obtained based on the comparison result.

4. The method according to claim 3, characterized in that, The weighting coefficients are dynamically calibrated based on the confidence level of each sensor data or the characteristics of the current driving scenario, and the sum of each weighting coefficient is 1.

5. The method according to claim 1, characterized in that, The response to the release detection result indicating a release state executes a preset hierarchical response procedure, including: In response to the release detection result indicating a release status, a release alarm process including at least two levels of prompts is executed; If no effective steering wheel interaction is detected after the hands-off alarm process is completed, the vehicle safety parking control will be triggered.

6. The method according to any one of claims 1 to 5, characterized in that, The step of determining the single-modal release state for each type of interaction state data based on the interaction state data includes: The contact sensing data is subjected to jitter removal processing, and a first judgment result is obtained based on the processed contact sensing data and a preset first threshold. The torque sensing data is filtered, and a second judgment result is obtained based on the filtered torque value, its statistical characteristics over a continuous time period, and a preset second threshold. The visual sensing data is processed to identify the hand region, and a third judgment result is obtained based on the processed visual sensing data and a preset third threshold.

7. The method according to claim 6, characterized in that, The preset second threshold includes a preset torque threshold and a preset variance threshold; The second judgment result is obtained based on the filtered torque value, its statistical characteristics over a continuous time period, and a preset second threshold, including: If the filtered torque value is less than the preset torque threshold, and its variance over a continuous period is less than the preset variance threshold, then the second judgment result is determined to be that there is a release state.

8. A steering wheel hands-off detection device, characterized in that, include: The acquisition module is used to acquire interaction status data from the steering wheel system, wherein the interaction status data includes at least contact sensor data, torque sensor data and visual sensor data. The analysis module is used to perform single-modal release state judgment on each type of interaction state data based on the interaction state data. The processing module is used to fuse the results of each single-modal release state judgment to obtain the corresponding release detection result; The execution module is used to execute a preset hierarchical response process in response to the release detection result indicating the release status.

9. A control device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 7.

Images