Steering wheel hand detection method and device, electronic equipment and movable platform

Abstract

The application discloses a steering wheel hand detection method and device, electronic equipment and movable platform. The embodiment of the application obtains steering wheel hand data of a user, the steering wheel hand data including at least two of a hand holding probability based on a sensor, a hand holding probability based on vision and a hand holding probability based on dynamics, and determines the weight of the steering wheel hand data based on external environment information, so as to determine the hand holding evaluation score of the user by using the weight and the steering wheel hand data, compare the hand holding evaluation score with a target threshold, and determine whether the user leaves the hand according to the comparison result. As can be seen, when the steering wheel is detected, the hand holding probability of the user is determined from different angles, and the difference caused by environmental factors is comprehensively considered, the weight of each hand holding probability is dynamically adjusted by using external environment information, the user can be more accurately determined to leave the hand, and the accuracy of the steering wheel hand detection is improved.

Description

Technical Field

[0001] This application belongs to the field of automotive technology, specifically relating to a method, device, electronic device, and mobile platform for detecting hand grip on a steering wheel. Background Technology

[0002] With technological advancements, current automotive functions are becoming increasingly intelligent. For example, in driver assistance systems, Hands-On Detection (HOD) technology is a key safety mechanism that ensures a smooth handover of user attention and vehicle control.

[0003] In related technologies, HOD (Hands-Off) technology generally relies on detection data from sensors or cameras to directly determine whether the user has taken their hands off the wheel while driving. However, in practice, it has been found that the accuracy of steering wheel HOD technology is insufficient, and it is prone to false alarms or missed alarms when the hands are off the wheel. Summary of the Invention

[0004] The purpose of this application is to provide a steering wheel hand grip detection method, device, electronic device, and mobile platform to solve the technical problem of insufficient accuracy in steering wheel hand grip detection technology in related technologies.

[0005] To address the aforementioned technical problems, the first aspect of this application provides a method for detecting hand grip on a steering wheel, the method comprising: Acquire user steering wheel grip data, which includes at least two of the following: sensor-based grip probability, vision-based grip probability, and dynamic grip probability; Obtain external environment information of the mobile platform, and determine the weight of the steering wheel grip data based on the external environment information; The user's hand grip assessment score is determined using the weights and the steering wheel grip data. The hand grip assessment score is compared with a target threshold, and the user is determined to have let go of the hand based on the comparison result.

[0006] A second aspect of this application provides a steering wheel hand grip detection device, the device comprising: The acquisition module is used to acquire the user's steering wheel grip data, which includes at least two of the following: sensor-based grip probability, vision-based grip probability, and dynamic grip probability. The acquisition module is also used to acquire external environment information of the mobile platform; The determination module is used to determine the weight of the steering wheel grip data based on the external environment information; The determining module is also used to determine the user's hand grip evaluation score using the weights and the steering wheel grip data; The comparison module is used to compare the hand grip assessment score with the target threshold; The determining module is also used to determine whether the user has let go of the device based on the comparison result.

[0007] A third aspect of this application provides an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0008] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0009] Fifthly, embodiments of this application provide a computer program product, the computer program product including a computer program or instructions, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0010] In a sixth aspect, embodiments of this application provide a mobile platform that includes an electronic device as described in the third aspect, or implements the steps of the method as described in the first aspect.

[0011] In this embodiment, by acquiring user steering wheel grip data, including at least two of the following: sensor-based grip probability, vision-based grip probability, and dynamic grip probability, and determining the weights of the steering wheel grip data based on external environmental information, a user grip evaluation score is determined using these weights and the steering wheel grip data. This score is then compared to a target threshold, and the comparison result determines whether the user has removed their hand. Therefore, this embodiment, when detecting steering wheel grip, determines the user's grip probability from different angles, comprehensively considers differences caused by environmental factors, and dynamically adjusts the weights of each grip probability using external environmental information. This allows for a more accurate determination of whether the user has removed their hand, thereby improving the accuracy of steering wheel grip detection. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the system architecture for steering wheel hand grip detection provided in an embodiment of this application; Figure 2 This is a flowchart illustrating a steering wheel hand grip detection method provided in an embodiment of this application; Figure 3This is a flowchart illustrating another steering wheel hand grip detection method provided in an embodiment of this application; Figure 4 This is a flowchart illustrating another steering wheel hand grip detection method provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a steering wheel hand grip detection device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0015] In some related technologies, the hand-grip detection technology in driver assistance systems generally relies on high-precision capacitive sensors (based on the steering wheel heating coil) and a fixed reference value calibration algorithm to determine whether the user has taken their hands off the wheel while driving. This approach has the following problems: 1) The capacitance reference value is fixed at the moment the vehicle starts. If the user's hands are already on the steering wheel before starting, it will cause the reference value to drift, resulting in a decrease in subsequent sensitivity.

[0016] 2) It is difficult to distinguish whether the hand covers the object or the back of the hand touches it accidentally, and it is greatly affected by temperature.

[0017] In other related technologies, gesture recognition and key hand point detection are used in the driver monitoring system controller (DMS) within the cockpit to determine whether the user has taken their hands off the wheel while driving. However, this method also suffers from the following problems: 1) Visual signals may be lost when there is strong backlight, when the user is wearing gloves, or when the steering wheel obstructs the view (e.g., the hands are blocked by the wheel spokes when turning the steering wheel).

[0018] 2) There is an algorithm delay, making it difficult to accurately detect the user's hand grip in the "stroking" state.

[0019] Some related technologies employ relatively simple fusion strategies, such as the fusion of capacitance and torque, to determine whether the user has taken their hands off the wheel during driving. However, this approach is less effective in complex scenarios, such as single-handed steering or uneven force application, as its decision-making logic is rigid and the false alarm rate is high.

[0020] Therefore, to address the aforementioned problems, this application provides a steering wheel hand grip detection method, device, electronic device, and mobile platform. When detecting hand grip on the steering wheel, the method determines the probability of the user's hand grip from different angles and comprehensively considers differences caused by environmental factors. It dynamically adjusts the weights of each hand grip probability using external environmental information, enabling more accurate judgment of whether the user has removed their hands, thereby improving the accuracy of steering wheel hand grip detection. Furthermore, this application employs personalized thresholds, enabling more precise hand grip detection for each user, thereby reducing false alarm rates, improving driving safety, and enhancing the driving experience.

[0021] The steering wheel hand grip detection method, device, electronic device, and mobile platform provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0022] Please see Figure 1 , Figure 1 This is a schematic diagram of the system architecture for steering wheel hand grip detection provided in an embodiment of this application. Figure 1 As shown, the system includes a perception layer, a decision-making layer, an execution layer, and a cloud service platform.

[0023] It is understood that the system provided in this application embodiment can be deployed on a mobile platform with driver control components (such as a steering wheel) installed inside. In this application embodiment, the driver is also referred to as the user. The mobile platform can be a vehicle, a train, an aircraft, or other forms of mobile platform requiring driver control of the driving direction; this application embodiment does not limit this. For ease of understanding, this application embodiment uses a vehicle as an example of a mobile platform.

[0024] In this embodiment, the perception layer can collect raw data using various controllers in the vehicle, specifically including the Electronic Control Unit (ECU), DMS controller, chassis domain controller, and intelligent driving domain controller.

[0025] The steering wheel ECU can collect data related to the steering wheel through sensors installed on (or near) the steering wheel, such as the multi-frequency impedance of the steering wheel, contact pressure, contact point, time point, user's skin surface humidity, and user's skin conductivity.

[0026] The DMS controller can collect vision-related information, such as 3D depth, RGB images, user arm bone points, head tilt angle, line of sight deviation, eyelid closure, and interior lighting, through cameras installed near the driver's seat.

[0027] The chassis domain controller can collect vehicle dynamics data, such as electric power steering (EPS) torque, steering wheel angle, steering wheel angular velocity, vehicle speed, yaw rate, vertical acceleration, and suspension travel. The intelligent driving domain controller can collect external environmental information, such as GPS location, weather type, road condition type, and external lighting.

[0028] In this embodiment, the decision layer includes a steering wheel ECU, a DMS controller, a chassis domain controller, a cockpit domain controller, and an intelligent driving domain controller. In the decision layer, the steering wheel ECU and each controller are used to perform fusion calculations on the data collected by the perception layer and to determine hand grip detection.

[0029] Specifically, the steering wheel ECU can call Model 1 (e.g., a hand physical feature decoupling model based on multi-frequency impedance spectrum) to perform multi-frequency impedance decoupling on the raw data collected by the steering wheel ECU; the DMS controller is used to extract features from visually related information; the chassis domain controller is used to extract features from vehicle dynamics data; the cockpit domain controller is used to identify the user and suggest a personalized baseline module; the intelligent driving domain controller can call Model 2 (e.g., a dynamic feature weighted fusion model based on attention mechanism) to perform attention-weighted fusion processing on the data output by the steering wheel ECU, DMS controller, and chassis domain controller, combined with the external environment information collected by the intelligent driving domain controller; the intelligent driving domain controller can also further combine the personalized baseline module established by the cockpit domain controller and call Model 3 (e.g., a personalized baseline transfer learning model based on user identification) to perform adaptive threshold decision, thereby outputting the decision result.

[0030] In this embodiment, the execution layer is used to execute corresponding tasks based on the decision results output by the decision layer. Specifically, the execution layer includes a chassis domain controller, a cockpit domain controller, and a smart driving domain controller.

[0031] For example, when the decision-making layer outputs a judgment that the user should release the hands, the chassis domain controller can perform actions such as EPS degradation, sending braking requests, or taking over requests through the vehicle control interface; the cockpit domain controller can output warning or reminder information through human-machine interaction (HMI), such as instrument / center console display, voice reminders, or steering wheel vibration; and the intelligent driving domain controller can generate alarm strategies through the Advanced Driver Assistance Systems (ADAS) interaction controller.

[0032] In this embodiment, the cloud service platform mainly implements the following: uploading user driving data to the cloud feature library through a gateway, creating user profiles for different users, dynamically updating user driving data for model optimization, and updating the various controllers on the vehicle side through Over-The-Air (OTA) technology.

[0033] In this embodiment, the architecture fully presents all collected data, controllers, and data flow, conforming to the "perception-decision-execution" three-layer system and supporting "personalized" closed-loop optimization. This architecture enables more accurate hand-grip detection for different users, effectively reducing false alarms and missed alarms related to hand-off detection, thereby improving driving safety and enhancing the user's driving experience.

[0034] Based on the above system architecture, this application provides a steering wheel hand grip detection method. This method can be applied to a steering wheel hand grip detection device (hereinafter referred to as the detection device). It is understood that the detection device may include, for example: Figure 1 The system architecture shown includes the steering wheel ECU, DMS controller, chassis domain controller, intelligent driving domain controller, and cockpit domain controller. Please refer to [link / reference needed]. Figure 2 , Figure 2 This is a flowchart illustrating a steering wheel hand grip detection method provided in an embodiment of this application, as shown below. Figure 2 As shown, the method includes the following steps.

[0035] S21. Obtain the user's steering wheel grip data.

[0036] In this embodiment, the steering wheel grip data is based on at least two of the following: sensor-based grip probability, vision-based grip probability, and dynamic grip probability. In a preferred embodiment, the steering wheel grip data includes all three.

[0037] Specifically, the detection device acquires the hand grip probability based on sensors, which can be obtained from the steering wheel ECU through a multi-frequency scan model. The sensor-based hand grip probability can be represented by f. MLS This indicates the probability of whether a user is holding the steering wheel, obtained from the processing of raw data collected by the steering wheel ECU. It is understood that the multi-frequency scanning model can be the hand physical feature decoupling model based on multi-frequency impedance spectrum mentioned earlier, or it can be two independent models; this application does not limit this.

[0038] Specifically, the detection device acquires the vision-based hand grip probability, which can be obtained by the DMS controller extracting visual features and outputting them through the visual DMS model. The vision-based hand grip probability can be represented by f. vision This indicates the probability of whether a user is holding the steering wheel, obtained from processing the raw data collected by the DMS controller.

[0039] Specifically, the detection device acquires the dynamic-based hand-grip probability, which can be obtained by the chassis domain controller extracting dynamic features and outputting them through the inverse model of vehicle dynamics. The dynamic-based hand-grip probability can be represented by f. dynamics This indicates the probability of whether a user is holding the steering wheel, obtained from processing the raw data collected by the chassis domain controller.

[0040] In some specific implementations, obtaining the sensor-based hand grip probability may include the following steps: S211. Obtain the contact resistance between the user and the steering wheel.

[0041] In this embodiment, the contact resistance between the user and the steering wheel can be represented by R. skin Specifically, this can refer to the skin contact resistance of the user's hand when touching the steering wheel. Generally, the R of dry skin... skin Around 1MΩ, the R of moisturized skin skin Around 100MΩ.

[0042] In this embodiment, R can be calculated by measuring the user's skin surface humidity and skin conductivity. skin Alternatively, it can directly look up the corresponding R based on the detected skin surface humidity. skin However, the embodiments in this application do not limit this.

[0043] In a specific implementation, the method for obtaining the contact resistance between the user and the steering wheel in step S211 can be as follows: The contact area between the user and the steering wheel is detected; the skin parameters of the user in contact with the steering wheel are obtained; and the contact resistance between the user and the steering wheel is determined based on the contact area and the skin parameters.

[0044] In this embodiment, skin parameters may include skin surface moisture and skin conductivity. Skin surface moisture affects the water content of the stratum corneum; increased moisture increases ion mobility, thus decreasing the skin's contact resistance. Skin conductivity reflects the amount of sodium in sweat. + Cl - K + Plasma concentration. During periods of stress or increased temperature, sweat gland secretion increases, and skin conductivity can rise by an order of magnitude. Therefore, a user's skin conductivity will vary in different scenarios, thus affecting R... skin The contact area between the user and the steering wheel also affects R. skin Therefore, by detecting the contact area between the user and the steering wheel, the skin surface humidity, and the skin conductivity, the contact resistance R between the user and the steering wheel can be calculated. skin .

[0045] In other implementations, other parameters may be introduced for calculation, and this application does not limit the specific implementation.

[0046] In an exemplary implementation, the contact resistance R can be pre-established. skin The function relating skin surface moisture and skin conductivity is as follows: (1) Among them, R skin (H, δ) is a function of skin surface humidity and skin conductivity, through which the contact resistance R between the user and the steering wheel can be obtained. skin R dry Representing skin resistance in a completely dry state, exemplarily, R dry Within the range of 100kΩ-1MΩ.

[0047] H represents the skin surface humidity, expressed as %RH, which can be measured using a humidity sensor or inferred from impedance spectroscopy. For example, at high frequencies, such as above 100kHz, the current primarily flows through the contact capacitance C. contact For contact resistance R skin The impact is relatively small. Therefore, the skin surface humidity H can be inferred by comparing the difference between the measured impedance and the calibrated impedance under dry conditions, combined with the change in impedance amplitude in the high-frequency band.

[0048] H0 represents the reference humidity; for example, H0 could be 50%RH. δ represents skin conductivity, measured in S / m. A represents the contact area between the user and the steering wheel, specifically the contact area between the user's hand and the steering wheel, measured in cm².2 Specifically, this can be measured using sensors.

[0049] α0 and β0 are constants, characterizing the effect of skin surface humidity H on contact resistance R. skin The nonlinear effect can be specifically obtained by measuring skin resistance under different humidity conditions and fitting the data. For example, α0 can be in the range of [2, 5] and β0 can be in the range of [1, 2]. In other examples, α0 and β0 can also take other values, which are not limited in the embodiments of this application.

[0050] Therefore, by detecting the contact area A between the user and the steering wheel and the skin parameters (skin surface humidity H and skin conductivity δ) when the user is in contact with the steering wheel, the contact resistance R between the user and the steering wheel can be calculated using the above function. skin .

[0051] It should be noted that the contact area A between the user and the steering wheel and the skin parameters of the user when in contact with the steering wheel can be obtained by the steering wheel ECU of the movable platform through sensors.

[0052] In this embodiment, the effects of skin surface humidity H, skin conductivity δ, and contact area A on contact resistance R under different scenarios are comprehensively considered. skin The influence of the contact resistance R obtained under different scenarios skin There will be differences, therefore, the contact resistance R can be increased. skin This improves the accuracy of the user's hand-holding assessment score obtained in subsequent processes, thereby enabling more accurate judgment of whether the user has taken their hands off the vehicle while driving on a mobile platform, thus enhancing driving safety and the user's driving experience.

[0053] S212, Detect the impedance information of the steering wheel.

[0054] In this embodiment, the impedance information of the steering wheel can be understood as the total impedance of the steering wheel. Specifically, the steering wheel ECU has a built-in multi-frequency impedance measurement circuit, which can use the four-wire AC impedance measurement method. For example, a 1kHz-1MHz sinusoidal excitation signal is injected into the steering wheel heating coil through one pair of electrodes, and the phase difference between the response voltage and current is measured by another pair of electrodes. The complex impedance Z(f) = V / I at each frequency point is calculated, thereby obtaining the total impedance of the steering wheel.

[0055] It is understood that steps S211 and S212 can be executed simultaneously or in a set order, and this application embodiment does not limit this.

[0056] S213. Determine the contact capacitance between the user and the steering wheel based on the contact resistance and impedance information.

[0057] In this embodiment of the application, in order to solve the problem that traditional single-point capacitance measurement cannot distinguish between "wet hands", "dry hands", "pressing the hand hard" and "ambient temperature drift", this embodiment of the application can establish a decoupling model of hand physical characteristics based on multi-frequency impedance spectrum (hereinafter referred to as Model 1). Specifically, it can be achieved by scanning multiple frequency points f to construct an impedance spectrum, thereby separating the contact resistance between the user and the steering wheel and the contact capacitance between the user and the steering wheel.

[0058] Specifically, Model 1 can decouple the steering wheel's impedance information by utilizing the contact resistance between the user and the steering wheel to obtain the contact capacitance between the user and the steering wheel. Then, using this contact capacitance and contact resistance, the sensor-based hand grip probability f can be calculated. MLS .

[0059] In other embodiments, the impedance information of the steering wheel can be decoupled to separate the contact capacitance between the user and the steering wheel, the contact resistance between the user and the steering wheel, and other parasitic parameters. This application does not limit this.

[0060] In an exemplary implementation, Model 1 can be defined as: (2) Where f represents the scanning frequency of Model 1 (unit: Hz), and for example, the scanning range can be from 1 kHz to 1 MHz.

[0061] Z total (f) represents the impedance information of the steering wheel measured at frequency f.

[0062] R skin (H, δ) is a function of skin surface humidity H and skin conductivity δ, and the contact resistance R between the user and the steering wheel can be calculated using this function. skin Understandably, R skin Model 1 can be achieved through the function R skin (H, δ) can be obtained by outputting (H, δ) or calculated by other means, but this application does not limit this.

[0063] C contact (F, A) represent grip strength F (unit: N) and contact area A (unit: cm²). 2 The capacitance is a function of the gripping force F. The greater the gripping force F, the larger the contact area A, and the higher the capacitance value.

[0064] L parasitic The parasitic inductance of the wiring harness in a movable platform can typically be obtained by connecting an impedance analyzer to the steering wheel wiring harness and measuring the inductive reactance at high frequencies. During the movement of the movable platform, the rate of change of the wiring harness's parasitic inductance can be monitored using high-frequency signals.

[0065] Rleak (T) represents the leakage resistance of the mobile platform as a function of temperature T, which can generally be obtained by measuring the insulation resistance of the wiring harness to the vehicle body using an insulation resistance tester. During the operation of the mobile platform, corrections can be made based on the temperature sensor readings and the temperature coefficient of the insulation material.

[0066] Therefore, using Model 1, the impedance information Z of the steering wheel can be obtained. total (f) Decouple the components and separate the contact resistance R between the user and the steering wheel. skin Contact capacitance C between the user and the steering wheel contact Parasitic inductance of wire harness L parasitic and leakage resistance R leak (T).

[0067] Understandably, the detection device obtains Z total (f), L parasitic R leak (T) and R skin Then, a pair of Z can be modeled. total (f) Decoupling is performed to obtain C contact .

[0068] S214. Determine the hand grip probability based on the contact resistance and the contact capacitance.

[0069] In this embodiment of the application, the detection device obtains the contact resistance R. skin and contact capacitance C contact After that, further research can be conducted based on R. skin And C contact Determine the sensor-based hand grip probability f MLS .

[0070] In some specific implementations, a sensor-based hand grip probability f is pre-established. MLS The function can be specifically: (3) Among them, C basline For each user's personalized initial capacitance parameters, the C values ​​for different users will vary. basline The difference is understandable; if the user is not a new user, the average historical contact capacitance of that user can be used as C. basline If the user is a new user, C can be determined by collecting driving data over a certain period of time. basline In other implementations, a fixed constant can be used, for example, setting different values ​​for C based on different genders, heights, or age groups. basline The user's C is determined based on their gender, height, or age. baslineHowever, the embodiments in this application do not limit this.

[0071] C th This is the average historical contact capacitance of all users who have driven this mobile platform.

[0072] Therefore, the embodiments of this application can use this function to determine the probability f of a user's hand gripping the vehicle based on sensors during driving. MLS .

[0073] In this embodiment of the application, the sensor-based hand grip probability f obtained through the above method is... MLS Taking into account factors such as hand dryness, pressure applied, and ambient temperature drift, the obtained f... MLS More precise.

[0074] It is understandable that the probability f of hand gripping based on sensors... MLS The contact capacitance C can be output in Model 1. contact Then, the detection device measures the contact resistance R. skin and contact capacitance C contact The result can be obtained through calculation or directly output from Model 1; however, this application does not limit the specific results in this embodiment.

[0075] In other specific embodiments, the detection device may obtain the vision-based hand grip probability in the following ways: By tracking the user's hand and extracting multiple hand skeletal points, the shortest Euclidean distance from key hand points to the steering wheel rim projection area is calculated. Then, the vision-based hand grip probability f is calculated using the contact point ratio and average distance. vision f vision The probability value is 0-1.

[0076] Specifically, the BlazePalm algorithm can be used to identify the user's hands, thereby determining the area of ​​the user's hands on the steering wheel. Then, through the hand key point recognition network, the 3D coordinates (x, y, z) of 21 key points of the hand are calculated in the hand area. These 21 key points can be understood as "joint markers" of the user's hands. The x and y coordinates can be understood as the relative positions of the key points in the image, and the z coordinate represents the distance of the point from the camera.

[0077] Furthermore, a 3D digital model corresponding to the steering wheel in the car is constructed. The positional relationship of this 3D steering wheel projected onto the camera image is calculated through camera calibration. Then, the outer rim of the steering wheel (the ring held by the user's hand) is broken down into many small points, forming a set of contour points. For each key point identified above, the straight-line distance from each key point to the nearest point on the steering wheel contour is calculated. The minimum or average value of these distances is determined as the shortest Euclidean distance from the hand key point to the steering wheel.

[0078] Further, the number of keypoints with a shortest Euclidean distance less than a preset threshold (e.g., 2 cm) is determined, and the ratio of this number to the total number of keypoints (21) is calculated to obtain the contact ratio. The contact ratio, the minimum of the shortest Euclidean distances of each keypoint, and the shortest Euclidean distance of each keypoint are input into a logistic regression model or a fully connected neural network to output the vision-based hand grip probability f. vision .

[0079] In some implementations, the user's body tilt angle (Pose), the user's hand key point deviation, and the user's gaze deviation (Gaze) can also be input into the visual DMS model, and the model can then output the user's vision-based hand grip probability f during driving. vision .

[0080] It is understandable that parameters such as the human body tilt angle pose, the deviation of the user's hand key points, and the deviation of the user's line of sight can be obtained by the DMS controller through the camera.

[0081] In some specific implementations, the detection device may obtain the probability of hand gripping based on dynamics in the following ways: Using the EPS torsion bar signal and motor current, a state observer determines the driver's actual applied torque T_driver and steering wheel speed ω based on the EPS torsion bar signal T_torsion, motor assist torque T_motor, and steering wheel moment of inertia J_wheel. Then, based on T_driver and ω, the dynamic probability f of the user's hand grip during driving is determined. dynamics .

[0082] Specifically, the steering wheel is equipped with a torsion bar torque sensor and a steering wheel angle sensor. These sensors can detect basic data such as the force applied by the user when turning the steering wheel, the angle of steering wheel rotation, and the steering wheel rotation speed. The electric motor assist torque T_motor can be calculated from the motor current, and the EPS torsion bar signal T_torsion can be obtained from the torsion bar detection. Based on the above basic data and the steering wheel rotational inertia J_wheel (which is a fixed value measured at the factory), the actual torque T_driver applied by the user to the steering wheel is calculated using the algorithm of the "state observer" (such as Kalman filter, Romberg observer).

[0083] In other words, the rotational acceleration of the steering wheel = (force applied by the user + T_motor - T_torsion - friction) ÷ weight of the steering wheel itself. The rotational acceleration of the steering wheel can be obtained by differentiating it with respect to the steering wheel rotational speed ω.

[0084] For example, T_driver and ω can be calculated using the following formula: (4) Where B is the damping coefficient of the steering wheel, with the unit being N·m·s / rad. For example, the value range of B can be [0.05, 0.3].

[0085] Furthermore, T_driver and ω are input into a logistic regression model or classifier to output the dynamic hand grip probability f. dynamics .

[0086] It is understandable that parameters such as the torsion bar signal and motor current of the EPS can be acquired by the chassis domain controller.

[0087] In an exemplary embodiment, the steering wheel ECU, DMS controller, and chassis domain controller respectively obtain f MLS f vision and f dynamics Then, it can be sent to the intelligent driving domain controller.

[0088] In this embodiment of the application, steering wheel grip data may include sensor-based grip probability f. MLS Vision-based hand grip probability f vision and the dynamic-based hand grip probability f dynamics Two of these methods can be used to detect the user's hand grip from multiple dimensions during driving, improving the accuracy of hand grip detection and thus enhancing driving safety and user experience.

[0089] In a preferred embodiment, the steering wheel grip data can include the above three types, thereby enabling more accurate detection of the user's grip, further improving the accuracy of grip detection, and further enhancing driving safety and experience.

[0090] In this embodiment, multi-frequency impedance spectroscopy decoupling solves the problem of false alarms caused by changes in humidity on the user's skin surface, such as during rainy days, effectively improving recognition accuracy. Through multi-dimensional coupling of hand grip probabilities, hand grip detection can be performed more quickly, achieving millisecond-level takeover response. Furthermore, this embodiment achieves high-precision detection without adding an additional dedicated HOD chip.

[0091] S22. Obtain external environment information of the mobile platform and determine the weight of steering wheel grip data based on the external environment information.

[0092] In this embodiment of the application, in order to solve the problem of different confidence levels of various sensors under different driving scenarios, such as bumpy roads, visual failure, and chaotic torque signals when cornering, this embodiment of the application obtains the external environment information of the mobile platform and determines the weight of steering wheel grip data based on the external environment information.

[0093] Specifically, steering wheel grip data includes sensor-based grip probability f. MLS Vision-based hand grip probability f vision and the dynamic-based hand grip probability f dynamics If at least two of the following are present, then the detection device can determine the weight corresponding to each hand grip probability based on external environmental information.

[0094] For example, the detection device may specifically determine the weights corresponding to the probabilities of each hand grip through the intelligent driving domain controller.

[0095] It is understandable that the probability f of hand gripping based on sensors... MLS Vision-based hand grip probability f vision and the dynamic-based hand grip probability f dynamics The corresponding weight coefficients are dynamically changing, and the weights are determined by environmental variables. The sum of the three weight coefficients is 1.

[0096] Of course, in other implementations, if the steering wheel grip data includes f MLS f vision and f dynamics There are two types, and the sum of the weighting coefficients corresponding to these two probabilities is 1.

[0097] In this embodiment, external environmental information may include road surface smoothness index, parasitic inductance change rate of the mobile platform, external ambient light intensity, cabin light intensity, steering wheel angular acceleration, etc., and f can be determined based on one or more of these factors. MLS f vision and f dynamics The corresponding weights can also be determined by combining the above external environmental information, f. MLS f vision and f dynamics The corresponding weights are not limited in the embodiments of this application.

[0098] In some specific implementations, steering wheel grip data includes sensor-based grip probability, vision-based grip probability, and dynamic grip probability. The specific method by which the detection device acquires external environmental information of the movable platform and determines the weights of the steering wheel grip data based on this information may include the following steps: S221. Obtain the current road surface smoothness index and the parasitic inductance change rate of the movable platform, and determine the first weight corresponding to the hand grip probability based on the road surface smoothness index and the parasitic inductance change rate.

[0099] In this embodiment, a first weight α(t) can be defined based on the hand grip probability from the sensor, to characterize the first weight at time t. This embodiment can calculate the first weight α(t) by obtaining the road surface smoothness index and the parasitic inductance change rate of the movable platform at the current time t.

[0100] In an exemplary implementation, α(t) can be calculated in the following manner: (5) Here, R(t) represents the road surface smoothness index at the current moment, which can be calculated using the accelerometer of the movable platform or the suspension travel. The larger R(t) is, the bumpier the road surface and the greater the capacitance signal noise. In this case, the weight of the sensor-based hand grip probability can be reduced, while the weight of the vision-based hand grip probability can be increased.

[0101] L(t) represents the rate of change of parasitic inductance of the mobile platform, which can be obtained through high-frequency signal monitoring as described above, and is used to characterize the degree of wiring harness interference. For example, L(t) can be calculated by comparing the imaginary part of the impedance at a measured high-frequency point (e.g., 500kHz) with the nominal value of the lower line.

[0102] W1 and W2 are coefficients of R(t) and L(t), respectively, used to characterize the attenuation strength of the first weight of the sensor-based hand grip probability by bumps and electromagnetic interference. W1 and W2 are constants; for example, W1 can be -1.2 and W2 can be -0.5.

[0103] Therefore, the embodiments of this application fully consider the impact of environmental changes on sensor-based hand grip probability, and can determine the weight of sensor-based hand grip probability in the entire hand grip evaluation score based on the current road surface smoothness index and parasitic inductance change rate, thereby helping to obtain a more accurate hand grip evaluation score.

[0104] S222. Obtain the current illumination data and determine the second weight corresponding to the vision-based hand grip probability based on the illumination data.

[0105] In this embodiment, a second weight β(t) corresponding to the visual grip probability can be defined to characterize the second weight at time t. This embodiment can calculate the second weight β(t) based on the illumination data obtained at the current time t.

[0106] It is understandable that illumination data can include the ambient light intensity Lux and the cabin illumination intensity (which can also be represented by occlusion degree Occ), thus allowing β(t) to be determined by combining the ambient light intensity Lux and the occlusion degree Occ. The ambient light intensity Lux can be obtained by the intelligent driving domain controller through sensors, while the cabin illumination intensity or occlusion degree Occ can be obtained by the DMS controller through cameras.

[0107] In an exemplary implementation, β(t) can be calculated in the following manner: (6) Therefore, the embodiments of this application fully consider the impact of changes in illumination on vision-based hand grip probability, and can determine the weight of vision-based hand grip probability in the entire hand grip evaluation score based on the current illumination data, thereby helping to obtain a more accurate hand grip evaluation score.

[0108] S223. Obtain the angular acceleration of the steering wheel and determine the third weight corresponding to the hand grip probability based on the angular acceleration.

[0109] In this embodiment, a third weight γ(t) corresponding to the dynamic grip probability can be defined to characterize the third weight at time t. This embodiment can calculate the third weight γ(t) by obtaining the angular acceleration of the steering wheel at the current time t.

[0110] In this embodiment, the angular acceleration of the steering wheel can be understood as the measured angular acceleration of the steering wheel, which can be obtained by superimposing the driver's active steering and road surface excitation when driving the mobile platform. Specifically, the angular acceleration of the steering wheel can be obtained by separating the driver's active component from the measured signal using a high-pass filter or a Kalman filter and taking the difference.

[0111] In an exemplary implementation, γ(t) can be calculated in the following manner: (7) in, This represents the measured angular acceleration of the steering wheel, when... When the value is large, the image captured by the camera may have motion blur, and the dynamic hand grip probability output by the visual DMS model may not be very accurate. Therefore, the embodiments of this application can reduce the third weight γ(t) of the dynamic hand grip probability.

[0112] It is understood that steps S221, S222 and S223 can be executed simultaneously or in a set order, and the embodiments of this application are not limited in this respect.

[0113] Therefore, the embodiments of this application fully consider the influence of the relevant parameters of the movable platform on the dynamic hand grip probability during the steering process. The weight of the dynamic hand grip probability in the entire hand grip evaluation score can be determined based on the measured angular acceleration of the steering wheel, which is conducive to obtaining a more accurate hand grip evaluation score.

[0114] Furthermore, after the detection device calculates α(t), β(t), and γ(t) based on the external environment information, it can normalize the three values ​​to obtain the values ​​of α(t), β(t), and γ(t) under different scenarios.

[0115] For example, in a highway or smooth road surface scenario, α(t), β(t), and γ(t) can be 0.28, 0.36, and 0.36, respectively; in a bumpy road surface scenario, α(t), β(t), and γ(t) can be 0.38, 0.55, and 0.07, respectively; in a nighttime scenario or a scenario with light obstruction (such as a tunnel), α(t), β(t), and γ(t) can be 0.28, 0.38, and 0.34, respectively; in a scenario with strong backlight (or visual impairment), α(t), β(t), and γ(t) can be 0.55, 0.1, and 0.35, respectively; and in a scenario with extreme bumpiness and strong backlight, α(t), β(t), and γ(t) can be 0.65, 0.25, and 0.1, respectively.

[0116] In this embodiment, the impact of different environments or scenarios on the accuracy of sensor-related parameters, vision-related parameters, and dynamic parameters is fully considered. Therefore, the detection device can determine the weights of the hand grip probability based on the sensor, the hand grip probability based on vision, and the hand grip probability based on dynamics based on external environment information, thereby obtaining a more accurate hand grip evaluation score and improving the accuracy of hand grip detection.

[0117] S23. Use weights and steering wheel grip data to determine the user's grip assessment score.

[0118] In this embodiment, the user's hand grip assessment score represents the probability that the user is holding the steering wheel while driving; specifically, it can be a score or a percentage. In other specific embodiments, the hand grip assessment score can be understood as a comprehensive hand grip confidence level.

[0119] In this embodiment of the application, after the detection device (such as the intelligent driving domain controller of the movable platform) obtains the weight and steering wheel grip data, it can use the weight to adjust the steering wheel grip data, thereby obtaining a more accurate grip evaluation score. Then the detection device can more accurately judge whether the user has taken off the wheel during driving.

[0120] In some specific implementations, steering wheel grip data includes sensor-based grip probability f. MLS Vision-based hand grip probability f vision and the dynamic-based hand grip probability f dynamics In this case, the specific method by which the detection device uses weights and steering wheel grip data to determine the user's grip assessment score may include the following steps: S231. The hand-holding probability based on the sensor is weighted according to the first weight to obtain the first hand-holding probability.

[0121] S232. The vision-based hand grip probability is weighted based on the second weight to obtain the second hand grip probability.

[0122] S233. The dynamic hand-holding probability is weighted based on the third weight to obtain the third hand-holding probability.

[0123] S234. Determine the user's hand grip evaluation score based on the first hand grip probability, the second hand grip probability, and the third hand grip probability.

[0124] In this embodiment of the application, the first weight α(t), the second weight β(t), and the third weight γ(t) are used to respectively adjust f. MLS f vision and f dynamics Weighted calculations are performed to obtain the first hand grip probability, the second hand grip probability, and the third hand grip probability, respectively. The three probabilities are then summed to obtain the user's hand grip evaluation score.

[0125] It is understood that the execution order of steps S231-S232 is not limited in the embodiments of this application.

[0126] In an exemplary implementation, a dynamic feature weighted fusion model based on an attention mechanism (hereinafter referred to as Model 2) is pre-established. Model 2 processes the weights and steering wheel grip data to output a user grip evaluation score. Specifically, Model 2 can be: (8) Among them, S HOD (t) represents the hand grip assessment score at time t, a continuous value between 0 and 1, S HOD (t)=0 indicates that the user has completely let go of their hands, S HOD (t)=1 indicates that the user is gripping the steering wheel tightly.

[0127] In this embodiment, by acquiring external environmental information of the mobile platform and determining the weight of steering wheel grip data based on this information, the steering wheel grip data is adjusted accordingly. This reduces the impact of environmental or driving scenario changes on the detected user data, resulting in a more accurate grip assessment score and improved grip detection precision. Furthermore, this method reduces missed detections when the user removes their hands, thus improving driving safety, and also reduces unnecessary false alarms, enhancing the driving experience.

[0128] S24. Compare the hand grip assessment score with the target threshold, and determine whether the user should let go of the hand based on the comparison result.

[0129] In this embodiment, the target threshold can be a pre-set value, a fixed value calculated based on the user's driving habits, or a dynamically updated value. Specifically, it can be updated based on the driving data of each driving session. This embodiment does not limit this.

[0130] In this embodiment of the application, after obtaining the hand grip assessment score, the detection device (such as the intelligent driving domain controller) can determine whether the user has let go of the hand by comparing the hand grip assessment score with the target threshold.

[0131] In some implementations, if the hand grip assessment score is greater than or equal to a target threshold, it indicates that the steering wheel is in a hand grip state; if the hand grip assessment score is less than the target threshold, it is determined that the user has removed their hands, and the chassis domain controller, cockpit domain controller, and intelligent driving domain controller can output corresponding reminders or alarms to remind the user that they have removed their hands. Alternatively, if the hand grip assessment score is less than the target threshold and assisted driving is currently suitable, the intelligent driving domain controller can perform intelligent driving; this application embodiment does not limit this.

[0132] In other implementations, the detection device can issue tiered alarms when the user leaves the device, for example, by issuing tiered alarms in the manner of instrument prompts → voice + vibration → ADAS downgrade / takeover request.

[0133] For example, alarms can be tiered based on the duration of user hand removal. When user hand removal is immediately detected, an alarm can be triggered via instrument notification. If user hand removal is detected continuously within 30 seconds, an alarm can be triggered via voice and vibration. If user hand removal is detected continuously within 1 minute, an alarm can be triggered via ADAS degradation / takeover request. Of course, the criteria for classifying the duration of hand removal in the tiered alarm system can be set as needed, and the alarm methods for each stage can also be set as required; this embodiment does not limit these aspects.

[0134] For example, alarms can also be tiered based on the difference between the hand grip assessment score and the target threshold. For instance, if the target threshold is 10, an alarm might be triggered via instrument prompts if the hand grip assessment score is 9.5, via voice and vibration if the score is 8, and via ADAS downgrade / takeover request if the score is 6. Of course, the criteria for tiered alarms based on the difference between the hand grip assessment score and the target threshold can be set as needed, and this embodiment does not limit this.

[0135] In this embodiment, when detecting hand grip on the steering wheel, the probability of the user's hand grip is determined from the perspectives of sensors, vision, and dynamics. The differences caused by environmental factors are taken into account, and the weights of each hand grip probability are dynamically adjusted using external environmental information. This allows for a more accurate determination of whether the user has taken their hand off the steering wheel, thereby improving the accuracy of steering wheel hand grip detection and reducing false alarm and false alarm rates. This, in turn, improves driving safety and enhances the driving experience.

[0136] This application also provides a method for detecting hand grip on a steering wheel. Please refer to [link / reference]. Figure 3 , Figure 3 This is a flowchart illustrating another steering wheel hand grip detection method provided in an embodiment of this application, as shown below. Figure 3 As shown, the method includes the following steps.

[0137] S31. Identify the user and obtain the user's historical driving data with the steering wheel based on the identification results.

[0138] In this embodiment of the application, when a user is driving a mobile platform, the user's identity can be identified through one or more methods such as facial recognition, fingerprint verification, and iris recognition, thereby determining the user's identity information.

[0139] In some implementations, the DMS controller can collect one or more of the user's facial image, facial video, and pupil data, and send them to the cockpit domain controller, which then identifies the user based on the data. In other implementations, the steering wheel ECU can collect the user's fingerprint information and send it to the cockpit domain controller, which then identifies the user based on the data. This application does not limit the implementation to these specific implementations.

[0140] In this embodiment, different users have different driving data. Therefore, the detection device can locally store the historical driving data corresponding to multiple users who have driven the mobile platform on the mobile platform. In some implementations, the detection device can upload the driving data of each user driving the mobile platform to a cloud service platform, where the cloud service platform stores the identity information of each user and the corresponding historical driving data. This embodiment does not limit this approach.

[0141] It is understood that historical driving data is used to characterize the user's driving habits, such as historical contact capacitance, historical average contact capacitance, historical grip strength, and historical grip strength standard deviation, etc., but this application embodiment does not limit this.

[0142] Specifically, the detection device can obtain the user's historical driving data based on the identity recognition result in the following ways: if the user's identity recognition result is that of a registered user, the historical driving data corresponding to the user can be obtained directly from the local or cloud service platform; if the user is a new user, the user's driving data for a period of time before driving (e.g., the first 30 seconds) can be obtained as the user's historical driving data. This application embodiment does not limit this.

[0143] It is understandable that a new user can be understood as someone who has never driven this mobile platform, or a mobile platform of the same brand, or whose historical driving data is not stored on the cloud service platform or locally.

[0144] In some specific implementations, historical driving data may include historical average contact capacitance. The specific method by which the detection device obtains the user's historical driving data with the steering wheel based on the identity recognition result may include the following steps: S311. If the identity recognition result indicates that the user is a new user, then predict the user's basic physiological data.

[0145] In this embodiment, basic physiological data may include height, weight, and gender. Specifically, the user's height can be predicted by combining video and images captured by in-cabin cameras with the distance between the user's head and the roof, as well as seat adjustment data. The user's weight can also be predicted using gravity information collected by seat gravity sensors. In other embodiments, the user's basic physiological data can be predicted in other ways, which are not limited in this embodiment.

[0146] In this embodiment of the application, when identifying a user and determining that the user is a new user, the user's height, weight, etc. can be predicted by collecting relevant information about the user.

[0147] S312. Detect the user's posture information and the user's grip strength on the steering wheel within a preset time period.

[0148] In this embodiment of the application, the preset time period can be understood as a period of time after the mobile platform is started. For example, it can be the first 30 seconds after startup, the first minute after startup, etc. This embodiment of the application does not limit the duration of the preset time period.

[0149] It is understood that the user's posture information can be used to characterize the user's habitual sitting posture. Specifically, it can be detected by a camera or the seat's gravity sensor or pressure sensor. For example, the gravity sensor can detect the position of the seat under force, and the pressure sensor can detect the position of other contact points between the user and the seat, etc. This application embodiment does not limit this.

[0150] In this embodiment, the steering wheel may have a built-in grip force sensor that can detect the user's grip force on the steering wheel in real time during driving, measured in Newtons (N). Therefore, the detection device can detect the user's grip force over a preset time period and record the average value. For example, the grip force is generally in the range of [5N, 50N].

[0151] S313. Determine the user's initial capacitance parameters based on basic physiological data, posture information, and grip strength, and use the initial capacitance parameters as the historical average contact capacitance between the user and the steering wheel.

[0152] In this embodiment, different users have completely different habitual grip strength, habitual position, and silent capacitance value when holding the steering wheel. Therefore, when the user is a new user, the detection device can determine the initial capacitance parameter that is suitable for the user based on the user's basic physiological data, posture information, and grip strength. This initial capacitance parameter can then be used as the historical average contact capacitance between the user and the steering wheel, so as to set a personalized threshold for the user.

[0153] In an exemplary implementation, the initial capacitance parameter is defined as C. baseline Therefore, in the case of a new user, C baseline This can be determined in the following ways: (9) Where, μ pop The average reference capacitance for people of the same vehicle model, specifically, μ pop This could be achieved by obtaining the historical average C of people using the same car model. contact It can be obtained by averaging; or by obtaining the historical average C of people in the same car model group who share one or more of the same gender, height, weight, etc. as the user. contact The average value is obtained by averaging the values, but this embodiment does not limit the scope of the application.

[0154] Height represents the user's height information, Weight represents the user's weight information, and Style... init This provides posture information and grip strength for a preset time period after the mobile platform is started.

[0155] ε posture This represents the deviation of the reference capacitance caused by the user's actual sitting posture compared to the standard sitting posture during driving. Specifically, ε posture The seat's fore-and-aft position, backrest angle, and torso offset can be measured using a camera, and the correction value is calculated using a regression model. In other words, ε... posture It can be calculated based on the acquired user's posture information and standard posture information. For example, ε posture It is generally in the range of [-10pF, 10pF].

[0156] λ1 is the contribution coefficient of height to the reference capacitance, which is a constant and is expressed in pF / cm. The higher the height, the more the equivalent arm length changes, and the slightly different the parasitic parameters become. For example, λ1 can be in the range of [0.1pF / cm, 0.3pF / cm].

[0157] λ2 is the contribution coefficient of body weight to the reference capacitance, which is a constant and is expressed in pF / kg. Body weight affects the equivalent ground capacitance of the torso. For example, λ2 can be in the range of [0.05 pF / kg, 0.15 pF / kg].

[0158] λ3 is the contribution coefficient of the initial grip style, which is a constant and represents the grip habits collected within 30 seconds after the mobile platform starts. For example, λ3 can be in the range of [0.2, 0.5].

[0159] In this embodiment of the application, when a user is identified as a new user, the initial capacitance parameter belonging to the user can be determined by detecting the user's basic physiological data, posture information and grip strength, etc., and the initial capacitance parameter can be used as the user's historical average contact capacitance.

[0160] Understandably, during subsequent driving, the detection device acquires the user's contact capacitance C output by Model 1. contact Using C baseline For C contact Perform corrections to accumulate the corrected C. contact The average value is then calculated and used as the historical average contact capacitance.

[0161] Understandably, after updating the historical average contact capacitance, the obtained historical average contact capacitance can be applied to the method for determining the target threshold in the next hand grip detection. Furthermore, the updated historical average contact capacitance serves as the new C.baseline The application will be used in the next determination of the sensor-based hand grip probability f. MLS In the function.

[0162] In this embodiment, the historical average contact capacitance can be updated after each hand grip detection or after each driving session; this embodiment does not limit the update in this way.

[0163] In this embodiment, the historical average contact capacitance can be dynamically updated as the user's driving habits change. Therefore, the target threshold determined by this method of dynamically updating the historical average contact capacitance is consistent with the user's driving habits over a period of time. Thus, the accuracy of the hand-holding evaluation score can be further improved, thereby reducing unnecessary hands-off warnings and further enhancing the user's driving experience.

[0164] In other specific implementations, historical driving data may also include the user's historical grip force standard deviation, used to determine whether the user is a "strong gripper" or a "gentle gripper." Introducing this parameter allows for adjustment of the alarm threshold based on the user's habits. Specifically, during driving, a grip force sensor can detect and record the grip force value each time the user grips the steering wheel, and the value can be calculated after accumulating N samples.

[0165] It is understood that the historical grip strength standard deviation can be updated after each grip test or after each driving session, and this application embodiment does not limit this.

[0166] In some other specific implementations, historical driving data may also include the user’s historical average grip strength, thereby determining the target threshold based on historical average contact capacitance, historical grip strength standard deviation, and historical average grip strength.

[0167] S32. Obtain the user's current status information.

[0168] In this embodiment, the user's current state information is used to characterize the user's alertness during driving. For example, when the user is fatigued, the system increases detection sensitivity and does not easily determine that the hand has left the driver's grip even if the grip strength is very small, thus avoiding false alarms when the driver is unconsciously relaxing.

[0169] Specifically, the current state information can be collected by the DMS controller to determine whether the user is currently fatigued, such as by measuring the user's eyelid closure degree and blink interval. Of course, this application embodiment can also determine the user's current state information through other methods, and this application embodiment is not limited to these methods.

[0170] S33. Determine the user's target threshold based on historical driving data and current status information.

[0171] In this embodiment, different users have completely different habitual grip strength, habitual position, and silent capacitance value when holding the steering wheel. Therefore, after obtaining the user's historical driving data and current status information, a target threshold specific to that user can be determined, thereby achieving "personalized" hand grip detection. The detection results are more accurate, effectively improving detection accuracy, reducing the possibility of false alarms and missed alarms when the hands are off, and greatly improving driving safety and driving experience.

[0172] In an exemplary implementation, historical driving data includes historical average contact capacitance and historical grip force standard deviation. The user's target threshold can be calculated based on historical driving data and current state information in the following manner: (10) Among them, Th HOD (User, State) represents the current target threshold for this user. The historical average contact capacitance for the user; if the user is a new user, The value is the initial capacitance parameter C. baseline Std(F) User ) represents the user's historical grip strength standard deviation, and Alertness represents the user's current status information. k1, k2, and k3 are constants.

[0173] Understandably, k1 represents the historical average contact capacitance. The weight of k1 reflects the typical grip strength of the user. For example, k1 can be in the range of [0.3, 0.7]. k2 is the weight of the historical grip strength standard deviation, reflecting the fluctuation of the user's grip. The more "unpredictable" the user is, the higher the target threshold should be to avoid false alarms. For example, k2 can be in the range of [0.1, 0.3]. k3 is the sensitivity compensation coefficient, reflecting the degree to which the threshold is reduced (sensitivity is increased) under fatigue. For example, k3 can be in the range of [0.2, 0.5].

[0174] S34. Obtain the user's steering wheel grip data.

[0175] S35. Obtain external environment information of the mobile platform and determine the weight of steering wheel grip data based on the external environment information.

[0176] S36. Use weights and steering wheel grip data to determine the user's grip assessment score.

[0177] S37. Compare the hand grip assessment score with the target threshold, and determine whether the user should let go of the hand based on the comparison result.

[0178] It is understood that steps S34-S37 can refer to the relevant descriptions of steps S21-S24 in the foregoing embodiments, and will not be repeated in this application embodiment.

[0179] In some implementations, after each hand-grip detection, the detection device updates the user's driving data to the cloud service platform, thereby optimizing historical driving data. For example, it updates the contact capacitance C obtained from the current hand-grip detection. contact Data such as grip strength is uploaded to a cloud service platform, which then updates its stored historical average contact capacitance and historical grip strength standard deviation based on the uploaded data. For example, the contact capacitance C obtained from this hand grip test... contact The system updates the historical average contact capacitance and historical grip force standard deviation stored locally, and then synchronizes the updated data to the cloud service platform.

[0180] In other implementations, after each hand grip test, the detection device can remove driving data under abnormal conditions and upload the removed driving data to a cloud service platform.

[0181] For example, when a user suddenly exceeds twice the standard deviation of grip strength and the duration is less than 1 second, the detection device can identify it as an abnormal point of sudden steering. After removing the abnormal point, the device can calculate the user's new average grip strength and new standard deviation of grip strength, and update the historical driving data through OTA to the cloud service platform.

[0182] It should be noted that the order of steps (S31-S33) and steps S34-S37 in determining the target threshold in this embodiment is not limited to this embodiment. In other possible implementations, steps S31-S33 can be executed at any time before step S37 compares the hand grip evaluation score with the target threshold. This embodiment does not limit this.

[0183] In this embodiment of the application, different users have different driving habits. This embodiment of the application adaptively adjusts the user's current target threshold based on the user's historical driving data and current status information, so as to more accurately match the user's current status. Furthermore, by dynamically updating the target threshold, it can determine whether the user has taken their hands off the wheel during driving, which can obtain more accurate judgment results, reduce the probability of false alarms and missed alarms, and thus improve driving safety and driving experience.

[0184] Furthermore, during driving, the user's hand grip assessment score is determined based on at least two of the following: sensor data, visual data, and dynamic data. Particularly when determining the sensor-based hand grip probability, the impact of environmental changes and user emotional changes on the user's hand humidity and conductivity is comprehensively considered, resulting in a more accurate hand grip probability. Moreover, when determining the user's hand grip assessment score, the weighting coefficients of each hand grip probability are dynamically adjusted in conjunction with environmental changes, further improving the accuracy of the hand grip assessment score. Additionally, by dynamically adjusting the user's target threshold based on the user's current state and historical driving data, and using this adaptively adjusted target threshold to determine whether the user has released their hand, more accurate judgment results are obtained, thereby reducing the probability of false alarms and missed alarms related to hand release, and ultimately improving driving safety and driving experience.

[0185] For example, please refer to Figure 4 , Figure 4 An exemplary flowchart of a steering wheel hand grip detection method is provided, which is applied to a steering wheel hand grip detection system (hereinafter referred to as the detection system), the detection system including a detection device.

[0186] like Figure 4 As shown, when a mobile platform (such as a vehicle) starts, the steering wheel ECU and various controllers in the data acquisition layer collect driving-related data in real time. This includes internal active data (steering wheel pressure, humidity of the user's skin in contact with the steering wheel, the point and time of contact with the steering wheel, etc.), internal monitoring data (human posture - height, distance, tilt angle, deviation, etc.; driving state - line-of-sight deviation, torso offset, blink interval, etc.), and external monitoring data (mobile platform speed, acceleration / deceleration, lane-changing trends, road surface undulation trends, weather, road conditions, environment, etc.). The detection system then extracts features based on the collected data, such as multi-frequency impedance features, visual features (hand key points, head posture, eyelid closure, etc.), and dynamic features (EPS torque, steering angle, road surface smoothness index, etc.). Simultaneously, the detection system identifies the user through facial recognition or fingerprint recognition. If the user is identified as a registered user (with existing driving data), historical driving data is retrieved. If the user is a new user, an initialization learning mode is initiated, collecting data from the 30 seconds prior to vehicle startup. It then enters the real-time processing flow.

[0187] In the real-time processing flow, the detection system inputs the feature data mentioned above into the response model. For example, it inputs multi-frequency impedance features (impedance information, skin surface humidity, conductivity, wiring harness parasitic inductance, leakage resistance, etc.) into Model 1 to perform multi-frequency impedance decoupling, thereby outputting the contact resistance R between the user and the steering wheel. skin and contact capacitance C contactAnd calculate the sensor-based hand grip detection probability f MLS For example, visual features are input into a visual model to output a vision-based hand grip probability f. vision And inputting dynamic features into a dynamic inverse model to output a dynamically based hand grip probability f dynamics .

[0188] Furthermore, the detection system determines f based on the current external ring information. MLS f vision and f dynamics The corresponding dynamic weights are used to dynamically weight and fuse the three hand grip probabilities using an attention mechanism to obtain the user's hand grip evaluation score S. HOD .

[0189] At the same time, the detection system determines the user's personalized threshold Th based on historical driving data and the user's current status information. HOD and for S HOD and Th HOD The results of the hand grip detection are obtained by comparison. If S HOD ≥Th HOD If the condition is met, output that the user is in a holding state; otherwise, output that the user is in a hands-free state.

[0190] Furthermore, the ADAS system responds based on the hand-grip detection results, for example, issuing tiered warnings when it detects the user removing their hands. After outputting the hand-grip detection results, the system records relevant driving data and uploads it to a cloud service platform. The cloud service platform updates the user's feature database in real time to optimize the user's historical driving data, and then repeats the cyclical processing flow, performing hand-grip detection in real time.

[0191] This application also provides a steering wheel hand grip detection device; please refer to [link / reference]. Figure 5 , Figure 5 This is a schematic diagram of the structure of a steering wheel hand grip detection device provided in an embodiment of this application, as shown below. Figure 5 As shown, the detection device 500 is applied in a steering wheel hand grip detection system. Specifically, the detection device includes an acquisition module 501, a determination module 502, and a comparison module 503, wherein: The acquisition module 501 is used to acquire the user's steering wheel grip data, which includes at least two of the following: sensor-based grip probability, vision-based grip probability, and dynamic grip probability.

[0192] The acquisition module 501 is also used to acquire external environment information of the mobile platform.

[0193] The determination module 502 is used to determine the weight of the steering wheel grip data based on external environment information.

[0194] The determining module 502 is also used to determine the user's hand grip assessment score using weights and steering wheel grip data.

[0195] Comparison module 503 is used to compare the hand grip assessment score with the target threshold.

[0196] The determination module 502 is also used to determine whether the user has let go of the device based on the comparison result.

[0197] In this embodiment of the application, when the detection device 500 detects hand grip on the steering wheel, it determines the probability of the user's hand grip from different angles and takes into account the differences caused by environmental factors. It dynamically adjusts the weight of each hand grip probability using external environmental information, which can more accurately determine whether the user has taken off their hand, thereby improving the accuracy of steering wheel hand grip detection.

[0198] In some feasible implementations, the steering wheel grip data includes sensor-based grip probabilities. The specific method by which the acquisition module 501 acquires the user's steering wheel grip data can be as follows: The system acquires the contact resistance between the user and the steering wheel; detects the impedance information of the steering wheel; determines the contact capacitance between the user and the steering wheel based on the contact resistance and impedance information; and determines the sensor-based hand grip probability based on the contact resistance and contact capacitance.

[0199] In some feasible implementations, the specific method by which the acquisition module 501 acquires the contact resistance between the user and the steering wheel can be: The system detects the contact area between the user and the steering wheel; obtains the skin parameters of the user in contact with the steering wheel; and determines the contact resistance between the user and the steering wheel based on the contact area and skin parameters.

[0200] In some feasible implementations, the steering wheel grip data includes sensor-based grip probability, vision-based grip probability, and dynamic grip probability. The specific method by which the acquisition module 501 acquires the external environment information of the mobile platform is as follows: Obtain the current road surface smoothness index, the parasitic inductance change rate of the movable platform, the current illumination data, and the angular acceleration of the steering wheel.

[0201] The specific method by which the determining module 502 determines the weight of the steering wheel grip data based on external environment information is as follows: The first weight corresponding to the hand grip probability based on the acquired road surface smoothness index and parasitic inductance change rate is determined; the second weight corresponding to the hand grip probability based on vision is determined based on the acquired illumination data; and the third weight corresponding to the hand grip probability based on dynamics is determined based on angular acceleration.

[0202] In some feasible implementations, the specific method by which the determining module 502 uses weights and steering wheel grip data to determine the user's grip assessment score can be as follows: The sensor-based hand-holding probability is weighted based on a first weight to obtain a first hand-holding probability; the vision-based hand-holding probability is weighted based on a second weight to obtain a second hand-holding probability; and the dynamic hand-holding probability is weighted based on a third weight to obtain a third hand-holding probability. The user's hand grip evaluation score is determined based on the first hand grip probability, the second hand grip probability, and the third hand grip probability.

[0203] In some feasible implementations, the detection device 500 may further include an identity recognition module 504, wherein: The identity recognition module 504 is used to identify the user.

[0204] The acquisition module 501 is also used to acquire the user's historical driving data with the steering wheel based on the identity recognition result of the identity recognition module 504, as well as to acquire the user's current status information.

[0205] The determining module 502 is also used to determine the user's target threshold based on the historical driving data and current status information obtained by the acquiring module 501.

[0206] In some feasible implementations, historical driving data includes historical average contact capacitance. The specific method by which the acquisition module 501 acquires the user's historical driving data with the steering wheel based on the identity recognition result can be as follows: If the identity recognition module 504 determines that the user is a new user, it predicts the user's basic physiological data. It detects the user's posture information and grip strength on the steering wheel within a preset time period. Based on the basic physiological data, posture information, and grip strength, it determines the user's initial capacitance parameters and uses these parameters as the historical average contact capacitance between the user and the steering wheel.

[0207] The steering wheel hand grip detection device in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0208] The steering wheel hand grip detection device in this embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this embodiment does not specifically limit it.

[0209] The steering wheel hand grip detection device provided in this application embodiment can achieve... Figures 2 to 4 The various processes implemented by the steering wheel hand grip detection device in the method embodiment will not be described again here to avoid repetition.

[0210] In this embodiment, during driving, the detection device determines the user's hand grip assessment score based on at least two of sensor data, visual data, and dynamic data. Specifically, when determining the sensor-based hand grip probability, it comprehensively considers the impact of environmental changes and user emotional changes on the user's hand humidity and conductivity, thereby obtaining a more accurate hand grip probability. Furthermore, when determining the user's hand grip assessment score, the detection device dynamically adjusts the weighting coefficients of each hand grip probability in conjunction with environmental changes, further improving the accuracy of the hand grip assessment score. Additionally, the detection device dynamically adjusts the user's target threshold based on the user's current state and historical driving data. By adaptively adjusting the target threshold to determine whether the user has released their hand, it obtains more accurate judgment results, thereby reducing the probability of false alarms and missed alarms related to hand release, and ultimately improving driving safety and driving experience.

[0211] This application also provides an electronic device 600, including a processor 601, a memory 602, and a program or instructions stored in the memory 602 and executable on the processor 601. When the program or instructions are executed by the processor 601, they implement the various processes of the above-described steering wheel hand grip detection method embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0212] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above. This application also provides a mobile platform, which may be, for example, a vehicle, including a steering wheel and the electronic equipment provided in the foregoing embodiments, or capable of implementing the various processes of the above-described steering wheel hand grip detection method embodiments and achieving the same technical effect. To avoid repetition, it will not be described again here.

[0213] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described steering wheel hand grip detection method embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0214] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0215] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steering wheel hand grip detection method provided in the various embodiments described above.

[0216] It should be noted that, in the specific embodiments of this application, all user-related data (including images) and the processing of user data (including facial recognition, fingerprint recognition, etc.) involved in this application must obtain the user's authorization or consent when the above embodiments of this application are applied to specific products or technologies, and the collection, use and processing of related data must comply with relevant laws and standards.

[0217] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0218] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0219] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for detecting hand grip on a steering wheel, characterized in that, The method includes: Acquire user steering wheel grip data, which includes at least two of the following: sensor-based grip probability, vision-based grip probability, and dynamic grip probability; Obtain external environment information of the mobile platform, and determine the weight of the steering wheel grip data based on the external environment information; The user's hand grip assessment score is determined using the weights and the steering wheel grip data. The hand grip assessment score is compared with a target threshold, and the user is determined to have let go of the hand based on the comparison result.

2. The method according to claim 1, characterized in that, The steering wheel grip data includes sensor-based grip probability, and acquiring the user's steering wheel grip data includes: Obtain the contact resistance between the user and the steering wheel; Detect the impedance information of the steering wheel; The contact capacitance between the user and the steering wheel is determined based on the contact resistance and the impedance information; The sensor-based hand grip probability is determined based on the contact resistance and the contact capacitance.

3. The method according to claim 2, characterized in that, The step of obtaining the contact resistance between the user and the steering wheel includes: Detect the contact area between the user and the steering wheel; Obtain the skin parameters of the user in contact with the steering wheel; The contact resistance between the user and the steering wheel is determined based on the contact area and the skin parameters.

4. The method according to claim 1, characterized in that, The steering wheel grip data includes sensor-based grip probability, vision-based grip probability, and dynamic grip probability. Acquiring external environment information of the mobile platform and determining the weights of the steering wheel grip data based on this information includes: Obtain the current road surface smoothness index and the parasitic inductance change rate of the mobile platform, and determine the first weight corresponding to the sensor-based hand grip probability based on the road surface smoothness index and the parasitic inductance change rate. Acquire the current illumination data, and determine the second weight corresponding to the vision-based hand grip probability based on the illumination data; The angular acceleration of the steering wheel is obtained, and the third weight corresponding to the dynamic hand grip probability is determined based on the angular acceleration.

5. The method according to claim 4, characterized in that, The process of determining the user's hand grip assessment score using the weights and the steering wheel grip data includes: The sensor-based hand-holding probability is weighted based on the first weight to obtain the first hand-holding probability. The vision-based hand-holding probability is weighted based on the second weight to obtain the second hand-holding probability. The third hand grip probability is obtained by weighting the dynamic hand grip probability based on the third weight. The user's hand grip evaluation score is determined based on the first hand grip probability, the second hand grip probability, and the third hand grip probability.

6. The method according to any one of claims 1-5, characterized in that, Before comparing the hand grip assessment score with the target threshold, the method further includes: The user is identified, and historical driving data of the user and the steering wheel are obtained based on the identification result. Obtain the user's current status information; The target threshold for the user is determined based on the historical driving data and the current status information.

7. The method according to claim 6, characterized in that, The historical driving data includes historical average contact capacitance. Obtaining the user's historical driving data with the steering wheel based on the identity recognition result includes: If the identity recognition result indicates that the user is a new user, then predict the user's basic physiological data; Detect the user's posture information and the user's grip strength on the steering wheel within a preset time period; The user's initial capacitance parameters are determined based on the basic physiological data, the posture information, and the grip strength, and the initial capacitance parameters are used as the historical average contact capacitance between the user and the steering wheel.

8. A steering wheel hand grip detection device, characterized in that, The device includes: The acquisition module is used to acquire the user's steering wheel grip data, which includes at least two of the following: sensor-based grip probability, vision-based grip probability, and dynamic grip probability. The acquisition module is also used to acquire external environment information of the mobile platform; The determination module is used to determine the weight of the steering wheel grip data based on the external environment information; The determining module is also used to determine the user's hand grip evaluation score using the weights and the steering wheel grip data; The comparison module is used to compare the hand grip assessment score with the target threshold; The determining module is also used to determine whether the user has let go of the device based on the comparison result.

9. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the steering wheel hand grip detection method as described in claims 1-7.

10. A mobile platform, characterized in that, The mobile platform includes the electronic device as described in claim 9.

Images