A method and device for intervening in pedal misuse

By monitoring the accelerator pedal input characteristics and the driver's emotional state, combined with vehicle environmental data, the system accurately identifies pedal misuse and sends emergency braking commands, thus resolving the safety risks caused by driver pedal misuse. This enables vehicle safety intervention and multi-dimensional feedback, ensuring driving safety.

CN123104682APending Publication Date: 2026-11-17MERCEDES BENZ GRP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611293390.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-25
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

With advanced driver assistance systems, drivers may mistakenly press the accelerator pedal instead of the brake pedal due to nervousness or misjudgment, causing the vehicle to accelerate and lurch forward. The risk of accidents is high and increasing, and existing technologies are unable to accurately identify and intervene in such misuse.

Method used

By monitoring the accelerator pedal input characteristics, collecting driver facial images and quantifying emotional state, and combining this with vehicle environmental data, the system determines whether there are obstacles. If there are no obstacles, it sends an emergency braking command to stop the vehicle and provides display and voice warnings to prevent secondary misoperation.

Benefits of technology

It enables accurate identification and intervention of pedal misuse under advanced driver assistance systems, ensuring vehicle safety, preventing accidents, and providing multi-dimensional HMI feedback to ensure driver safety.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123104682A_ABST
    Figure CN123104682A_ABST
Patent Text Reader

Abstract

This invention discloses a method and device for intervening in pedal misuse, belonging to the field of intelligent driving technology. One specific implementation of the method includes: monitoring the input characteristics of the accelerator pedal when the advanced driver assistance system (ADAS) is activated; in response to the input characteristics exceeding a preset first threshold, acquiring a driver's facial image and inputting the facial image into a preset recognition model to obtain a quantitative value of the emotional state; in response to the emotional state quantification value exceeding a preset second threshold, detecting whether there are obstacles to the side and / or rear of the vehicle; and if there are no obstacles to the side and / or rear of the vehicle, sending an emergency braking command to the brake actuator to stop the vehicle. This method, through multimodal information fusion, solves the technical problem of inaccurate pedal misuse identification, greatly improving the technical effect of vehicle safety in intelligent driving scenarios.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology, and in particular to a method and device for intervening in pedal misuse. Background Technology

[0002] Currently, with the significant enhancement of vehicle power performance and the widespread adoption of Advanced Driver Assistance Systems (ADAS), drivers' feet are usually off the pedal area or in a relaxed state. In sudden emergencies, due to tension and misjudgment, drivers are prone to mistakenly pressing the accelerator pedal instead of the brake pedal. Once the accelerator pedal is mistakenly pressed, the vehicle will rush forward with extremely high acceleration, leaving the driver with very little time to react. Therefore, the risk of accidents caused by pedal misuse is extremely high and is increasing. Summary of the Invention

[0003] In view of this, the present invention provides a method and device for intervening in pedal misuse, which can solve the technical problem of not being able to accurately identify pedal misuse through multimodal information fusion, and greatly improve vehicle safety in intelligent driving scenarios.

[0004] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for intervening in pedal misuse, comprising: monitoring the input characteristics of the accelerator pedal when the advanced driver assistance system is activated; in response to the input characteristics being greater than a preset first threshold, acquiring a driver's facial image, inputting the facial image into a preset recognition model to obtain an emotional state quantification value; in response to the emotional state quantification value being greater than a preset second threshold, detecting whether there are obstacles to the side and / or rear of the vehicle; and, if there are no obstacles to the side and / or rear of the vehicle, sending an emergency braking command to a brake actuator to control the vehicle to stop.

[0005] Optionally, the monitoring of the input characteristics of the accelerator pedal includes: monitoring the pedal speed and pedal opening of the accelerator pedal sensor; The step of acquiring a driver's facial image in response to the input characteristic being greater than a preset first threshold includes: acquiring a driver's facial image in response to the pedal speed being greater than a preset speed threshold and the pedal opening being greater than a preset opening threshold.

[0006] Optionally, after determining that there are no obstacles to the side and / or rear of the vehicle, the method further includes: acquiring high-precision map data to detect whether there is a potential lateral collision area on the current driving segment; and, if there is no potential lateral collision area on the current driving segment, sending an emergency braking command to the brake actuator to control the vehicle to stop.

[0007] Optionally, after the condition that there are no obstacles to the side and / or rear of the vehicle, the method further includes: calculating the collision time between the vehicle and the following vehicle, and if the collision time is greater than a preset time threshold, sending an emergency braking command to the brake actuator to control the vehicle to stop.

[0008] Optionally, after detecting whether there are obstacles to the side and / or rear of the vehicle, the method further includes: if there are obstacles to the side and / or rear of the vehicle, detecting whether the vehicle has a deceleration control command based on an automatic emergency braking algorithm; if the vehicle has a deceleration control command based on an automatic emergency braking algorithm, sending an emergency braking command to the brake actuator to control the vehicle to stop.

[0009] Optionally, after sending an emergency braking command to the brake actuator, the method further includes controlling the accelerator pedal to output the maximum reaction force.

[0010] Optionally, after sending the emergency braking command to the brake actuator, the method further includes: generating display warning information and / or voice warning information according to a preset display warning template and / or a preset voice warning template, and outputting it to a display device and / or a voice playback device.

[0011] Optionally, sending the emergency braking command to the brake actuator includes: acquiring the braking deceleration of the vehicle, generating the emergency braking command, and sending it to the brake actuator.

[0012] Secondly, embodiments of the present invention provide a pedal misuse intervention device, comprising: a monitoring unit, a detection unit, and a processing unit, wherein the monitoring unit is used to monitor the input characteristics of the accelerator pedal when the advanced driver assistance system is activated; the detection unit is used to, in response to the input characteristics being greater than a preset first threshold, acquire a driver's facial image, input the facial image into a preset recognition model, and obtain an emotional state quantification value; in response to the emotional state quantification value being greater than a preset second threshold, detect whether there are obstacles to the side and / or rear of the vehicle; the processing unit is used to, when there are no obstacles to the side and / or rear of the vehicle, send an emergency braking command to the brake actuator to control the vehicle to stop.

[0013] Thirdly, embodiments of the present invention provide a vehicle including the pedal misuse intervention device described in the second aspect.

[0014] Fourthly, embodiments of the present invention provide a vehicle including a processor, a memory, and a display, wherein the processor is configured to acquire and execute code in the memory to perform the method provided in the first aspect embodiment.

[0015] Fifthly, embodiments of the present invention provide an in-vehicle electronic device for intervening in pedal misuse, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement a pedal misuse intervention method as described in the above embodiments of the present invention.

[0016] In a sixth aspect, embodiments of the present invention provide a computer-readable storage medium for intervening in pedal misuse, wherein a computer program for intervening in pedal misuse is stored thereon, and the computer program, when executed by an on-board processor, implements a method for intervening in pedal misuse according to embodiments of the present invention.

[0017] The technical solution of the above invention has the following advantages or beneficial effects: This invention breaks through the traditional reliance on pedal input for judgment. During the operation of the advanced driver assistance system, it continuously monitors the driver's input characteristics to the accelerator pedal, the driver's psychological state, and the vehicle's environmental safety data. Based on preset judgment conditions, it accurately identifies pedal misuse and immediately sends an emergency braking command to the brake actuator, thereby achieving multimodal and precise pedal misuse judgment and ensuring the safety of intervention. Furthermore, this invention achieves rapid and multi-dimensional HMI feedback by displaying warnings, providing voice warnings, and controlling the accelerator pedal to output maximum reaction force, preventing secondary driver misoperation. Simultaneously, after accurately identifying pedal misuse, this invention does not simply cut off the accelerator but directly executes emergency braking to stop the vehicle with the shortest distance, minimizing the risk of accidents. Moreover, before executing emergency braking, this invention can consider deceleration control commands that may be executed for obstacles in front of the vehicle and acceleration situations to prevent side collisions and rear-end collisions, thereby achieving precise pedal misuse intervention. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the main steps of the first pedal misuse intervention method provided according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the main steps of the second pedal misuse intervention method provided according to an embodiment of the present invention; Figure 3 This is a timing diagram of the pedal misuse intervention method provided according to an embodiment of the present invention; Figure 4 This is a schematic diagram of a pedal misuse intervention system provided according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the main unit of a pedal misuse intervention device according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a vehicle according to an embodiment of the present invention; Figure 7 This is an exemplary vehicle system architecture diagram to which embodiments of the present invention can be applied; Figure 8 This is a schematic diagram of the structure of a computer system suitable for implementing embodiments of the present invention. Detailed Implementation

[0019] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0020] It should be noted that, unless otherwise specified, the embodiments of the present invention and the technical features thereof can be combined with each other.

[0021] Furthermore, the terms "first," "second," and "third," etc., included in the terminology of this invention are used to distinguish similar objects and are not necessarily used to describe a specific number or order. It should be understood that such terms can be used interchangeably where appropriate; this is merely a distinguishing method used in the embodiments of this invention when describing objects with the same attributes.

[0022] Furthermore, the vehicles involved in the embodiments of the present invention may be internal combustion engine vehicles that use an engine as a power source, hybrid vehicles that use an engine and an electric motor as power sources, electric vehicles that use an electric motor as a power source, etc.

[0023] Figure 1 This is a schematic diagram illustrating the main steps of a first method for intervening in pedal misuse according to an embodiment of the present invention. Figure 1 As shown, this method for intervening in pedal misuse mainly includes the following steps: Step S101: With the advanced driver assistance system activated, monitor the input characteristics of the accelerator pedal.

[0024] In this embodiment of the invention, step S101 can determine the usage status of the vehicle's advanced driver assistance system. When the advanced driver assistance system (ADAS is an active safety technology based on sensor and navigation data that improves driving safety and comfort through monitoring, warning, and intervention) is in the activated state, the input characteristics of the accelerator pedal are monitored as the first mode in the multimodal method for determining pedal misuse. The input characteristics of the accelerator pedal may include, but are not limited to, pedal speed and pedal opening. Therefore, the input characteristics of the accelerator pedal sensor, namely pedal speed and pedal opening, can be monitored.

[0025] Step S102: In response to the input characteristic being greater than a preset first threshold, a driver's facial image is acquired, and the facial image is input into a preset recognition model to obtain a quantitative value of the emotional state.

[0026] In this embodiment of the invention, the first threshold can be set with different thresholds for different input characteristics of the accelerator pedal. For example, the input characteristics of the accelerator pedal may include pedal speed and pedal opening, and the first threshold may include a speed threshold and an opening threshold. In a further embodiment, responding to the input characteristics being greater than a preset first threshold may include responding to the pedal speed being greater than a preset speed threshold and the pedal opening being greater than a preset opening threshold.

[0027] As an embodiment of the present invention, step S102 can acquire a driver's facial image to obtain a quantitative value of the emotional state (as the second modality in the multimodal judgment of pedal misuse), thereby realizing the monitoring of the driver's psychological state (e.g., tension, panic, surprise, etc.). Preferably, a Driver Monitoring System (DMS) can be established to store the quantitative value of the emotional state, which can serve as a necessary condition for a high-confidence judgment of pedal misuse. The specific implementation process may include: acquiring a driver's facial image (e.g., using a camera to acquire the driver's facial image), inputting the facial image into a preset recognition model (the recognition model may include, but is not limited to, a convolutional neural network (CNN) model, a deep learning VGG model, a deep learning ResNet model, and a large artificial intelligence model, etc.), and obtaining a quantitative value of the emotional state, for example, the quantitative value of the emotional state is represented as $S_{panic}$, with a value from 0 to 100. Preferably, the range of values ​​for quantified emotional states can be divided, with different ranges corresponding to different psychological states. Specific implementations may include: dividing the quantified emotional state values ​​into three psychological state levels: a first level with a value range of 0 to A1, a second level with a value range of A1 to A2, and a third level with a value range of A2 to 100, with the emotion gradually intensifying from the first to the third level. Examples, but not limited to: a first level with a value range of 1 to 30, representing tension; a second level with a value range of 31 to 60, representing panic; and a third level with a value range of 61 to 100, representing surprise.

[0028] Step S103: In response to the quantification value of the emotional state being greater than a preset second threshold, detect whether there are obstacles on the side and / or behind the vehicle.

[0029] As an embodiment of the present invention, step S103 can compare the emotional state quantification value with a preset second threshold. If the emotional state quantification value is greater than the preset second threshold (for example: $S_{panic} > 75$), it is determined that the emotional state quantification value meets the preset condition for pedal misuse.

[0030] In this embodiment of the invention, step S103 can also detect the third mode in the multimodal approach for determining pedal misuse: the vehicle's environmental safety data. Specifically, it can detect whether there are obstacles to the side and / or rear of the vehicle, thereby ensuring the safety of pedal misuse intervention. For example, environmental sensors (including but not limited to millimeter-wave radar, ultrasonic radar, etc.) can be used to assess the immediate collision risk to the side and / or rear of the vehicle.

[0031] Step S104: If there are no obstacles to the side and / or rear of the vehicle, send an emergency braking command to the brake actuator to control the vehicle to stop.

[0032] In this embodiment of the invention, the emergency braking command may include the vehicle's braking deceleration (braking deceleration refers to the vehicle's ability to rapidly reduce its speed until it stops while driving). The specific implementation process includes: acquiring the vehicle's braking deceleration (for example, braking deceleration can be represented as $a_{brake\_max}$), generating the emergency braking command, and sending it to the brake actuator.

[0033] As an embodiment of the present invention, in order to more accurately determine the environmental safety data of the vehicle, when there are no obstacles to the side and / or rear of the vehicle, the present invention can acquire high-precision map data to detect whether there are potential lateral collision areas on the current driving segment (potential collision areas may include, but are not limited to, intersections, side roads, zebra crossings, curves, ramps, etc.). If there are no potential lateral collision areas on the current driving segment, an emergency braking command is sent to the brake actuator to control the vehicle to stop. It can be seen that, in order to avoid new dangers caused by misuse of the pedal (for example, to prevent side collisions), the present invention quickly checks the vehicle's lateral emergency avoidance needs before performing emergency braking.

[0034] Furthermore, this invention is not limited to the above-described technical solution for determining whether vehicle acceleration is to prevent a side collision. It can also determine whether vehicle acceleration is to prevent a side collision by detecting whether the vehicle is in a lateral movement state (the lateral movement state may include steering wheel angle, turning rate, lane lateral offset, etc.). For example: if the vehicle is in a lateral movement state, and it is determined that the vehicle acceleration is to prevent a side collision, then emergency braking is not performed; if the vehicle is not in a lateral movement state, and it is determined that the vehicle acceleration is not to prevent a side collision, then an emergency braking command is sent to the brake actuator to control the vehicle to stop.

[0035] In other embodiments of the present invention, to more accurately determine the environmental safety data of the vehicle, the present invention, when there are no obstacles to the side and / or rear of the vehicle, can calculate the time to collision (TTC, representing the time required for the vehicle to collide with an obstacle while maintaining its current speed or acceleration, typically calculated using relative distance and relative speed or acceleration). If the collision time exceeds a preset time threshold, an emergency braking command is sent to the brake actuator to control the vehicle to stop. It can be seen that, to avoid new dangers caused by misuse of the pedal (for example, to prevent being rear-ended), the present invention quickly checks the emergency avoidance needs of the vehicle's rear before performing emergency braking.

[0036] Furthermore, this invention is not limited to the above-mentioned technical solution for determining whether the vehicle acceleration is to prevent a rear-end collision. Alternatively, it can collect driver facial image data and monitor the number of times the driver observes the rearview mirror. If the number of times the driver observes the rearview mirror within a preset time period is greater than a preset number, it is determined that the vehicle acceleration is to prevent a rear-end collision, and emergency braking is not performed. If the number of times the driver observes the rearview mirror within a preset time period is less than or equal to a preset number, it is determined that the vehicle acceleration is not to prevent a rear-end collision, and an emergency braking command is sent to the brake actuator to control the vehicle to stop.

[0037] In some further embodiments of the present invention, in order to more accurately determine the environmental safety data of the vehicle, the present invention can also consider the deceleration control command that the vehicle may execute based on the Automatic Emergency Braking (AEB) algorithm for obstacles in front before performing emergency braking. The specific implementation process includes: when there are obstacles on the side and / or behind the vehicle, detecting whether the vehicle has a deceleration control command based on the Automatic Emergency Braking (AEB) algorithm; when the vehicle has a deceleration control command based on the Automatic Emergency Braking (AEB) algorithm, sending an emergency braking command to the brake actuator to control the vehicle to stop.

[0038] In this embodiment of the invention, after step S104 sends an emergency braking command to the brake actuator, clear and non-negligible HMI (Human Machine Interface) feedback can be provided within a preset time (for example, 70ms) to prevent secondary driver misoperation. The specific implementation process includes: generating display warning information and / or voice warning information according to a preset display warning template and / or a preset voice warning template, and outputting it to a display device and / or a voice playback device, thereby achieving visual and auditory warnings. For example: a red warning is displayed on the instrument panel: "Emergency Intervention: Pedal misuse detected and braking is implemented." A loud warning sound is played: "Brake activated."

[0039] In this embodiment of the invention, after step S104 sends an emergency braking command to the brake actuator, the HMI feedback can also control the accelerator pedal to output the maximum reaction force, thereby achieving a hard limit on the accelerator pedal and allowing the driver to clearly perceive that the accelerator pedal has been rejected by the system.

[0040] like Figure 2 A schematic diagram illustrating the main steps of the second method for intervening in pedal misuse is shown. In this embodiment of the invention, the steps include the following: Step S201: Determine whether the advanced driver assistance system of the vehicle is activated. If yes ( Figure 2 If the symbol is marked as X, then proceed to step S203; otherwise ( Figure 2 If the identifier is Y, then proceed to step S202.

[0041] Step S202: Continue the current driving operation.

[0042] Step S203: Monitor the input characteristics of the accelerator pedal sensor: pedal speed and pedal opening, and determine whether the pedal speed is greater than a preset speed threshold and the pedal opening is greater than a preset opening threshold. If yes, proceed to step S204; otherwise, proceed to step S202.

[0043] Step S204: Acquire a driver's facial image, input the facial image into a preset recognition model to obtain an emotional state quantification value, and determine whether the emotional state quantification value is greater than a preset second threshold. If yes, proceed to step S205; otherwise, proceed to step S202.

[0044] Step S205: Detect whether there are obstacles on the side and / or rear of the vehicle. If so, proceed to step S206; otherwise, proceed to step S207.

[0045] Step S206: Do not trigger pedal misuse intervention.

[0046] Step S207: Send an emergency braking command to the brake actuator to control the vehicle to stop, and proceed to step S208.

[0047] Step S208: Generate display warning information and / or voice warning information according to the preset display warning template and / or preset voice warning template, and output them to the display device and / or voice playback device, and control the accelerator pedal to output the maximum reaction force.

[0048] According to the embodiments of the present invention, the present invention can accurately determine pedal misuse and perform emergency safety intervention by accelerating to braking by integrating the driver's emotional state, pedal input characteristics and vehicle environmental safety data in the mode of advanced driver assistance system.

[0049] like Figure 3 A timing diagram of a pedal misuse intervention method is shown. In this embodiment of the invention, it includes pedal opening H, emotional state quantification value J, misuse intervention decision module K, pedal speed L, brake actuator M, and HMI feedback N. The pedal opening H and pedal speed L are obtained using an accelerator pedal sensor, and the driver's facial image is collected and input into a preset recognition model to obtain the emotional state quantification value J. Furthermore, W represents time; W1, W2, and W3 are negative times, representing the initialization before the misuse intervention decision module K is triggered, for example: W1 is T_trigger -500ms, W2 is T_trigger -200ms, and W3 is T_trigger -100ms; W4 is 0, representing the triggering time (intervention activation) of the misuse intervention decision module K, for example: W4 is T_trigger (0ms); W5, W6, W7, and W8 are positive times, representing the braking execution and state maintenance after the misuse intervention decision module K is triggered, for example: W5 is T_trigger +50ms, W6 is T_trigger +100ms, W7 is T_trigger +200ms, and W8 is T_trigger +1000ms.

[0050] In the embodiment, at time W1 before the misuse intervention decision module K is triggered, the pedal opening H can be initialized: the initial state H1 is Init P=0%, which means the pedal is not depressed; the emotional state quantization value J is initialized: the initial state J1 is Init S<threshold(theta_S), which means the second threshold is not reached and there is no emergency intervention; the misuse intervention decision module K is initialized: the initial state K1 is Init OFF, which means the intervention decision module is turned off; the pedal speed L is initialized: the initial state L1 is Init low, which means the pedal speed is low and there is no abnormality; the brake actuator M is initialized: the initial state M1 is Init 0, which means there is no braking force; the HMI feedback N is initialized: the initial state N1 is Init OFF, which means there is no warning.

[0051] At time W2 before the misuse intervention decision module K is triggered, the pedal speed L can reach the state L2 of increased pedal speed. And at time W3 before the misuse intervention decision module K is triggered, the pedal opening H can reach the abnormal input state H2 of misstepping the pedal to fully open.

[0052] At the trigger time W4 of the misuse intervention decision module K, the emotional state quantization value J is determined to be the state J2 greater than the preset second threshold (crosses theta_S); the misuse intervention decision module K determines that the vehicle meets the conditions that when the advanced driving assistance system is in the on state, the pedal speed is greater than the preset speed threshold, the pedal opening is greater than the preset opening threshold, the emotional state quantization value is greater than the preset second threshold, and there are no obstacles on the side and / or rear of the vehicle, and switches to the activation state K2; the pedal speed L is in state L3: cut off or reset to zero, to suppress the acceleration trend.

[0053] At time W5 after the misuse intervention decision module K is triggered, the brake actuator M is in state M2: executing braking deceleration. At time W6 after the misuse intervention decision module K is triggered, the HMI feedback N is in state N2: issuing a warning prompt to the driver.

[0054] At time W7 after the misuse intervention decision module K is triggered, the pedal opening H is in state H3: controlled attenuation; the emotional state quantization value J is in state J3: gradually decreasing; the misuse intervention decision module K switches to state K3: maintaining the braking state; the pedal speed L is in state L4: kept at 0 to completely suppress acceleration; the brake actuator M is in state M3: maintaining the braking deceleration to stabilize the braking effect; the HMI feedback N is in state N3: continuously outputting warnings to the driver to strengthen the safety prompt.

[0055] At time W8 after the misuse intervention decision module K is triggered, the pedal opening H is in state H4: the time window for pedal misuse intervention is closed; the emotional state quantification value J is in state J4: the time window for pedal misuse intervention is closed; the misuse intervention decision module K is in state K4: the time window for pedal misuse intervention is closed; the pedal speed L is in state L5: the time window for pedal misuse intervention is closed; the brake actuator M is in state M4: the time window for pedal misuse intervention is closed; and the HMI feedback N is in state N4: the time window for pedal misuse intervention is closed.

[0056] like Figure 4 A schematic diagram of a pedal misuse intervention system is shown. In this embodiment of the invention, the pedal misuse intervention system includes a perception input module A, a feature processing module B, a misuse intervention decision module C, a pedal control module D, a braking execution module E, and an HMI feedback module F. The perception input module A can collect raw data. Specifically, it can collect the input characteristics of the accelerator pedal, a driver's facial image, and radar data from the side and / or rear of the vehicle. The feature processing module B processes the accelerator pedal input characteristics, driver's facial image, and radar data from the side and / or rear of the vehicle collected by the perception input module A to obtain pedal speed, pedal opening, a quantitative value of emotional state, and detection data on the presence of obstacles to the side and / or rear of the vehicle. Specifically, it can input the facial image into a preset recognition model for processing to obtain a quantitative value of emotional state, determine the presence of obstacles based on the radar data from the side and / or rear of the vehicle, and obtain pedal speed and pedal opening from the input characteristics of the accelerator pedal. The misuse intervention decision module C can determine that the vehicle meets the following conditions when the advanced driver assistance system is activated: the pedal speed is greater than a preset speed threshold, the pedal opening is greater than a preset opening threshold, the emotional state quantification value is greater than a preset second threshold, and there are no obstacles to the side and / or rear of the vehicle. In this case, the braking execution module E sends an emergency braking command to the brake actuator to stop the vehicle. The HMI feedback module F generates display warning information and / or voice warning information based on preset display warning templates and / or preset voice warning templates and outputs them to the display device and / or voice playback device, and controls the accelerator pedal to output maximum reaction force. Furthermore, the misuse intervention decision module C can determine that the vehicle meets the following conditions when the advanced driver assistance system is activated: the pedal speed is greater than a preset speed threshold, the pedal opening is greater than a preset opening threshold, the emotional state quantification value is greater than a preset second threshold, and there are no obstacles to the side and / or rear of the vehicle. In this case, the pedal control module D limits or manipulates the accelerator pedal. Additionally, the pedal control module D can also cause the braking execution module E to send a braking command to the brake actuator.

[0057] The above steps are provided only to help understand the method, structure, and core ideas of this invention. Those skilled in the art can make various improvements and modifications to this invention without departing from its principles, and these improvements and modifications also fall within the scope of protection of the claims.

[0058] Figure 5 This is a schematic diagram of the main unit of a pedal misuse intervention device according to an embodiment of the present invention. Figure 5 As shown, an embodiment of the present invention provides a pedal misuse intervention device 500, comprising: a monitoring unit 501, a detection unit 502, and a processing unit 503. The monitoring unit 501 monitors the input characteristics of the accelerator pedal when the advanced driver assistance system is activated. The detection unit 502, in response to the input characteristics exceeding a preset first threshold, acquires a driver's facial image and inputs the facial image into a preset recognition model to obtain a quantitative value of the emotional state. In response to the emotional state quantification value exceeding a preset second threshold, it detects whether there are obstacles to the side and / or rear of the vehicle. The processing unit 503, when there are no obstacles to the side and / or rear of the vehicle, sends an emergency braking command to the brake actuator to control the vehicle to stop.

[0059] In this embodiment of the invention, the monitoring unit 501 is also used to monitor the pedal speed and pedal opening of the accelerator pedal sensor; The detection unit 502 is also used to acquire a driver's facial image in response to the pedal speed being greater than a preset speed threshold and the pedal opening being greater than a preset opening threshold.

[0060] In this embodiment of the invention, the processing unit 503 is further configured to acquire high-precision map data and detect whether there is a potential lateral collision area in the current driving segment when there are no obstacles to the side and / or rear of the vehicle; and send an emergency braking command to the brake actuator to control the vehicle to stop when there is no potential lateral collision area in the current driving segment.

[0061] In this embodiment of the invention, the processing unit 503 is further configured to calculate the collision time between the vehicle and the following vehicle when there are no obstacles to the side and / or rear of the vehicle, and send an emergency braking command to the brake actuator to control the vehicle to stop when the collision time is greater than a preset time threshold.

[0062] In this embodiment of the invention, the processing unit 503 is further configured to detect whether the vehicle has a deceleration control command based on an automatic emergency braking algorithm when there are obstacles to the side and / or rear of the vehicle; and to send an emergency braking command to the brake actuator to control the vehicle to stop when there is a deceleration control command based on an automatic emergency braking algorithm.

[0063] In this embodiment of the invention, the processing unit 503 is also used to control the accelerator pedal to output the maximum reaction force.

[0064] In this embodiment of the invention, the processing unit 503 is further configured to generate display warning information and / or voice warning information according to a preset display warning template and / or a preset voice warning template, and output them to a display device and / or a voice playback device.

[0065] In this embodiment of the invention, the processing unit 503 is further configured to acquire the braking deceleration of the vehicle, generate the emergency braking command, and send it to the brake actuator.

[0066] like Figure 6 As shown, an embodiment of the present invention provides a vehicle 600, which may include the pedal misuse intervention device 500 provided in the above embodiments.

[0067] Figure 7 An exemplary vehicle system architecture 700 is shown where the pedal misuse intervention method or pedal misuse intervention device of the present invention can be applied.

[0068] like Figure 7 As shown, the vehicle system architecture 700 may include various systems, such as an intelligent driving system 701, a powertrain system 702, a sensor system 703, a control system 704, one or more peripheral devices 705, a power supply 706, a computer system 707, and a user interface 708. Optionally, the vehicle system architecture 700 may include more or fewer systems, and each system may include multiple components. Furthermore, each system and component of the vehicle system architecture 700 can be interconnected via wired or wireless means.

[0069] The vehicle system architecture 700 includes an intelligent driving system 701, which can be in a fully or partially automated driving mode. For example, the intelligent driving system 701 can automatically control the vehicle's driving without human interaction; the intelligent driving system 701 can also control the vehicle's automated driving while in automated driving mode, and can also adjust its automated driving behavior through human interaction. Specifically, when the advanced driver assistance system is activated, the intelligent driving system 701 can monitor the input characteristics of the accelerator pedal; in response to the input characteristics exceeding a preset first threshold, it can acquire a driver's facial image, input the facial image into a preset recognition model, and obtain an emotional state quantification value; in response to the emotional state quantification value exceeding a preset second threshold, it can detect whether there are obstacles to the side and / or rear of the vehicle; if there are no obstacles to the side and / or rear of the vehicle, it can send an emergency braking command to the brake actuator to control the vehicle to stop.

[0070] The powertrain 702 may include components that provide power to the vehicle. For example, the powertrain 702 may include an engine, an energy source, a transmission, wheels, tires, etc. The engine may be an internal combustion engine, an electric motor, an air-compressed engine, or a combination of other types of engines, such as a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air-compressed engine. The engine converts the energy source into mechanical energy to supply the transmission. Examples of energy sources may include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other electrical sources. The energy source may also provide energy to other systems in the vehicle. Furthermore, the transmission may include a gearbox, a differential, a drive shaft, and a clutch, etc.

[0071] Sensor system 703 may include sensors for sensing the vehicle's surrounding environment. Examples include a positioning system (which may be a Global Positioning System (GPS), BeiDou Navigation Satellite System, or other positioning systems), radar, a laser rangefinder, an inertial measurement unit (IMU), and a camera. The positioning system can be used to determine the vehicle's geographical location. The IMU is used to sense changes in the vehicle's position and orientation based on inertial acceleration. In one embodiment, the IMU may be a combination of an accelerometer and a gyroscope. The radar can use radio signals to sense objects in the vehicle's surrounding environment. In some embodiments, in addition to sensing objects, the radar can also be used to sense the speed and / or direction of travel of objects.

[0072] To monitor environmental information and objects located in front of, behind, or to the sides of the vehicle, radar, cameras, and other devices can be configured at appropriate locations on the exterior of the vehicle. For example, to acquire an image of the front of the vehicle, a camera can be configured inside the vehicle and close to the windshield. Alternatively, the camera can be configured around the front bumper or radiator grille. Similarly, to acquire an image of the rear of the vehicle, a camera can be configured inside the vehicle and close to the rear window. Alternatively, the camera can be configured around the rear bumper, trunk, or tailgate. To acquire images of the sides of the vehicle, a camera can be configured inside the vehicle and close to at least one of the side windows. Alternatively, the camera can be configured around the side mirrors, fenders, or doors.

[0073] Laser rangefinders use lasers to sense objects in the environment in which a vehicle is located.

[0074] A camera can be used to capture multiple images of the vehicle's surroundings. The camera can be a still camera or a video camera.

[0075] The control system 704 may include software systems for implementing autonomous driving, such as a route planning system, an obstacle avoidance system, and a vision system for image analysis. The control system 704 may also include hardware systems such as an accelerator and steering wheel system. Furthermore, the control system 704 may add or replace components other than those shown and described. Alternatively, some of the components shown above may be reduced.

[0076] The control system 704 interacts with external sensors, other autonomous driving devices, other computer systems, or users via peripheral devices 705. Peripheral devices 705 may include wireless communication systems, on-board computers, microphones, and / or speakers.

[0077] In some embodiments, peripheral device 705 provides a means for user interaction with the control system 704 via a user interface. For example, an onboard computer may provide information to a user of the vehicle. The user interface may also operate the onboard computer to receive user input. The onboard computer may be operated via a touchscreen. In other cases, peripheral device may provide a means for communicating with other devices located within the vehicle. For example, a microphone may receive audio (e.g., voice commands or other audio input) from a user of the control system 704. Similarly, a speaker may output audio to a user of the control system 704.

[0078] Wireless communication systems can communicate wirelessly with one or more devices, either directly or via a communication network. For example, wireless communication systems can use networks such as cellular networks, WiFi, and wireless local area networks (WLANs), or they can use infrared links, Bluetooth, or ZigBee to communicate directly with devices. Other wireless protocols include those used in various autonomous driving communication systems.

[0079] The power source 706 can provide power to various components of the vehicle. The power source 706 can be a rechargeable lithium-ion or lead-acid battery.

[0080] The computer system 707 controls some or all of the functions enabling autonomous driving. The computer system 707 may include at least one processor that executes instructions stored in a non-transitory computer-readable medium such as memory. The computer system 707 provides the execution code for the aforementioned intelligent driving system to enable autonomous driving.

[0081] The processor can be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, the processor can be a special-purpose device such as an application-specific integrated circuit (ASIC) or other hardware-based processor. Those skilled in the art will understand that the processor, computer, or memory can actually include multiple processors, computers, or memories that may or may not be stored in the same physical housing. For example, memory can be a hard disk drive or other storage media located in a housing different from that of a computer. Therefore, references to processors or computers will be understood to include references to a collection of processors or computers or memories that may or may not operate in parallel. Unlike using a single processor to perform the steps described herein, some components, such as steering and deceleration components, may each have their own processor that performs calculations only related to the component's specific function.

[0082] User interface 708 is used to provide information to or receive information from users of the vehicle. Optionally, user interface 708 may include one or more input / output devices within a set of peripheral devices 705, such as wireless communication systems, on-board computers, microphones, and speakers.

[0083] It should be understood that the components described above are merely an example. In actual applications, components in the various modules or systems mentioned above may be added or removed as needed. Figure 7 This should not be construed as a limitation on the embodiments of this application.

[0084] The following is for reference. Figure 8 It shows a schematic diagram of the structure of a computer system 800 suitable for implementing embodiments of the present invention. Figure 8 The computer system shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0085] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 802 or programs loaded from storage section 808 into random access memory (RAM) 803. The RAM 803 also stores various programs and data required for the operation of the system 800. The CPU 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0086] The following components are connected to I / O interface 805: an input section 806; an output section 807 including devices such as cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers; a storage section 808 including devices such as hard disks; and a communication section 809 including network interface cards such as LAN cards and modems. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 810 as needed so that computer programs read from it can be installed into storage section 808 as needed.

[0087] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 88. When the computer program is executed by central processing unit (CPU) 801, it performs the functions defined above in the system of this invention.

[0088] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0089] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0090] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be housed in a processor; for example, a processor can be described as including a monitoring unit, a detection unit, and a processing unit. The names of these units do not necessarily limit the module itself; for example, the monitoring unit can also be described as a "module for monitoring the input characteristics of the accelerator pedal."

[0091] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs that, when executed by the device, cause the device to: monitor the input characteristics of the accelerator pedal when the advanced driver assistance system is activated; in response to the input characteristics being greater than a preset first threshold, acquire a driver's facial image, input the facial image into a preset recognition model, and obtain an emotional state quantification value; in response to the emotional state quantification value being greater than a preset second threshold, detect whether there are obstacles to the side and / or rear of the vehicle; and, if there are no obstacles to the side and / or rear of the vehicle, send an emergency braking command to a brake actuator to control the vehicle to stop.

[0092] According to the technical solution of the present invention, the technical problem of not being able to accurately identify pedal misuse can be solved by multimodal information fusion, which greatly improves vehicle safety in intelligent driving scenarios.

[0093] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A pedal misuse intervention method characterized by, include: With the advanced driver assistance system activated, monitor the input characteristics of the accelerator pedal; In response to the input characteristic being greater than a preset first threshold, a driver's facial image is acquired, and the facial image is input into a preset recognition model to obtain a quantitative value of the emotional state. In response to the quantification value of the emotional state being greater than a preset second threshold, the presence of obstacles on the side and / or rear of the vehicle is detected; If there are no obstacles to the side and / or rear of the vehicle, an emergency braking command is sent to the brake actuator to bring the vehicle to a stop.

2. The pedal misuse intervention method of claim 1, wherein, The monitoring of the input characteristics of the accelerator pedal includes: monitoring the pedal speed and pedal opening of the accelerator pedal sensor; The step of acquiring a driver's facial image in response to the input characteristic being greater than a preset first threshold includes: acquiring a driver's facial image in response to the pedal speed being greater than a preset speed threshold and the pedal opening being greater than a preset opening threshold.

3. The pedal misuse intervention method of claim 1, wherein, After the condition that there are no obstacles to the side and / or rear of the vehicle, the method further includes: Acquire high-precision map data to detect potential lateral collision zones on the current driving route; If there is no potential lateral collision zone in the current driving section, an emergency braking command is sent to the brake actuator to bring the vehicle to a stop.

4. The pedal misuse intervention method of claim 1, wherein, After the condition that there are no obstacles to the side and / or rear of the vehicle, the method further includes: The collision time between the vehicle and the following vehicle is calculated. If the collision time exceeds a preset time threshold, an emergency braking command is sent to the brake actuator to control the vehicle to stop.

5. The pedal misuse intervention method of claim 1, wherein, After detecting whether there are obstacles to the side and / or rear of the vehicle, the method further includes: In the event of an obstacle to the side and / or rear of the vehicle, detect whether the vehicle has a deceleration control command based on an automatic emergency braking algorithm; When the vehicle has a deceleration control command based on an automatic emergency braking algorithm, an emergency braking command is sent to the brake actuator to control the vehicle to stop.

6. The pedal misuse intervention method of claim 1, wherein, After sending the emergency braking command to the brake actuator, the method further includes: Control the accelerator pedal to output the maximum reaction force.

7. The pedal misuse intervention method of claim 1, wherein, After sending the emergency braking command to the brake actuator, the method further includes: Based on the preset display warning template and / or preset voice warning template, generate display warning information and / or voice warning information and output them to the display device and / or voice playback device.

8. A pedal misuse intervention method according to any one of claims 1-7, characterized in that, Sending an emergency braking command to the brake actuator includes: The vehicle's braking deceleration is obtained, the emergency braking command is generated, and sent to the brake actuator.

9. A pedal misuse intervention device characterized by, include: The system comprises a monitoring unit, a detection unit, and a processing unit, among which... The monitoring unit is used to monitor the input characteristics of the accelerator pedal when the advanced driver assistance system is activated. The detection unit is configured to, in response to the input characteristic being greater than a preset first threshold, acquire a driver's facial image, input the facial image into a preset recognition model to obtain an emotional state quantification value; and, in response to the emotional state quantification value being greater than a preset second threshold, detect whether there are obstacles on the side and / or rear of the vehicle. The processing unit is configured to send an emergency braking command to the brake actuator to control the vehicle to stop when there are no obstacles to the side and / or rear of the vehicle.

10. A vehicle characterized by comprising: include: The pedal misuse intervention device as described in claim 9.