Anti-mistaken takeover control method, device and system and storage medium
By detecting the driver's line of sight and assessing the risk value, operation inhibition commands are sent to address the problem of driver misoperation in Level 3 autonomous driving, thereby improving the safety and coordination of vehicle control.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ANHUI ZHIJIE NEW ENERGY VEHICLE CO LTD
- Filing Date
- 2026-03-31
- Publication Date
- 2026-05-01
AI Technical Summary
In existing technologies, the risks caused by driver error during Level 3 autonomous driving are difficult to identify and control effectively, leading to reduced vehicle safety and the risk of unexpected exit from the autonomous driving system.
By detecting the driver's line of sight and assessing the risk value, an operation inhibition command is sent to prevent the vehicle from responding to the driver's operation signals. By combining a deep learning model and a risk quantification assessment model, the safety and coordination of vehicle control are ensured.
It reduces the risk of unintentional misoperation leading to mistaken control, improves the safety and comprehensiveness of vehicle control, and avoids dangerous situations caused by misjudgment.
Smart Images
Figure CN121947556A_ABST
Abstract
Description
A method, device, system, and storage medium for preventing accidental disconnection. Technical Field
[0001] This application relates to the field of autonomous driving technology, and in particular to a method, device, system and storage medium for preventing accidental takeover control. Background Technology
[0002] Level 3 Autonomous Driving, also known as conditional autonomous driving, allows drivers to completely relinquish driving duties to the system under certain conditions. However, drivers may become complacent and enter a state of "cognitive disengagement" due to excessive trust in the system (such as using a mobile phone or falling asleep). This creates a dangerous "uncontrolled" zone, where any driver error that is interpreted as a takeover attempt poses a significant risk to vehicle safety. Current technologies primarily focus on protecting against mishandling of controls, such as misoperation of the accelerator or steering wheel. These approaches not only fail to effectively identify the driver's true intention to take over but also fail to comprehensively coordinate the control of all components, leading to the risk of unexpected system disengagement and potentially causing serious accidents such as lane departure or collisions with guardrails.
[0003] Therefore, how to provide a control method to prevent accidental takeover, reduce the risk of driver misoperation, and improve the safety of vehicle control has become an urgent technical problem to be solved. Summary of the Invention
[0004] This application provides a method, device, system, and storage medium for preventing accidental takeover, in order to reduce the risk of accidental takeover caused by driver error and improve the safety of vehicle control.
[0005] This application provides a method for preventing accidental takeover control, comprising: detecting the driver's gaze when a driver operation signal is received; sending a first operation suppression command to the vehicle motion control system when the driver's gaze leaves the road ahead, the operation suppression command being used to prohibit the vehicle motion control system from responding to the driver operation signal; assessing a first risk value corresponding to the driver takeover operation and a second risk value corresponding to the autonomous driving system continuing to control the vehicle when the driver's gaze is not leaving the road ahead; and sending a second operation suppression command to the vehicle motion control system when the first risk value is greater than the second risk value.
[0006] The beneficial effects of this application are as follows: When a driver's operation signal is received, the driver's gaze is detected, and when the driver's gaze leaves the road ahead, a first operation suppression command is sent to the vehicle motion control system. This operation suppression command prevents the vehicle motion control system from responding to the driver's operation signal, thereby reducing the risk of unintentional mishandling. Even if the driver's gaze remains on the road ahead, a first risk value corresponding to the driver's takeover operation and a second risk value corresponding to the autonomous driving system continuing to control the vehicle are assessed. Control is only allowed to transfer when it is confirmed that takeover is safer. When the first risk value is greater than the second risk value, a second operation suppression command is sent to the vehicle motion control system, further avoiding the risk of unintentional takeover due to driver misjudgment. Furthermore, this application sends suppression commands to the entire control system, rather than controlling individual components, improving the comprehensiveness and coordination of overall vehicle risk prevention and control, and further enhancing the safety of vehicle control.
[0007] In one embodiment, detecting the driver's gaze includes: acquiring a facial image of the driver; locating multiple key feature points in the facial image based on a deep learning model, the key feature points including at least eye contour points, pupil centers, eyebrows, nose contour points, and mouth corner contour points; estimating the driver's real-time gaze direction based on the relative positional relationship between the located pupil centers and predetermined eye corner reference points; and determining whether the driver's visual focus has deviated from the road area in front of the vehicle by combining the driver's head posture data and the gaze direction.
[0008] In one embodiment, after sending a first operation suppression command to the vehicle motion control system, the method further includes: generating and outputting a reminder message to the driver, the reminder message containing a textual description and correction guidance for the specific erroneous operation currently suppressed; and simultaneously triggering at least one tactile alarm, the tactile alarm including seat vibration and / or seat belt pretensioning.
[0009] In one embodiment, assessing a first risk value corresponding to a driver takeover operation and a second risk value corresponding to the autonomous driving system continuing to control the vehicle includes: acquiring current driving environment data containing the vehicle's dynamic parameters and the state of surrounding obstacles while the driver's gaze remains on the road ahead; substituting the current driving environment data into two different predefined scenario models—one for driver takeover control and the other for autonomous driving system maintenance control—to obtain driving data corresponding to the two scenarios; and calculating and outputting the probability values of collisions occurring within a future preset time window based on a risk quantification assessment model, thereby obtaining the first risk value and the second risk value.
[0010] In one embodiment, after the step of sending a second operation inhibition command to the vehicle motion control system when the first risk value is greater than the second risk value, the method further includes: generating and outputting intervention reminder information to the driver, wherein the intervention reminder information is used to warn the driver of the high-risk consequences of the current operation and explain the reason for the system to perform operation inhibition.
[0011] In one embodiment, the method includes: when the driver's gaze is not taken off the road ahead and the first risk value is less than or equal to the second risk value, responding to the driver's operation and switching control of the vehicle to the driver; after the switch is completed, outputting a notification message to the driver confirming that the vehicle has been taken over.
[0012] In one embodiment, the method further includes: cyclically detecting the driver's gaze state and whether there is a continuous driver operation signal; when it is detected that the driver's gaze has returned to the road ahead and the same type of driver operation signal still exists, re-evaluating the first risk value corresponding to the driver takeover operation and the second risk value corresponding to the autonomous driving system continuing to control the vehicle; when the re-evaluated first risk value is less than or equal to the second risk value, canceling the operation suppression command and allowing the vehicle motion control system to respond to the driver operation signal.
[0013] This application also provides a misoperation prevention control device, comprising: a first detection module for detecting the driver's gaze when a driver operation signal is received; a first sending module for sending a first operation suppression command to the vehicle motion control system when the driver's gaze leaves the road ahead, the operation suppression command being used to prohibit the vehicle motion control system from responding to the driver operation signal; a first evaluation module for evaluating a first risk value corresponding to the driver's takeover operation and a second risk value corresponding to the autonomous driving system continuing to control the vehicle when the driver's gaze is not leaving the road ahead; and a second sending module for sending a second operation suppression command to the vehicle motion control system when the first risk value is greater than the second risk value.
[0014] In one embodiment, the detection module includes: a first acquisition submodule for acquiring a driver's facial image; a localization submodule for locating multiple key feature points in the facial image based on a deep learning model, wherein the key feature points include at least eye contour points, pupil centers, eyebrows, nose contour points, and mouth corner contour points; an estimation submodule for estimating the driver's real-time gaze direction based on the relative positional relationship between the located pupil center and a predetermined eye corner reference point; and a judgment submodule for determining whether the driver's visual focus has deviated from the road area in front of the vehicle by combining the driver's head posture data and the gaze direction.
[0015] In one embodiment, the device further includes: a first generation module for generating and outputting reminder information to the driver, the reminder information including a textual description and correction guidance for the specific erroneous behavior currently suppressed; and a triggering module for triggering at least one tactile alarm, the tactile alarm including seat vibration and / or seat belt pretensioning.
[0016] In one embodiment, the evaluation module includes: a second acquisition submodule, configured to acquire current driving environment data including the vehicle's dynamic parameters and the status of surrounding obstacles while the driver's line of sight remains on the road ahead; a substitution submodule, configured to substitute the current driving environment data into two different predefined scenario models: driver takeover control and autonomous driving system maintenance control, to obtain driving data corresponding to the two scenarios; and a calculation submodule, configured to calculate and output the risk probability values of collisions occurring within a future preset time window based on a risk quantification assessment model, to obtain the first risk value and the second risk value.
[0017] In one embodiment, the device further includes: a second generation module, configured to generate and output intervention reminder information to the driver, the intervention reminder information being used to warn the driver of the high-risk consequences of their current operation and to explain the reason for the system's operation suppression.
[0018] In one embodiment, the device includes: a response module, configured to respond to a driver's operation and switch control of the vehicle to the driver when the driver's gaze is not taken off the road ahead and the first risk value is less than or equal to the second risk value; and a confirmation module, configured to output a notification message to the driver confirming that the vehicle has been taken over after the switch is completed.
[0019] In one embodiment, the device further includes: a second detection module for cyclically detecting the driver's gaze state and whether there is a continuous driver operation signal; a second evaluation module for re-evaluating a first risk value corresponding to the driver takeover operation and a second risk value corresponding to the autonomous driving system continuing to control the vehicle when the driver's gaze has returned to the road ahead and the same type of driver operation signal still exists; and a cancellation module for canceling the operation suppression command when the re-evaluated first risk value is less than or equal to the second risk value, allowing the vehicle motion control system to respond to the driver operation signal.
[0020] This application also provides a control system for preventing accidental takeover, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to implement the control method for preventing accidental takeover described in any of the above embodiments.
[0021] This application also provides a computer-readable storage medium, which, when the instructions in the storage medium are executed by the processor corresponding to the anti-misoperation control system, enables the anti-misoperation control system to implement the anti-misoperation control method described in any of the above embodiments.
[0022] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.
[0023] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description
[0024] The accompanying drawings are provided to further understand this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings: Figure 1 is a flowchart of a method for preventing misoperation control according to an embodiment of this application; Figure 2 is a schematic diagram of signal interaction of a control system for preventing misoperation according to an embodiment of this application; Figure 3 is a flowchart of the operation of a control system for preventing misoperation according to an embodiment of this application; Figure 4 is a flowchart of the monitoring process of a driver monitoring system according to an embodiment of this application; Figure 5 is a schematic diagram of the structure of a device for preventing misoperation control according to an embodiment of this application; Figure 6 is a schematic diagram of the hardware structure of a control system for preventing misoperation according to an embodiment of this application. Detailed Implementation
[0025] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application.
[0026] Figure 1 is a flowchart of a method for preventing accidental takeover control according to an embodiment of this application. As shown in Figure 1, the method can be implemented as follows: Step S101-S104: In step S101, when a driver operation signal is received, the driver's gaze is detected; In step S102, when the driver's gaze leaves the road ahead, a first operation suppression command is sent to the vehicle motion control system, the operation suppression command being used to prohibit the vehicle motion control system from responding to the driver operation signal; In step S103, when the driver's gaze has not left the road ahead, a first risk value corresponding to the driver takeover operation and a second risk value corresponding to the autonomous driving system continuing to control the vehicle are evaluated; In step S104, when the first risk value is greater than the second risk value, a second operation suppression command is sent to the vehicle motion control system.
[0027] Figure 2 is a schematic diagram of the signal interaction of the anti-misoperation control system in one embodiment of this application. As shown in Figure 2, the anti-misoperation control system is a general system composed of multiple functional systems. The functional systems include: Driver Monitoring System (DMS), Automated Driving System (ADS), and Vehicle Motion Control System (VMCS). They interact with each other through the vehicle network and ultimately communicate with the driver through the Human-Machine Interaction System (HMS). The system comprises several components: a driver monitoring system and an autonomous driving system. The driver monitoring system uses infrared or RGB cameras installed in the cockpit (e.g., above the A-pillar or dashboard) to capture real-time video streams of the driver's face. The system analyzes the video streams and uses computer vision algorithms (e.g., facial landmark detection, head pose estimation, gaze estimation) to output a binary judgment result indicating whether the driver's gaze is on the road. The vehicle motion control system uses sensors (e.g., torque sensors, stroke sensors) installed on the steering wheel, accelerator pedal, and brake pedal to collect the driver's physical operation signals. These raw sensor signals are processed into standard vehicle control commands (e.g., steering wheel angle requests, throttle opening requests, brake pressure requests). The autonomous driving system receives data from both the driver monitoring system and the vehicle motion control system, performs multi-layered logical judgments, and outputs the final control commands. The first layer of decision-making determines whether to allow driver intervention based on whether the driver's gaze is on the road. The second layer of decision-making further assesses the risk of driver intervention if the driver's gaze is normal, comparing the risk of driver intervention with the risk of autonomous driving.
[0028] Figure 3 is a flowchart of the anti-misoperation control system in one embodiment of this application. As shown in Figure 3, in this application, when a driver's operation signal is received, the driver's gaze is detected. The vehicle motion control system monitors and identifies the driver's operation signal. Specifically, it collects signals such as steering wheel, accelerator, and brake by monitoring lateral control (e.g., electronic power steering), longitudinal control (e.g., longitudinal integrated controller), and powertrain control (e.g., vehicle control unit and drive motor). Lateral control mainly monitors and identifies whether the driver operates the steering wheel, longitudinal control mainly monitors and identifies whether the driver presses the brake pedal, and powertrain control mainly monitors and identifies whether the driver intervenes with the accelerator pedal. The vehicle motion control system monitors and identifies driver operation information, and inputs valid driver operation information to the autonomous driving system after identification.
[0029] When the autonomous driving system receives a driver operation signal, it acquires the driver's gaze monitoring results provided in real time by the driver monitoring system. Figure 4 is a flowchart of the driver monitoring system in one embodiment of this application. As shown in Figure 4, the driver monitoring system monitors the driver's state in real time. By acquiring the driver's facial image, it locates multiple key feature points in the facial image based on a deep learning model, such as eye contour points, pupil center, eyebrows, nose contour points, and mouth corner contour points. By calculating the distance ratio between the upper and lower eyelids, it determines whether the driver's eyes are closed. It can also estimate the driver's real-time gaze direction and field of vision, such as forward, left, right, and downward, based on the relative positional relationship between the located pupil center and the predetermined eye corner reference point, thereby determining whether the driver's gaze is monitoring the road ahead in real time. On the other hand, by measuring the rotation angle of the driver's head (yaw, pitch, roll), it determines whether the driver is looking down (looking at a mobile phone), turning their head (looking at the rearview mirror or side window), or nodding (dozing off), thereby determining whether the driver's gaze is monitoring the road ahead in real time. Of course, the two monitoring methods can be combined to determine whether the driver's visual focus has shifted away from the road area in front of the vehicle, by combining the driver's head posture data and the direction of his gaze.
[0030] When the driver's gaze leaves the road ahead, a first operation suppression command is sent to the vehicle motion control system. This operation suppression command prevents the vehicle motion control system from responding to the driver's operation signal. When the vehicle control system receives the driver suppression information from the autonomous driving system, it does not respond to any driver actions, including but not limited to the driver touching or dragging the steering wheel, pressing the accelerator pedal, or pressing the brake pedal. Simultaneously, a reminder message is generated and output to the driver. This reminder message includes a text description and correction guidance for the specific erroneous operation currently suppressed. For example, a text and voice reminder may be given through the human-machine interface and speaker of the human-machine interaction system: "You have accidentally pressed the accelerator / brake / accidentally touched the steering wheel. Please pay attention to the road ahead." At least one tactile alarm is also triggered, including seat vibration and / or seatbelt pretensioning.
[0031] When the driver's eyes are not off the road ahead, assess the first risk value corresponding to the driver taking over and the second risk value corresponding to the autonomous driving system continuing to control the vehicle. The specific assessment process is as follows: (1) Under the condition that the driver’s eyes are not off the road ahead, obtain the current driving environment data including the dynamic parameters of the vehicle and the status of the surrounding obstacles; specifically including: the dynamic parameters of the vehicle, such as the real-time speed, acceleration, steering angle, braking status, remaining power (fuel quantity), tire pressure, etc.; the status of the surrounding obstacles, such as the position, speed, driving direction, and vehicle type of the surrounding vehicles, the position, speed, and direction of movement of pedestrians and non-motorized vehicles, and the position and type of static road obstacles (such as guardrails, construction barriers, road signs, etc.); road environment information, such as road type (highway, urban road, rural road, etc.), number of lanes, lane width, road curvature, slope, traffic light status, traffic sign content, current weather (sunny, rainy, snowy, foggy, etc.), light intensity (daytime, nighttime, tunnel, etc.); it can also include driver status data, such as the driver’s heart rate, blood pressure, blinking frequency, facial expression (through facial image recognition) and other physiological status data, as well as the driver’s driving history data (such as average reaction time, emergency operation frequency, violation records, etc.).
[0032] (2) Substitute the current driving environment data into two different predefined scenario models: driver takeover control and autonomous driving system maintenance control, to obtain driving data corresponding to the two scenarios. The driver takeover control scenario model can combine the driver's driving history data and current physiological state data to simulate the possible actions (such as acceleration, deceleration, steering, braking, etc.) that the driver might take over the vehicle, and the possible consequences of these actions in the current driving environment. Of course, factors such as the driver's reaction time and operational accuracy can also be considered to dynamically adjust the model. The autonomous driving system maintenance control scenario model can be based on the autonomous driving system's algorithm logic and historical operating data to simulate the autonomous driving system's control strategies (such as lane keeping, following other vehicles, automatic lane changing, etc.) in the current driving environment, and the risks these strategies may face (such as sensor misjudgment, algorithm decision-making errors, etc.). Simultaneously, the model is updated in real time by combining road environment information and surrounding obstacle status data.
[0033] (3) Based on the risk quantification assessment model, calculate and output the risk probability values of collisions occurring within a future preset time window for the driving data corresponding to the two scenarios, and obtain the first risk value and the second risk value. According to the driving environment and operating conditions, such as the vehicle's heading angle, speed, and the identification of obstacles in front of, behind, and to the side of the vehicle at the time of takeover, determine whether there is a collision risk within a future preset time (e.g., within 3 seconds, which can be calibrated), and assess whether the risk of takeover is greater than the risk of autonomous driving.
[0034] Specifically, the risk quantification assessment model's indicator system includes a multi-dimensional risk assessment indicator system encompassing collision risk, operational stability, and environmental adaptability. Collision risk indicators include the probability of collision with vehicles in front, behind, or to the side, pedestrians, and static obstacles; operational stability indicators include driving smoothness, steering stability, and braking stability after vehicle takeover; environmental adaptability indicators include the driver's or autonomous driving system's ability to adapt to current road conditions, weather conditions, and light intensity. Weights are assigned to different indicators; for example, a combination of the Analytic Hierarchy Process (AHP) and entropy weighting is used to allocate weights to each risk assessment indicator. The AHP is used to determine the subjective weights of the indicators based on expert experience and domain knowledge, while the entropy weighting method is used to determine the objective weights of the indicators based on the dispersion of the data, ultimately resulting in a comprehensive weight for each indicator. Finally, based on the constructed risk assessment indicator system and the determined indicator weights, fuzzy comprehensive evaluation methods or neural network algorithms are used to calculate the risk values under driver takeover control scenarios and autonomous driving system maintenance control scenarios, respectively. During the calculation process, different types of risk indicators are quantified, and the interrelationships between various indicators are considered. In addition, historical accident data and real-time driving environment data can be combined to correct the calculated risk values. For example, in adverse weather conditions such as rain or fog, the weight of collision risk indicators is appropriately increased; when the driver's physiological state is poor (such as excessively fast heart rate or excessive blinking frequency), the risk value under driver takeover control scenarios is appropriately increased.
[0035] Since shifting attention takes time, it is necessary to consider not only the driver's field of vision but also the duration of attention. In one embodiment, the risk quantification assessment model can also acquire the vehicle's driving direction, the driver's line of sight direction, and the field of vision range. Based on the driver's line of sight direction and field of vision range, it determines whether obstacles in the vehicle's driving direction are within the driver's field of vision and the duration of attention, and adjusts the collision risk based on the field of vision and the duration of attention.
[0036] When the first risk value is greater than the second risk value, a second operation suppression command is sent to the vehicle motion control system. An intervention reminder message is generated and output to the driver, which warns the driver of the high-risk consequences of their current operation and explains the reason for the system's operation suppression.
[0037] When the driver's gaze remains on the road ahead and the first risk value is less than or equal to the second risk value, the system responds to the driver's operation and transfers control of the vehicle to the driver; after the transfer is completed, a notification message confirming that the vehicle has been taken over is output to the driver.
[0038] The system continuously monitors the driver's gaze and the presence of persistent driver operation signals. When the driver's gaze returns to the road ahead and a similar type of driver operation signal still exists, the system reassesses the first risk value corresponding to the driver takeover and the second risk value corresponding to the autonomous driving system continuing to control the vehicle. If the reassessed first risk value is less than or equal to the second risk value, the operation suppression command is revoked, allowing the vehicle motion control system to respond to the driver's operation signal. For example, the system continuously monitors the driver's gaze and the persistence of operation signals at fixed intervals of 100ms to 500ms. Only when the driver's gaze stably returns to the road ahead (e.g., for ≥300ms) and a persistent operation signal of the same type, direction, and intensity as the previously suppressed operation is emitted, is a new round of risk assessment initiated. During the assessment phase, the system re-collects comprehensive driving data, including vehicle status, surrounding obstacles, and road environment, and calls the optimized scenario model to calculate the first risk value for driver takeover and the second risk value for the autonomous driving system maintaining control. If the first risk value is less than or equal to the second risk value, the system will trigger a human-machine confirmation loop. The system requires the driver to confirm the operation within 1.5 seconds via steering wheel buttons, voice, or continuous operation. Upon confirmation, the operation suppression command is immediately revoked, and control is smoothly transferred. If the first risk value is still greater than the second risk value, the suppression state is maintained and a secondary intervention reminder is triggered. This mechanism supports multi-round cycle detection, and all assessments are completed locally on the vehicle's onboard computing unit to ensure low latency and high reliability. At the same time, the data of each revocation operation is encrypted and stored for system optimization. This avoids the mechanical logic of "taking over upon return of sight" and realizes intelligent takeover decision-making that is "conscious, capable, and confirmed," significantly improving the dynamic adaptability of human-machine collaboration and operational safety.
[0039] The beneficial effects of this application are as follows: When a driver's operation signal is received, the driver's gaze is detected, and when the driver's gaze leaves the road ahead, a first operation suppression command is sent to the vehicle motion control system. This operation suppression command prevents the vehicle motion control system from responding to the driver's operation signal, thereby reducing the risk of unintentional mishandling. Even if the driver's gaze remains on the road ahead, a first risk value corresponding to the driver's takeover operation and a second risk value corresponding to the autonomous driving system continuing to control the vehicle are assessed. Control is only allowed to transfer when it is confirmed that takeover is safer. When the first risk value is greater than the second risk value, a second operation suppression command is sent to the vehicle motion control system, further avoiding the risk of unintentional takeover due to driver misjudgment. Furthermore, this application sends suppression commands to the entire control system, rather than controlling individual components, improving the comprehensiveness and coordination of overall vehicle risk prevention and control, and further enhancing the safety of vehicle control.
[0040] In one embodiment, step S101 can be implemented as follows: Step A1, acquiring a facial image of the driver; Step A2, locating multiple key feature points in the facial image based on a deep learning model, the key feature points including at least eye contour points, pupil center, eyebrows, nose contour points, and mouth corner contour points; Step A3, estimating the driver's real-time gaze direction based on the relative positional relationship between the located pupil center and a predetermined eye corner reference point; Step A4, combining the driver's head posture data and the gaze direction, determining whether the driver's visual focus has deviated from the road area in front of the vehicle.
[0041] In one embodiment, after step S102 above, the method may also be implemented as follows: in step B1, a reminder message is generated and output to the driver, the reminder message containing a textual description and correction guidance for the specific erroneous behavior that is currently suppressed; in step B2, at least one tactile alarm is triggered, the tactile alarm including seat vibration and / or seat belt pretensioning.
[0042] In one embodiment, step S103 can be implemented as follows: C1-C3: In step C1, while the driver's line of sight remains on the road ahead, current driving environment data including the vehicle's dynamic parameters and the status of surrounding obstacles is acquired; in step C2, the current driving environment data is substituted into two different predefined scenario models: driver takeover control and automatic driving system maintenance control; in step C3, based on the risk quantification assessment model, the risk probability values of a collision occurring within a future preset time window under the two scenarios are calculated and output respectively, to obtain the first risk value and the second risk value.
[0043] In one embodiment, after step S104 above, the method may also be implemented as follows: generating and outputting intervention reminder information to the driver, the intervention reminder information being used to warn the driver of the high-risk consequences of the current operation and to explain the reason for the system's operation inhibition.
[0044] In one embodiment, the method may also be implemented as follows: in step D1, when the driver's gaze is not taken off the road ahead and the first risk value is less than or equal to the second risk value, the driver's operation is responded to and the control of the vehicle is switched to the driver; in step D2, after the switch is completed, a notification message confirming that the vehicle has been taken over is output to the driver.
[0045] In one embodiment, the method may also be implemented as follows: in step E1, the driver's gaze state and the presence of a continuous driver operation signal are cyclically detected; in step E2, when it is detected that the driver's gaze has returned to the road ahead and the same type of driver operation signal still exists, the first risk value corresponding to the driver takeover operation and the second risk value corresponding to the autonomous driving system continuing to control the vehicle are reassessed; in step E3, when the reassessed first risk value is less than or equal to the second risk value, the operation suppression command is revoked, allowing the vehicle motion control system to respond to the driver operation signal.
[0046] Figure 5 is a schematic diagram of a misoperation prevention control device according to an embodiment of this application. As shown in Figure 5, the device includes: a first detection module 501, used to detect the driver's gaze when a driver operation signal is received; a first sending module 502, used to send a first operation suppression command to the vehicle motion control system when the driver's gaze leaves the road ahead, the operation suppression command being used to prohibit the vehicle motion control system from responding to the driver operation signal; a first evaluation module 503, used to evaluate a first risk value corresponding to the driver's takeover operation and a second risk value corresponding to the autonomous driving system continuing to control the vehicle when the driver's gaze has not left the road ahead; and a second sending module 504, used to send a second operation suppression command to the vehicle motion control system when the first risk value is greater than the second risk value.
[0047] In one embodiment, the detection module includes: a first acquisition submodule for acquiring a driver's facial image; a localization submodule for locating multiple key feature points in the facial image based on a deep learning model, wherein the key feature points include at least eye contour points, pupil centers, eyebrows, nose contour points, and mouth corner contour points; an estimation submodule for estimating the driver's real-time gaze direction based on the relative positional relationship between the located pupil center and a predetermined eye corner reference point; and a judgment submodule for determining whether the driver's visual focus has deviated from the road area in front of the vehicle by combining the driver's head posture data and the gaze direction.
[0048] In one embodiment, the device further includes: a first generation module for generating and outputting reminder information to the driver, the reminder information including a textual description and correction guidance for the specific erroneous behavior currently suppressed; and a triggering module for triggering at least one tactile alarm, the tactile alarm including seat vibration and / or seat belt pretensioning.
[0049] In one embodiment, the evaluation module includes: a second acquisition submodule, configured to acquire current driving environment data including the vehicle's dynamic parameters and the status of surrounding obstacles while the driver's line of sight remains on the road ahead; a substitution submodule, configured to substitute the current driving environment data into two different predefined scenario models: driver takeover control and autonomous driving system maintenance control; and a calculation submodule, configured to calculate and output the risk probability values of a collision occurring within a preset time window under the two scenarios based on a risk quantification assessment model, thereby obtaining the first risk value and the second risk value.
[0050] In one embodiment, the device further includes: a second generation module, configured to generate and output intervention reminder information to the driver, the intervention reminder information being used to warn the driver of the high-risk consequences of their current operation and to explain the reason for the system's operation suppression.
[0051] In one embodiment, the device includes: a response module, configured to respond to a driver's operation and switch control of the vehicle to the driver when the driver's gaze is not taken off the road ahead and the first risk value is less than or equal to the second risk value; and a confirmation module, configured to output a notification message to the driver confirming that the vehicle has been taken over after the switch is completed.
[0052] In one embodiment, the device further includes: a second detection module for cyclically detecting the driver's gaze state and whether there is a continuous driver operation signal; a second evaluation module for re-evaluating a first risk value corresponding to the driver takeover operation and a second risk value corresponding to the autonomous driving system continuing to control the vehicle when the driver's gaze has returned to the road ahead and the same type of driver operation signal still exists; and a cancellation module for canceling the operation suppression command when the re-evaluated first risk value is less than or equal to the second risk value, allowing the vehicle motion control system to respond to the driver operation signal.
[0053] Figure 6 is a schematic diagram of the hardware structure of a misoperation prevention control system according to an embodiment of this application. As shown in Figure 6, the misoperation prevention control system includes: at least one processor 620; and a memory 604 communicatively connected to the at least one processor 620; wherein, the memory 604 stores instructions that can be executed by the at least one processor 620, and the instructions are executed by the at least one processor 620 to implement the misoperation prevention control method described in any of the above embodiments.
[0054] Referring to FIG6, the anti-misoperation control system 600 may include one or more of the following components: processing component 602, memory 604, power supply component 606, input / output (I / O) interface 608, sensor component 610, and communication component 612.
[0055] Processing component 602 typically controls the overall operation of the anti-misoperation control system 600. Processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-described method. Furthermore, processing component 602 may include one or more modules to facilitate interaction between processing component 602 and other components. The processor 620 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.
[0056] Memory 604 is configured to store various types of data to support the operation of the misoperation prevention control system 600. Examples of this data include instructions for any application or method operating on the misoperation prevention control system 600. Memory 604 can be an internal storage unit of the terminal device, such as a hard disk or memory of the terminal device. Memory 604 can also be an external storage device of the terminal device, such as a plug-in hard disk equipped on the terminal device. Memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Memory 604 is used to store programs and data required by this application. Memory 604 can also be used to temporarily store data that has been output or will be output.
[0057] Power supply component 606 provides power to various components of the anti-misoperation control system 600. Power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the anti-misoperation control system 600.
[0058] I / O interface 608 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc.
[0059] The sensor assembly 610 includes one or more sensors for providing status assessments of various aspects of the misoperation prevention control system 600. Additionally, the sensor assembly 610 can detect the on / off state of the misoperation prevention control system 600, the relative positioning of components, and the operational status of the misoperation prevention control system 600 or a component of it. In some embodiments, the sensor assembly 610 may include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor, etc.
[0060] Communication component 612 is configured to enable the anti-misoperation control system 600 to provide wired or wireless communication capabilities with other devices and cloud platforms. The anti-misoperation control system 600 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 616 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
[0061] In an exemplary embodiment, the anti-misoperation control system 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the anti-misoperation control method described in any of the above embodiments.
[0062] This application also provides a computer-readable storage medium, which, when the instructions in the storage medium are executed by the processor corresponding to the anti-misoperation control system, enables the anti-misoperation control system to implement the anti-misoperation control method described in any of the above embodiments.
[0063] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.
[0064] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.
[0065] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.
[0066] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.
[0067] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A method for preventing accidental disconnection control, characterized in that, include: When a driver's operation signal is received, the driver's line of sight is detected; When the driver's gaze leaves the road ahead, a first operation suppression command is sent to the vehicle motion control system. The operation suppression command is used to prevent the vehicle motion control system from responding to the driver's operation signal. When the driver's eyes are not off the road ahead, assess the first risk value corresponding to the driver taking over and the second risk value corresponding to the autonomous driving system continuing to control the vehicle. When the first risk value is greater than the second risk value, a second operation suppression command is sent to the vehicle motion control system.
2. The method for preventing misoperation control as described in claim 1, characterized in that, The detection of the driver's gaze includes: acquiring a facial image of the driver; locating multiple key feature points in the facial image based on a deep learning model, the key feature points including at least eye contour points, pupil centers, eyebrows, nose contour points, and mouth corner contour points; estimating the driver's real-time gaze direction based on the relative positional relationship between the located pupil center and a predetermined eye corner reference point; and determining whether the driver's visual focus has deviated from the road area in front of the vehicle by combining the driver's head posture data and the gaze direction.
3. The method for preventing misoperation control as described in claim 1, characterized in that, After sending a first operation suppression command to the vehicle motion control system, the method further includes: generating and outputting a reminder message to the driver, the reminder message containing a textual description and correction guidance for the specific erroneous operation currently suppressed; and simultaneously triggering at least one tactile alarm, the tactile alarm including seat vibration and / or seat belt pretensioning.
4. The method for preventing misoperation control as described in claim 1, characterized in that, Assessing the first risk value corresponding to driver takeover and the second risk value corresponding to continued vehicle control by the autonomous driving system includes: acquiring current driving environment data containing the vehicle's dynamic parameters and the status of surrounding obstacles while the driver's gaze remains on the road ahead; substituting the current driving environment data into two different predefined scenario models—one for driver takeover control and the other for continued control by the autonomous driving system—to obtain driving data corresponding to the two scenarios; and calculating and outputting the probability values of collisions occurring within a future preset time window based on the risk quantification assessment model, thereby obtaining the first risk value and the second risk value.
5. The method for preventing misoperation control as described in claim 4, characterized in that, After the step of sending a second operation suppression command to the vehicle motion control system when the first risk value is greater than the second risk value, the method further includes: generating and outputting intervention reminder information to the driver, wherein the intervention reminder information is used to warn the driver of the high-risk consequences of the current operation and explain the reason for the system to perform operation suppression.
6. The method for preventing misoperation control as described in claim 1, characterized in that, The method includes: when the driver's gaze is not taken off the road ahead and the first risk value is less than or equal to the second risk value, responding to the driver's operation and switching control of the vehicle to the driver; after the switch is completed, outputting a notification message to the driver confirming that the vehicle has been taken over.
7. The method for preventing misoperation control as described in claim 1, characterized in that, The method further includes: cyclically detecting the driver's gaze state and whether there are continuous driver operation signals; when it is detected that the driver's gaze has returned to the road ahead and the same type of driver operation signal still exists, re-evaluating the first risk value corresponding to the driver takeover operation and the second risk value corresponding to the autonomous driving system continuing to control the vehicle; when the re-evaluated first risk value is less than or equal to the second risk value, canceling the operation suppression command and allowing the vehicle motion control system to respond to the driver operation signal.
8. A device for preventing misoperation of control pipes, characterized in that, include: The first detection module is used to detect the driver's line of sight when it receives a driver operation signal; The first sending module is used to send a first operation suppression command to the vehicle motion control system when the driver's gaze is taken off the road ahead. The operation suppression command is used to prevent the vehicle motion control system from responding to the driver's operation signal. The first assessment module is used to assess the first risk value corresponding to the driver's takeover operation and the second risk value corresponding to the autonomous driving system continuing to control the vehicle when the driver's eyes are not off the road ahead. The second sending module is used to send a second operation suppression command to the vehicle motion control system when the first risk value is greater than the second risk value.
9. A control system for preventing accidental disconnection, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to implement the anti-mistakeover control method as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor corresponding to the anti-misoperation control system, the anti-misoperation control system is able to implement the anti-misoperation control method as described in any one of claims 1-7.