Vehicle driver incapacitation emergency control system and method

Abstract

The present application relates to the field of vehicle electronic control and active safety technology, and proposes a kind of vehicle driver incapacitation emergency control system and method.It includes: through driver monitoring camera and processing unit, based on eyelid closure degree and driving operation signal joint determination driver incapacitation, automatically trigger emergency response state;In response to the state, wake up forward perception unit to identify lane line and shoulder area;Local decision unit executes a finite state machine, sequentially executes lighting double flash warning light, speed gradually reduces and lane keeps based on lane line detection, and after vehicle speed reduces to threshold, guide vehicle to right side safe shoulder lane change and brake parking, while sending alarm message to terminal server in parallel to prepare subsequent remote manual takeover.The present application can fundamentally solve the timeliness and reliability problem of emergency control after driver sudden incapacitation without relying on driver active operation and remote communication large cycle, and can be deployed only with low-cost embedded hardware.

Description

Technical Field

[0001] This invention relates to the field of vehicle electronic control and active safety technology, and in particular to a vehicle driver incapacitation emergency control system and method. Background Technology

[0002] With the rapid development of autonomous driving technology, intelligent connected vehicles have gradually become an important development direction for the automotive industry. Some intelligent driving vehicles have already achieved highly accurate and reliable intelligent driving or intelligent assisted driving. However, a large number of existing vehicles still lack the conditions for implementing intelligent driving. How to achieve intelligent assistance in these existing vehicles at low cost, especially when drivers experience sudden illness, loss of consciousness, or other disabilities, where drivers often instantly lose consciousness and the ability to act, is a very important research direction for achieving emergency avoidance with low modification costs.

[0003] Existing technologies have proposed a scenario where, in the event of a driver's sudden illness, vehicle status and road condition video are transmitted to a remote control center via an in-vehicle network, allowing a remote operator to take over and control the vehicle to ensure safety.

[0004] However, this existing concept suffers from fundamental and insurmountable inherent flaws in its system architecture. First, it presents a trigger paradox: the solution heavily relies on the driver actively pressing a button to initiate a takeover request at the moment of illness. However, in real-world sudden disability scenarios such as acute myocardial infarction or stroke, the driver often instantly loses consciousness and mobility, rendering the safety mechanism ineffective. Second, it suffers from fatal latency: the solution's end-to-cloud-to-human loop of "on-board data collection → cloud upload → remote personnel observation and decision-making → issuing control commands → vehicle execution" introduces accumulated latency in perception, transmission, human reaction, and execution. In critical situations at high speeds, a delay of hundreds of milliseconds can lead to loss of control and missing the optimal window for avoidance. These flaws render the solution impractical in terms of functional safety. Summary of the Invention

[0005] To address the timeliness and reliability issues of emergency control in the event of sudden driver incapacitation when driving existing vehicles that lack the conditions for implementing intelligent driving, this invention discloses a vehicle driver incapacitation emergency control system and method, which solves the cost and local implementation problems of vehicle safety assistance and vehicle takeover in the event of driver incapacitation.

[0006] A vehicle driver incapacity emergency control system includes a driver status monitoring and incapacity determination module, an environmental perception module, and a local emergency driving decision-making and execution module; The driver status monitoring and disability determination module is configured to: collect the driver's biometric signals in real time, and automatically trigger an emergency response state when the driver is determined to be in a disabled state based on a preset disability determination logic. The environmental perception module is configured to be activated in response to the triggering of the emergency response state, acquire image information of the vehicle's external environment, and at least identify lane lines and drivable areas; The local emergency driving decision and execution module is configured to: in response to the emergency response state, execute a predefined finite state machine sequence, the finite state machine sequence including at least: a first state, generating and executing a vehicle warning signal; a second state, performing lane keeping operation based on the lane lines and generating a speed reduction control command; and a third state, when the vehicle speed drops to a preset threshold and the environmental perception module confirms that there is a safe parking area in the vehicle's driving direction, generating and sending a parking command with a specified trajectory to the actuator, so that the vehicle automatically parks in the safe parking area.

[0007] In the above or some embodiments, the driver status monitoring and disability determination module is specifically configured to: acquire driver facial images through an infrared auxiliary camera and calculate eyelid closure degree; when the eyelid closure degree exceeds a first threshold and the duration exceeds a second threshold, determine that the driver is in a disabled state.

[0008] In the above or some embodiments, the driver state monitoring and disability determination module is further configured to: jointly evaluate the steering wheel angle sensor signal and / or pedal displacement sensor signal when the eyelid closure degree is in a fuzzy range between a first threshold and a lower threshold; and only determine that the driver is in a disabled state when the steering wheel or pedal is not effectively operated within a preset time window at the same time.

[0009] In the above or some embodiments, the environment perception module is configured to: run a lightweight semantic segmentation model to perform pixel-level classification of the image in front of the vehicle to distinguish between paved roads and solid shoulders; the safe stopping area specifically refers to a continuous area that is classified as a solid shoulder, has a width sufficient to accommodate the vehicle, and has no negative obstacles or moving targets detected outside its boundary.

[0010] In the above or some embodiments, the lightweight semantic segmentation model uses depthwise separable convolution as the encoder backbone network, the number of model parameters does not exceed 10M, and it is deployed on the environment perception module through INT8 quantization.

[0011] In the above or some embodiments, the local emergency driving decision and execution module is further configured to: if the safe stopping area cannot be confirmed within a preset timeout period during the execution of the third state, then forcibly switch to a minimum risk state and perform braking and stopping within the current driving lane defined by the lane line.

[0012] In the above or some embodiments, a communication and remote handover module is also included, configured to: at the moment the emergency response state is triggered, send an alarm message to a preset terminal server in parallel with the execution of the finite state machine sequence; the alarm message includes at least: vehicle identification, location information and the triggering reason code of the emergency response state.

[0013] In the above or some embodiments, the communication and remote handover module is further configured to: receive and verify a remote takeover request from the terminal server after the vehicle has been safely parked, and after successful verification, grant remote control access to the vehicle and access to a high-bandwidth data stream channel.

[0014] In the above or some embodiments, the communication between the local emergency driving decision and execution module and the vehicle actuator is completed via the vehicle CAN bus; the CAN message corresponding to the control command issued by the module is configured to have a higher priority than the message corresponding to the driver's control input, so as to realize arbitration coverage of the driver's command in case of conflict.

[0015] A method for emergency control of driver incapacitation in vehicles, the method comprising: The driver status monitoring and disability assessment module automatically determines that the driver is disabled and triggers an emergency response state. In response to the emergency response state, the environmental perception module is activated and begins to identify lane lines and drivable areas; Simultaneously, the local emergency driving decision and execution module activates a finite state machine to execute vehicle warnings, lane-line-based speed reduction, and lane keeping in parallel. Once the vehicle speed drops to a preset threshold and the environmental perception confirms a safe stopping area, the system automatically guides the vehicle to change lanes and stop in the safe stopping area.

[0016] The beneficial effects of this application are:

[0017] First, the dual verification mechanism significantly reduces the false alarm rate. By logically ANDing abnormal driver biometrics with abnormal vehicle control status, the false alarm rate is significantly reduced compared to a single biometric monitoring scheme.

[0018] Second, the on-demand wake-up mechanism significantly reduces computing power consumption and costs. The environmental awareness module is only woken up when an emergency response state is triggered, greatly reducing computing power requirements and meeting the requirements for low-cost installation and use.

[0019] Third, finite state machine design improves control reliability. By adopting a deterministic finite state machine architecture, the transition conditions of each state are clear and verifiable, avoiding the uncertainty brought about by complex decision trees, and significantly improving the stability and predictability of the system.

[0020] Fourth, the parallel communication mechanism improves response speed and communication reliability. By adopting a mechanism of alarm first and then link establishment, an alarm message with a very small amount of data is sent immediately at the moment an emergency response is triggered. Under the 5G network, the end-to-end latency can be controlled within 10ms, and the risk of communication failure is small.

[0021] Fifth, a dual safety mechanism ensures system reliability. By combining local autonomous risk avoidance and remote takeover, a dual safety guarantee is formed. Even in extreme cases where remote communication is completely interrupted, the vehicle can autonomously complete basic risk avoidance and avoid loss of control accidents.

[0022] Sixth, deployment and application can be completed at low cost. The system is set up with four core functional modules, which exchange data efficiently through the vehicle's CAN bus to form a complete emergency control closed loop. This application can be rapidly deployed and applied to existing vehicles that cannot achieve intelligent driving at extremely low cost. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the overall architecture of the vehicle driver incapacitation emergency control system of the present invention.

[0024] Figure 2 This is a schematic diagram of the workflow of the driver status monitoring and disability determination module of the present invention.

[0025] Figure 3 This is a flowchart illustrating the emergency control method for vehicle driver incapacitation according to the present invention. Detailed Implementation

[0026] The technical solutions provided by the present invention will now be described in more detail with reference to the accompanying drawings. The descriptions of exemplary embodiments are merely illustrative and are in no way intended to limit this disclosure or its application or use. This disclosure can be implemented in many different forms and is not limited to the embodiments described herein. These embodiments are provided to make this disclosure thorough and complete, and to fully express the scope of this disclosure to those skilled in the art. It should be noted that, unless otherwise specifically stated, the relative arrangement of components and steps, the composition of materials, numerical expressions, and values ​​set forth in these embodiments should be interpreted as merely exemplary and not as limiting.

[0027] The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different parts. Terms such as "comprising" or "including" mean that the element preceding the word covers the element listed after the word, and do not exclude the possibility of covering other elements as well.

[0028] Example 1: Emergency control system for vehicle driver incapacitation.

[0029] like Figure 1 As shown, this embodiment provides a vehicle driver incapacity emergency control system 100. The system includes four core functional modules: a driver status monitoring and incapacity determination module 101, an environmental perception module 102, a local emergency driving decision-making and execution module 103, and a communication and remote handover module 104. Each module interacts with the other via the vehicle's CAN bus, forming a complete emergency control closed loop.

[0030] The driver status monitoring and disability assessment module 101 includes an infrared auxiliary camera, an image processing unit, and a disability assessment logic unit. The infrared auxiliary camera is mounted above the dashboard or in the center of the steering wheel to capture images of the driver's face. The image processing unit uses a deep learning-based facial landmark detection algorithm to calculate physiological feature parameters such as eyelid closure, gaze direction, and head posture in real time. The disability assessment logic unit has a complete eyelid closure threshold of 80%-100% and a fuzzy range threshold of 60%-80%. When the eyelid closure exceeds the complete eyelid closure threshold, the driver is directly assessed as disabled; when the eyelid closure is within the fuzzy range threshold, the disability assessment logic unit further reads vehicle control status data for a comprehensive assessment.

[0031] Vehicle control status data is acquired from the vehicle chassis control system via the CAN bus, including steering wheel angle sensor data, accelerator pedal pressure sensor data, and brake pedal pressure sensor data. The disability determination logic unit has a preset time window of 1-5 seconds; in this embodiment, a preset time window is preferably 3 seconds. Within this time window, if no valid operation is detected—either a steering wheel angle greater than 5 degrees or a pedal pressure change exceeding 10%—and the eyelid closure degree is within the fuzzy threshold range, the driver is determined to be in a disabled state. This dual verification mechanism effectively reduces false triggering caused by normal driving behaviors such as briefly closing the eyes or turning the head to look in the rearview mirror.

[0032] The environmental perception module 102 includes a forward-facing wide-angle camera, an embedded computing unit, and a lightweight semantic segmentation model. The forward-facing wide-angle camera is mounted in front of the rearview mirror or at the center of the top of the windshield, with a resolution of 1920×1080 pixels, a frame rate of 30fps, and a field of view of 120 degrees, capable of covering the road environment in front of and to the sides of the vehicle. The embedded computing unit adopts an edge computing architecture, such as an NVIDIA Jetson Nano or a chip of equivalent performance. The embedded computing unit is an automotive-grade neural network processor with computing power ranging from 1 to 4 TOPS (INT8).

[0033] After the environmental perception module 102 is activated, it collects image information of the vehicle's external environment, runs a lightweight semantic segmentation model, and performs pixel-level classification on the image in front of the vehicle to distinguish between paved roads and solid shoulders. The safe parking area specifically refers to a continuous area that is classified as a solid shoulder, has a width sufficient to accommodate the vehicle, and has no negative obstacles or moving targets detected outside its boundary.

[0034] In the above or some embodiments, the lightweight semantic segmentation model adopts an encoder-decoder structure. The encoder backbone network uses MobileNetV3-Small, and its core convolutional module is a depthwise separable convolution. The decoder adopts an LR-ASPP structure, which contains only one 1×1 convolution and two 3×3 depthwise separable convolutional branches with different dilation rates.

[0035] Lightweight semantic segmentation model training data and annotations. The training dataset contains no fewer than 100,000 images collected from real-world road scenes, covering various road types such as highways, national roads, urban roads, and rural roads, as well as different lighting and weather conditions such as sunny days, cloudy days, rainy days, and dusk. Each image is semantically annotated at the pixel level, with annotation categories including at least: paved road surface, solid shoulder (including compacted dirt roads and gravel roads), non-drivable areas (including guardrails, walls, ditches, water bodies, and steep slopes), moving objects (including vehicles, pedestrians, and cyclists), lane lines, and sky. The boundary annotations for "solid shoulder" and "non-drivable areas" were strictly drawn by trained annotators according to drivability standards and underwent double review.

[0036] Lightweight semantic segmentation model training. Training was conducted using the PyTorch framework, with a weighted combination of cross-entropy loss and Dice loss as the loss function. The optimizer was Adam W, with an initial learning rate of 0.001, and cosine annealing was used for learning rate decay. Data augmentation strategies such as random horizontal flipping, random brightness / contrast adjustment, and random cropping were employed during training.

[0037] Lightweight semantic segmentation model quantization and deployment. The trained FP32-precision model is converted to INT8 precision using post-training quantization. A quantization calibration dataset of 1000 representative images is randomly selected from the training set. The quantized model is exported in ONNX format and deployed on an embedded computing unit using TVM or NCNN inference frameworks. This is done on an automotive-grade NPU with a typical computing power of 2 TOPS (INT8).

[0038] The local emergency driving decision and execution module 103 adopts a finite state machine architecture, containing three sequential states: warning phase, lane keeping and deceleration phase, and parking phase. This module sends control commands to the body control module (BCM), braking system ECU, and electric power steering system (EPSECU) via the CAN bus. The CAN identifiers for all emergency control commands are set in the range of 0x100-0x10F to ensure the highest priority in CAN bus arbitration, covering both driver steering and braking inputs.

[0039] The communication and remote handover module 104 includes an onboard communication unit (TCU) and a remote takeover processing unit. The onboard communication unit supports 5G / 4G wireless communication. Upon triggering a driver incapacity assessment, it sends an alarm message (no more than 100 bytes) to the terminal server via UDP protocol. This alarm message includes key information such as the vehicle's unique identification number (VIN), precise GPS coordinates, incapacity trigger cause code, current vehicle control status, vehicle speed, and timestamp. Under 5G networks, the UDP single-packet transmission latency can be controlled to within 10ms. To improve reliability, the alarm message is retransmitted three times within the first second after triggering, with an interval of approximately 300ms. Upon receiving the alarm message, the terminal server of the remote takeover processing unit immediately replies with an acknowledgment message and performs verification, discarding messages that fail verification. After receiving the alarm message, the terminal server immediately assigns an available remote technician and pushes the VIN, precise GPS coordinates, etc., to the remote technician's terminal control module. The remote technician can view the vehicle status, but typically does not intervene until local emergency evacuation is completed, unless the local evacuation strategy is deemed inappropriate and mandatory intervention is required. Once the vehicle has parked, the system verifies that the vehicle speed is 0 for more than 1 second, the parking brake is activated, and the vehicle is in P gear. When all three conditions are met, a high-bandwidth communication link is established via TCP protocol to transmit complete data such as video stream, vehicle status stream, and GPS location stream. The system then transfers the vehicle's steering, braking, and power control authority to the terminal server.

[0040] In the above or some embodiments, such as Figure 2 As shown, the specific implementation method for driver condition monitoring and disability assessment is as follows:

[0041] S201: An infrared auxiliary camera captures images of the driver's face. The infrared auxiliary camera also uses a near-infrared LED with a wavelength of 850nm as a supplementary light source, enabling clear facial image capture at night or in low-light conditions. After receiving the raw image data, the image processing unit first performs face detection, employing a fast detection algorithm based on Haar or HOG features to locate the face region.

[0042] S201: Facial Key Point Detection and Feature Extraction. After detecting a face, the image processing unit further locates 68 or 106 facial key points, including the positions of the eyes, nose, mouth, eyebrows, and facial contours. The eyelid closure degree is calculated using the EAR algorithm for eye aspect ratio; the specific calculation formula is as follows:

[0043] in to This provides the coordinates of six key points for each eye. When the average EAR (Eye Area Value) for both eyes is less than 0.2, the eyelids are considered closed; when the average EAR is greater than 0.3, the eyelids are considered open; and when the average EAR is between 0.2 and 0.3, the eyelids are considered blurred. The system maps the average EAR to a percentage of eyelid closure, where 0% represents fully open and 100% represents fully closed.

[0044] S203: Eyelid Closure Threshold Determination. The disability determination logic unit sets a complete eyelid closure threshold of 80%-100%. When the eyelid closure exceeds this threshold and lasts for more than 1.5 seconds, the driver is directly determined to be in a disabled state, triggering an emergency response. This determination logic corresponds to the extreme case of the driver's eyes being completely closed or unconscious, and disability can be confirmed without additional verification.

[0045] S204: Joint Judgment of Fuzzy Intervals. When the eyelid closure degree is within the fuzzy interval threshold of 60%-80%, the disability judgment logic unit initiates the joint judgment process. This fuzzy interval corresponds to the driver's state of half-closed eyes, squinting, or brief eye closure, which may be caused by normal driving behaviors such as blinking or turning the head to look in the rearview mirror, or abnormal states such as fatigue or confusion. It is necessary to make a comprehensive judgment in conjunction with the vehicle control status.

[0046] The disability assessment logic unit reads vehicle control status data via the CAN bus, including steering wheel angle, accelerator pedal pressure, and brake pedal pressure. The system has a preset time window of 3 seconds. Within this window, if a change in steering wheel angle greater than 5 degrees or a change in pedal pressure exceeding 10% is detected, it is considered a valid operation, indicating the driver is in a normal state, and disability assessment is not triggered. If no valid operation is detected within the preset time window, the driver is deemed to have lost normal control of the vehicle. Combined with the fact that the eyelid closure is within an ambiguous range, the system comprehensively determines that the driver is in a disabled state, triggering an emergency response.

[0047] S205: Assisted Judgment of Vehicle Lateral Control Instability. As an optional auxiliary judgment condition, the system can also analyze the vehicle's lateral control stability. Lane line detection is performed using a forward-facing camera, and the vehicle's lateral offset is calculated. The system defines "lane crossing" as any wheel of the vehicle crossing the lane line for more than 0.5 seconds. The number of lane crossings in the past 60 seconds is counted. When the number of lane crossings is greater than or equal to 3, it is judged as "frequent lane crossing," indicating lateral control instability. This auxiliary judgment condition can be used in conjunction with eyelid closure and vehicle control status data to further improve the accuracy of disability assessment.

[0048] The entire disability assessment process has a delay of less than 200ms, and the total delay from the occurrence of an anomaly to the triggering of an emergency response is controlled within 500ms, ensuring that the system can respond quickly to the driver's disability status.

[0049] In the above or some embodiments, the environmental perception module adopts an on-demand wake-up mechanism, remaining in a dormant state under normal driving conditions and only being woken up after a driver incapacity determination is triggered. The specific implementation process is as follows.

[0050] Step 1: Module Wake-up. When the driver status monitoring and disability assessment module triggers the "emergency response state" signal, this signal is transmitted to the environmental perception module via an internal hardware interrupt or a high-priority software signal. The embedded computing unit immediately switches from sleep mode to full-speed operation mode, with a wake-up delay of less than 10ms. After wake-up, the processing unit immediately begins reading the video stream from the forward-facing camera and starts the lightweight semantic segmentation model.

[0051] Step 2: Image Preprocessing. The embedded computing unit preprocesses the raw image captured by the forward-facing camera, including: 1. Cropping: Only retaining the region of interest (ROI), such as the road area in the lower half of the image, and removing irrelevant areas such as the sky and buildings; 2. Scaling: Reducing the image resolution from 1920×1080 to 640×360 to speed up processing; 3. Normalization: Normalizing the pixel values ​​to the range of 0-1 to adapt to the input requirements of the neural network.

[0052] Step 3: Lightweight Semantic Segmentation. The lightweight semantic segmentation model uses MobileNetV3-Small as its backbone network to extract image features. The feature maps output by the backbone network are processed by a lightweight decoder, a simplified version based on DeepLabV3+, employing depthwise separable convolutions and dilated convolutions for upsampling and feature fusion to generate pixel-level semantic segmentation results. The model output includes the following categories: asphalt pavement, cement, solid shoulders, grass, soil, obstacles, lane lines, etc. The model has 8.2M parameters and achieves an inference speed of 33fps on an embedded AI chip, with a single-frame processing latency of approximately 30ms, meeting real-time requirements.

[0053] Step 4: Lane detection. The system simultaneously runs a lane detection algorithm, employing a fast detection method based on Hough transform or deep learning. The lane detection results are used to calculate the vehicle's lateral offset and yaw angle relative to the lane centerline, providing a basis for subsequent lane keeping control. The lane detection algorithm identifies lane line types, such as solid lines, dashed lines, and double yellow lines, to determine whether a lane change is permitted.

[0054] Step 5: Drivable Area Identification. Based on semantic segmentation results, the system identifies the drivable area to the right of the vehicle. The criteria for determining the drivable area include: 1. Road surface material: Solid surfaces such as asphalt and cement are considered drivable, while soft surfaces such as mud and grass are considered impassable or high-risk; 2. Area width: The width should be greater than the vehicle width plus a 0.5-meter safety margin. For example, since the vehicle width is generally 1.8-2.0 meters, the drivable area width should be greater than 2.3-2.5 meters; 3. No obstacles: The system uses target detection algorithms to identify whether there are static obstacles such as parked vehicles, trash cans, and bollards, or dynamic targets such as pedestrians and bicycles in the shoulder area; 4. No significant slope: By analyzing road surface height information, and optionally configuring millimeter-wave radar or binocular cameras to obtain depth information, the system determines that the slope of the shoulder area is less than 10 degrees; 5. No road surface depressions or obstacles below the road surface: The system determines whether there are depressions, ditches, or puddles with a depth greater than 10 cm in the shoulder area.

[0055] Step 6: Output drivable area information. The system outputs a Boolean value (True / False) indicating whether a safe drivable area exists on the right, and the location information of this area relative to the vehicle's lateral and longitudinal distances. This information is transmitted to the local emergency driving decision and execution module, which decides whether the vehicle should pull over or brake to a stop within its original lane.

[0056] In the above or some embodiments, the local emergency driving decision-making and execution module adopts a finite state machine architecture, which includes three sequential states: warning stage, lane keeping and deceleration stage, and parking stage. The specific implementation process is as follows:

[0057] State 1: Warning Phase. Entry condition is receiving an "Emergency Response State" signal. The system sends a control command to the Body Control Module (BCM) via the CAN bus to illuminate the vehicle's hazard warning lights. The CAN message format is: CAN ID 0x100, ensuring the highest priority; the first byte of the data field is 0x01, indicating that the hazard lights are activated. After the hazard lights are illuminated, the system can optionally activate in-vehicle audible and visual alarm devices, such as a buzzer or voice prompt, to alert the driver and passengers that the vehicle is in an abnormal state. This state lasts approximately 0.5 seconds. Exit condition is completion of the hazard warning lights illuminating, specifically confirmed by reading the feedback signal from the BCM via the CAN bus, or forced entry into the next state after a 0.5-second timeout.

[0058] State 2: Lane Keeping and Deceleration Phase. Entry conditions are derived from State 1. This state includes two parallel subtasks: longitudinal control caused by deceleration and lateral control caused by lane keeping.

[0059] Longitudinal control: The control objective is to gradually reduce vehicle speed at a constant deceleration of -0.2g, approximately -1.96 m / s², to avoid rear-end collisions or passenger injuries caused by sudden braking. The system sends the target deceleration command to the braking system ECU via the CAN bus. The CAN message format is: CAN ID 0x101, ensuring high priority. The data field contains the target deceleration value in units of 0.01 m / s², with a value range of 0-255, corresponding to a target deceleration of 0-2.55 m / s². Based on the target deceleration, the braking system ECU controls the braking pressure of each wheel through the Electronic Stability Program (ESP) or Electronic Brakeforce Distribution (EBD) system to achieve smooth deceleration. The control frequency is 50Hz, with a control command sent every 20ms to ensure smooth control.

[0060] Lateral Control: The control objective is to keep the vehicle centered in the current lane, avoiding lane departure or collisions with vehicles in adjacent lanes. The system uses a PID controller to calculate the vehicle's lateral offset (distance between the vehicle's centerline and the lane's centerline) and the yaw angle (angle between the vehicle's direction of travel and the lane direction) based on lane line detection results, and outputs the target steering angle. In this embodiment, the PID controller parameters are set as follows: proportional coefficient Kp = 0.5, integral coefficient Ki = 0.1, and derivative coefficient Kd = 0.2. Specific parameters need to be determined through actual vehicle testing based on vehicle characteristics. To avoid sharp turns, the rate of change of the target steering angle is limited to within 30 degrees / second, and the steering angle amplitude is limited to within ±10 degrees. These values ​​are applicable to lane-keeping scenarios where large steering angles are not required. The system sends the target steering angle command to the EPSECU steering system via the CAN bus. The CAN message format is: CAN ID 0x102, ensuring high priority. The data field contains the target steering angle value, in units of 0.1 degrees, with a range of -3600 to +3600 degrees. The target steering angle corresponds to -360 degrees to +360 degrees. The control frequency is 50Hz, synchronized with the longitudinal control.

[0061] State Duration: This state continues until the vehicle speed drops below the preset low-speed threshold of 40 km / h. For example, with an initial speed of 60 km / h and a deceleration of -0.2g, it takes approximately 2.8 seconds to decrease to 40 km / h, covering a distance of approximately 39 meters. With an initial speed of 100 km / h, it takes approximately 8.5 seconds to decrease to 40 km / h, covering a distance of approximately 170 meters. The exit condition is when the vehicle speed is less than 40 km / h, determined by reading the vehicle speed signal via the CAN bus.

[0062] Minimum Risk Strategy: To avoid the vehicle remaining in an unstable state for an extended period, such as when the environmental perception module fails to identify a safe driving area for an extended period or when the vehicle speed decreases too slowly, the system implements a timeout monitoring mechanism in State 2. When the execution time of this stage exceeds a preset timeout threshold, the system forcibly transitions to the minimum risk state. The timeout threshold can be selected from 5 to 30 seconds, with 15 seconds being preferred: braking to a stop within the original lane at a deceleration of 0.3-0.6g. The timeout determination logic is as follows: When the system enters State 2, a timer is started. The timer value is checked every 20ms in each control cycle. When the timer value exceeds the preset timeout threshold, the minimum risk state is immediately triggered. Minimum risk state execution: 1. Lateral control is canceled, and the vehicle maintains the current steering wheel angle, typically close to 0 degrees, i.e., driving straight; 2. Longitudinal control is switched to emergency braking mode, braking at a deceleration of 0.3-0.6g; 3. The hazard warning lights remain constantly on; 4. When the vehicle speed drops to 0, it automatically engages P gear and pulls the electronic parking brake.

[0063] State 3: Parking at the side of the road. This state transitions from State 2, and the vehicle speed has decreased to below 40 km / h. Different parking strategies are implemented based on the drivable area identification results from the environmental perception module.

[0064] Scenario A: A safe driving area exists on the right. The system controls the vehicle to smoothly change lanes to the right, approaching the shoulder area, and gradually decelerating until it comes to a complete stop as it approaches the curb. Lateral control uses a trajectory tracking controller to plan a smooth trajectory from the current lane centerline to the shoulder parking position. The trajectory is a cubic polynomial curve, starting at the current vehicle position and ending at the shoulder parking position. A lateral deviation of approximately 3-4 meters corresponds to a longitudinal distance of approximately 30-50 meters. During trajectory planning, the lateral acceleration is ensured to be less than 0.3g to avoid vehicle skidding or passenger discomfort. Longitudinal control continues to reduce the vehicle speed at a deceleration of -0.2g. When the vehicle is less than 5 meters from the curb, the deceleration increases to -0.4g to accelerate the stopping speed. When the vehicle speed drops below 5 km / h, the parking brake is switched on, and the vehicle comes to a complete stop. The parking position is approximately 0.3-0.5 meters from the curb on the right side of the vehicle, ensuring the vehicle is completely stopped within the shoulder area and does not obstruct the driving lane.

[0065] Scenario B: No safe driving area is found on the right or it is never detected. The system executes the minimum risk strategy, braking directly to a stop within the original lane without making a lateral lane change. Lateral control keeps the vehicle centered in the current lane and continues lane keeping control. Longitudinal control uses a deceleration of -0.3g, slightly greater than the deceleration in Scenario B, to accelerate the stop and reduce speed until the vehicle comes to a complete stop.

[0066] Duration of state 3: From entering state 3 to the vehicle coming to a complete stop, the typical time is 5-10 seconds, and the travel distance is 30-80 meters, depending on the initial speed and road conditions. The exit condition is when the vehicle comes to a complete stop, i.e., the speed is 0, and the parking brake is activated.

[0067] Post-parking procedures: After the vehicle comes to a complete stop, the system automatically engages P gear (parking gear), pulls the electronic parking brake, keeps the hazard warning lights on, and switches to "waiting for remote takeover" mode.

[0068] In the above or some embodiments, the communication and remote handover module adopts a parallel communication mechanism of "alarm first, link establishment later", such as Figure 3 As shown, the specific implementation process is as follows:

[0069] S301: Triggering Parallel Communication. When the driver status monitoring and disability assessment module triggers the "emergency response state" signal, this signal is simultaneously received by both the local emergency driving decision-making and execution module and the communication and remote handover module. The local emergency driving decision-making and execution module immediately enters the state-one warning stage and begins to execute emergency avoidance actions; the communication and remote handover module simultaneously begins assembling alarm messages and sending them to the terminal server via the 5G / 4G network. The startup time difference between the two modules is less than 10ms, and they can be considered to start simultaneously.

[0070] S302: Assemble alarm messages. Alarm messages are designed with extremely small data size, with a single packet size of approximately 40 bytes, strictly limited to no more than 100 bytes. The message content includes: 1. Vehicle Identifier (VIN): 17 bytes, used to uniquely identify the vehicle; 2. Precise GPS coordinates: 8 bytes each of longitude and latitude, double-precision floating-point numbers, with an accuracy of approximately 10 cm; 3. Disability trigger reason code: 1 byte, enumerated type, e.g., 0x01 = eyelid closure timeout, 0x02 = gaze deviation timeout, 0x03 = abnormal head posture, 0x04 = frequent line crossing, 0x05 = comprehensive judgment of disability; 4. Current vehicle control status: 1 byte, enumerated type, e.g., 0x01 = driver control, 0x02 = local autonomous control entering emergency avoidance, 0x03 = remote control, 0x04 = out of control; 5. Vehicle speed: 2 bytes, unit km / h, range 0-255; 6. Timestamp: 8 bytes, Unix timestamp, millisecond precision; 7. Checksum: 2 bytes, using the CRC-16 checksum algorithm. The total is approximately 40 bytes.

[0071] S303: Send alarm message. The alarm message is transmitted using the UDP (User Datagram Protocol) instead of the TCP (Transmission Control Protocol). UDP's advantages include low latency, simplicity, efficiency, and non-blocking. In 5G networks, the latency of a single UDP packet can be controlled to within 10ms. To improve reliability, the alarm message is retransmitted three times within the first second after triggering, with an interval of approximately 300ms, ensuring that at least one message successfully reaches the terminal server. Upon receiving the alarm message, the terminal server immediately replies with an ACK message. The vehicle-side device stops retransmitting after receiving the ACK. If the vehicle-side device does not receive an ACK within one second, it continues to retransmit the alarm message at a lower frequency every two seconds. The message contains a CRC-16 checksum, which the terminal server verifies upon receipt, discarding messages that fail the checksum verification.

[0072] S304: Remote Takeover Preparation. Upon receiving the alarm message, the terminal server immediately assigns an available remote technician and pushes vehicle information, including the VIN code, GPS location, and cause of the malfunction, to the remote technician's terminal control module. The remote technician can view the vehicle status, but typically does not intervene until local emergency evacuation is complete, unless the local evacuation strategy is deemed inappropriate and mandatory intervention is required.

[0073] S305: Independent execution of local hazard avoidance. The local emergency driving decision and execution module does not wait for feedback from the communication module and independently completes the entire emergency hazard avoidance process. That is, it adopts a finite state machine architecture, state one → state two → state three, to ensure that the vehicle can autonomously complete basic hazard avoidance even in the event of communication failure or remote response delay.

[0074] S306: Vehicle coming to a complete stop verification. After the vehicle has come to a complete stop, the system performs the following verifications: 1. The vehicle speed is 0 for more than 1 second; 2. The parking brake is activated, confirmed by reading the parking brake status signal via the CAN bus; 3. The vehicle is in P gear, confirmed by reading the gear position signal via the CAN bus. When all three conditions are met simultaneously, the system determines that the vehicle has come to a safe stop and can open the high-bandwidth communication channel.

[0075] S307: Establish a high-bandwidth communication link. The remote end sends a "Data Stream Establishment Request" message to the vehicle end. Upon receiving the request, the vehicle end establishes a TCP connection via the 5G / 4G network and begins transmitting complete data, including video streams from the forward-facing camera and driver monitoring camera, vehicle status streams (such as vehicle speed, steering angle, gear position, braking status, and battery level), and GPS location streams. The data stream establishment delay is approximately 1-3 seconds, primarily including the TCP handshake and video encoder startup.

[0076] S308: Transfer of Control. After the high-bandwidth communication link is established, the system transfers control of the vehicle's steering, braking, and power to the remote control center. Remote technicians can then use terminal control modules, including simulated steering wheel, pedals, and gearshift levers, to perform remote driving operations and execute follow-up tasks such as precise parking, emergency assistance, and passenger reassurance.

[0077] If the interaction relationships between the modules are displayed in a timeline format, the workflow of this solution in actual use is as follows:

[0078] At time T0, during normal driving: the driver status monitoring module continues to run, monitoring the driver's status; the environmental perception module is in sleep mode; the local emergency driving decision and execution module is in standby mode; and the communication module maintains a heartbeat connection with the remote terminal.

[0079] At time T1, driver incapacity was detected: The driver status monitoring module detected abnormal biometrics, such as eyelid closure greater than 50% for more than 1.5 seconds and frequent lane crossings indicating lateral control instability. Both verification conditions were met simultaneously, leading to a driver incapacity determination. The determination delay was less than 200ms.

[0080] At time T2, an emergency response is triggered (T2 = T1 + approximately 200ms): the system broadcasts an "emergency response status" signal internally. The environmental perception module is activated, with a activation delay of less than 10ms. The local emergency driving decision and execution module enters state one, the warning stage, and controls the hazard lights to illuminate via the CAN bus. The communication module assembles alarm messages and sends them to the terminal server via UDP protocol, with a transmission delay of less than 10ms.

[0081] At time T3, the vehicle enters state two (T3 = T2 + approximately 0.5 seconds): the hazard lights illuminate, and the local emergency driving decision-making and execution module enters state two, the lane keeping and deceleration phase. The environmental perception module begins lane line detection and drivable area identification, with a single-frame processing latency of less than 30ms. Longitudinal control reduces the vehicle speed with a deceleration of -0.2g, while lateral control maintains the vehicle centered in the lane via a PID controller.

[0082] At time T4, the remote terminal receives an alarm (T4 = T2 + approximately 10-50ms). The terminal server receives the alarm message (UDP transmission delay approximately 10ms; on a 5G network), and immediately replies with an ACK confirmation message. The terminal server assigns an available remote technician and pushes vehicle information to the technician's terminal control module. The remote technician checks the vehicle status and prepares to take over.

[0083] At time T5, the vehicle speed drops below 40 km / h. T5 = T3 + approximately 2-8 seconds: When the vehicle speed drops below the preset low-speed threshold of 40 km / h, the local emergency driving decision and execution module enters state three, the pullover stage. The environmental perception module outputs the drivable area identification result.

[0084] At time T6, the vehicle comes to a complete stop. T6 = T5 + approximately 5-10 seconds: The vehicle comes to a complete stop on the shoulder or in its original lane, automatically shifts into Park (P), engages the electronic parking brake, and keeps the hazard lights on. The system switches to "awaiting remote takeover" mode. The total time from T1 to T6 is approximately 10-20 seconds, and the travel distance is approximately 100-200 meters, depending on the initial speed and road conditions.

[0085] At time T7, a complete data stream is established. T7 = T6 + approximately 1-3 seconds: The remote terminal sends a "Data Stream Establishment Request" message to the vehicle terminal. Upon receiving the request, the vehicle terminal establishes a TCP connection and begins transmitting video streams, vehicle status streams, GPS location streams, etc. The data stream establishment delay is approximately 1-3 seconds.

[0086] At time T8, remote takeover occurs. T8 = T7 + approximately 5-30 seconds: Remote technicians take over vehicle control via a terminal control module to perform subsequent operations such as precise parking, emergency assistance, and passenger reassurance. The timing of the takeover is determined by the remote technician based on the actual situation and is not fixed.

[0087] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the inventive concept of this application, and these all fall within the protection scope of this application.

Claims

1. A vehicle driver incapacity emergency control system, comprising a driver status monitoring and incapacity determination module (101), an environmental perception module (102), and a local emergency driving decision-making and execution module (103), characterized in that: The driver status monitoring and disability determination module (101) is configured to: collect the driver's biometric signals in real time, and automatically trigger an emergency response state when the driver is determined to be in a disabled state based on the preset disability determination logic. The environmental perception module (102) is configured to be awakened in response to the triggering of the emergency response state, collect image information of the external environment of the vehicle, and at least identify lane lines and drivable areas; The local emergency driving decision and execution module (103) is configured to: in response to the emergency response state, execute a predefined finite state machine sequence, the finite state machine sequence including at least: a first state, generating and executing a vehicle warning signal; a second state, performing lane keeping operation based on the lane lines and generating a speed reduction control command; a third state, when the vehicle speed drops to a preset threshold and the environmental perception module (102) confirms that there is a safe parking area in the vehicle's driving direction, generating and sending a parking command with a specified trajectory to the actuator, so that the vehicle automatically parks in the safe parking area.

2. The system according to claim 1, characterized in that, The driver status monitoring and disability determination module (101) is specifically configured to: acquire driver facial images through an infrared auxiliary camera and calculate eyelid closure degree; when the eyelid closure degree exceeds a first threshold and the duration exceeds a second threshold, determine that the driver is in a disabled state.

3. The system according to claim 2, characterized in that, The driver status monitoring and disability determination module (101) is further configured to: jointly evaluate the steering wheel angle sensor signal and / or pedal displacement sensor signal when the eyelid closure degree is in the fuzzy range between a first threshold and a lower threshold; and only determine that the driver is in a disabled state when the steering wheel or pedal is not effectively operated within a preset time window at the same time.

4. The system according to claim 1, characterized in that, The environment perception module (102) is configured to run a lightweight semantic segmentation model to perform pixel-level classification of the image in front of the vehicle to distinguish between paved roads and solid shoulders; the safe stopping area specifically refers to a continuous area that is classified as a solid shoulder, has a width sufficient to accommodate the vehicle, and has no negative obstacles or moving targets detected outside its boundary.

5. The system according to claim 1, characterized in that, The lightweight semantic segmentation model uses depthwise separable convolution as the encoder backbone network, with no more than 10M model parameters, and is deployed on the environment perception module (102) through INT8 quantization.

6. The system according to claim 1, characterized in that, The local emergency driving decision and execution module (103) is further configured to: if the safe stopping area cannot be confirmed within a preset timeout period during the execution of the third state, then forcibly switch to a minimum risk state and perform braking and stopping within the current driving lane defined by the lane line.

7. The system according to claim 1, characterized in that, It also includes a communication and remote handover module (104), configured to send an alarm message to a preset terminal server in parallel with the execution of the finite state machine sequence at the moment the emergency response state is triggered; The alarm message includes at least: vehicle identification, location information, and the trigger reason code for the emergency response status.

8. The system according to claim 7, characterized in that, The communication and remote handover module (104) is also configured to: after the vehicle is safely parked, receive and verify a remote takeover request from the terminal server, and after successful verification, grant remote control access to the vehicle and access to a high-bandwidth data stream channel.

9. The system according to any one of claims 1 to 8, characterized in that, The local emergency driving decision and execution module (103) communicates with the vehicle actuator via the vehicle CAN bus; the CAN message corresponding to the control command it issues is configured to have a higher priority than the message corresponding to the driver's control input, so as to achieve arbitration coverage of the driver's command in case of conflict.

10. A method for emergency control of driver incapacitation in vehicles, characterized in that, Applied to the system as described in any one of claims 1 to 9, the method comprises: The driver status monitoring and incapacity determination module (101) automatically determines that the driver is incapacitated and triggers an emergency response state; In response to the emergency response state, the environmental perception module (102) is activated and begins to identify lane lines and drivable areas; At the same time, the local emergency driving decision and execution module (103) starts the finite state machine and executes vehicle warning, speed reduction based on lane lines and lane keeping in parallel; Once the vehicle speed drops to a preset threshold and the environmental perception confirms a safe stopping area, the system automatically guides the vehicle to change lanes and stop in the safe stopping area.

Images