Driver takeover reminding mode adaptive regulation and control method based on reinforcement learning

Abstract

The invention discloses a driver takeover reminding mode adaptive regulation and control method based on reinforcement learning, and relates to the technical field of automatic driving takeover, and the method comprises the steps: carrying out an automatic driving takeover experiment, and collecting the data of each takeover as experimental sample data; key factors influencing takeover performance and takeover comfort are analyzed, a deep reinforcement learning Deep Q-Network model is constructed, and the trained deep reinforcement learning Deep Q-Network model is utilized to recommend an optimal takeover reminding mode according to current feature factors; on the basis of improving the takeover performance and ensuring the driving safety, the influence of takeover reminding on the subjective comfort of the driver is considered.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving takeover technology, specifically a driver takeover alert modal adaptive control method based on reinforcement learning. Background Technology

[0002] Intelligent vehicles represent the future trend of vehicle development, and intelligent driving technology is expected to significantly reduce traffic accidents. However, in the short term, fully autonomous driving is unlikely. Currently, some intelligent vehicles are equipped with Level 3 autonomous driving systems, which can achieve autonomous driving within the designed operating range of the intelligent vehicle, without requiring the driver to perform environmental monitoring or vehicle control tasks. However, when the vehicle exceeds the designed operating range of the autonomous driving system, the driver still needs to take over passively.

[0003] After a takeover alert is issued, the driver's ability to promptly take over and respond to driving risks determines the driving safety of intelligent vehicles. Takeover alerts issued by autonomous driving systems significantly impact the driver's takeover performance and comfort, with the modality used to issue the takeover request information being a crucial factor. Autonomous driving takeover alerts include single-modal and multi-modal alerts. During autonomous driving, drivers are highly likely to engage in various non-driving tasks such as watching videos or listening to music. If the sensory channel corresponding to the takeover alert modality is occupied by non-driving tasks, it may lead to a prolonged takeover reaction time, preventing timely takeover. In this case, a single alert modality (such as visual, auditory, or tactile) may not meet the takeover requirements. While multi-modal alerts solve the problem of prolonged takeover reaction time due to occupied single sensory channels, they may lead to information overload, causing a surge in the driver's mental workload at the time the takeover alert is issued, thus adversely affecting their situational awareness recovery and hazard avoidance actions. Furthermore, frequent multi-modal alerts may also cause driver frustration, affecting their perceived comfort.

[0004] Therefore, how to adaptively adjust the takeover alert mode according to the takeover situation and driver state characteristics, so as to improve takeover performance and ensure driving safety while taking into account the impact of takeover alerts on the driver's subjective comfort, is an important problem that urgently needs to be solved. Summary of the Invention

[0005] The purpose of this invention is to provide an adaptive control method for driver takeover alert modality based on reinforcement learning. This method comprehensively considers the multidimensional influencing factors of the driver's own state, real-time state, and takeover type. It utilizes a reinforcement learning Deep Q-Network model to determine the optimal takeover alert modality under the current feature factors, aiming to improve takeover performance and takeover comfort. This method improves takeover performance and ensures driving safety while also taking into account the impact of takeover alerts on the driver's subjective comfort.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A driver takeover alert modal adaptive control method based on reinforcement learning includes the following steps: (1) Collect experimental sample data: Conduct autonomous driving takeover experiments, allowing multiple drivers in different states to take over the vehicle after being reminded by different takeover reminder modalities in different driving situations, and collect the data of each takeover as experimental sample data; (2) Constructing an experimental sample database: Based on the experimental sample data in step (1), analyze the key factors affecting takeover performance and takeover comfort, obtain the data of the key factors, quantify the takeover performance and takeover comfort, and take the key factor data other than the takeover reminder modality as feature factors; construct an experimental sample database using takeover performance, takeover comfort, takeover reminder modality and feature factors. (3) Construct a deep reinforcement learning Deep Q-Network model. The model aims to improve takeover performance and takeover comfort. Determine the optimal takeover reminder mode under the current feature factors. Use the experimental sample database in step (2) to train the model and determine the optimal parameters of the model. (4) Use the trained Deep Q-Network model to recommend the optimal takeover alert mode based on the current feature factors.

[0007] The driver takeover alert modal adaptive control method based on reinforcement learning, wherein the experimental sample data in step (1) includes takeover time data, takeover heart rate data, image data, sound data, and text data, wherein: The system acquires facial video image data of the driver during the driving process, collects sound data in the driver's cab, and collects text data of the driver's multidimensional state through questionnaires. The multidimensional state of the driver includes the driver's own state and the driver's real-time state.

[0008] The driver takeover alert modal adaptive control method based on reinforcement learning includes the driver's own state, such as age, driving experience, medical history, and driving style; and the driver's real-time state, such as visual occupancy, auditory occupancy, emotional state, and fatigue level. The driver's visual occupancy status, emotional state, and fatigue level are captured by recognizing the driver's facial area; the driver's auditory occupancy status is obtained by analyzing the sound data in the cab. When the decibel level in the cab is higher than the threshold, it is considered that the hearing is occupied; otherwise, it is considered that the hearing is not occupied.

[0009] The driver takeover alert modal adaptive control method based on reinforcement learning includes driver emotional states such as anger, disgust, fear, happiness, sadness, surprise, and neutrality. A CNN deep learning model is trained using the FER2013 dataset, and the trained CNN deep learning model is used to identify the driver's emotional state.

[0010] The driver takeover alert modal adaptive control method based on reinforcement learning describes the takeover performance as the takeover duration, which is the time interval from the issuance of the takeover request to the driver pressing the takeover button (with a sensor installed on the button) with their right hand. This time interval is the takeover time data, which can be collected by the vehicle's own system and the experimental background system. The takeover performance is divided into four categories: poor, moderate, good, and excellent based on the takeover duration.

[0011] The driver takeover alert modal adaptive control method based on reinforcement learning describes the takeover comfort level as represented by the driver's heart rate change value. Real-time data is obtained by the driver wearing a heart rate monitor, and the real-time data is synchronized with the experimental backend system to obtain accurate takeover heart rate data. The takeover comfort level is divided into four categories: low, lower, medium, and higher, based on the driver's heart rate change value.

[0012] The driver takeover alert modality adaptive control method based on reinforcement learning includes visual alerts, auditory alerts, tactile alerts, dual visual and auditory alerts, dual visual and tactile alerts, dual auditory and tactile alerts, and triple visual, auditory, and tactile alerts.

[0013] The driver takeover alert modality adaptive control method based on reinforcement learning includes driver multi-dimensional modalities and takeover events as feature factors. The takeover events include emergency braking of the vehicle in front, sudden lane change of the vehicle in the adjacent lane, sudden appearance of a fixed obstacle in front of the road, fog, approaching a construction zone, and disappearance of lane lines.

[0014] The driver takeover alert modal adaptive control method based on reinforcement learning, the total feedback function F of the Deep Q-Network model is constructed as follows: Where R is the reward function and P is the penalty function; ; ; ; In the formula, R p To take over performance bonuses, R c To take over the comfort reward, ω1 is the weight of the performance reward, and ω2 is the weight of the comfort reward; ; ; ; In the formula, It is an indicator function, which is 1 when the condition is true and 0 otherwise; ω3 is the weight of the performance penalty and ω4 is the weight of the comfort penalty.

[0015] The driver takeover alert modal adaptive control method based on reinforcement learning, and the training process of the Deep Q-Network model are as follows: The data generates a trajectory (s, a, r, s'), where s represents a multi-dimensional state vector consisting of the driver's own state, real-time state, and takeover event; a represents the takeover alert mode adopted; r represents the total feedback F; and s' represents the next state. Data is stored in a replay buffer, and a batch of data is randomly sampled from the buffer for training each time. The target Q-value for each data point is calculated using the following formula: ; In the formula, r represents the immediate reward obtained after taking action a in the current state; γ is a discount factor used to balance the importance of current rewards and future rewards. This represents the Q-value predicted by the target network in the next state s'; The gradient descent method is used to define the loss function L and update the parameters of the Q network to make its predicted value closer to the target Q value. ; In the formula, Q(s,a) represents the prediction of the Q network for the current state s and action a; N represents the batch size; Based on the parameters of the Q network, the parameters of the target network are updated periodically; the above steps are repeated iteratively until the model converges.

[0016] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention comprehensively considers the multi-dimensional influencing factors of the driver's own state, real-time state, and takeover type, integrates multi-source data to monitor the driver's psychological and physiological multi-dimensional state characteristics under autonomous driving conditions, and constructs a reinforcement learning model to achieve the purpose of adaptively adjusting the takeover reminder modality strategy according to the driver's real-time state, so as to ensure the comfort of the driver during the takeover process while ensuring the takeover performance.

[0017] 2. This invention comprehensively considers the multidimensional influencing factors of the driver's own state, real-time state, and takeover type. By processing and recognizing text data, real-time image data, and sound data of the driver's own state, as well as processing takeover time and driver's heart rate characteristics, it uses reinforcement learning to process multidimensional state and action space to achieve the goal of adaptively adjusting the optimal takeover reminder modality strategy according to real-time data, ensuring both takeover performance and driver comfort during the driving process. Attached Figure Description

[0018] Figure 1 This is a flowchart of the method of the present invention.

[0019] Figure 2 This is a schematic diagram showing the location of key points in this invention. Detailed Implementation

[0020] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0021] like Figure 1 , 2 As shown, this invention discloses a driver takeover alert modal adaptive control method based on reinforcement learning, comprising: (a) First, conduct autonomous driving takeover experiments. Drivers in different states and under different driving scenarios are reminded by different reminder modalities to take over the vehicle, and collect data from each takeover as experimental sample data.

[0022] The independent variables in the experiment included basic driver information, real-time driver status, takeover alert modalities, and takeover types. Basic driver information was collected before the experiment and mainly included age, driving experience, medical history, and driving style. Real-time driver status referred to the different states exhibited by the driver during driving, mainly including visual occupancy, auditory occupancy, emotional state, and fatigue level. Images and sounds were collected via cameras and microphones as multi-source real-time data. There were seven takeover alert modalities: visual (icon displaying "Please take over"), auditory (playing a beeping sound), tactile (vibration device installed in the driver's seat), visual + auditory, visual + tactile, auditory + tactile, and visual + auditory + tactile. Takeover types included seven main takeover events and their urgency levels.

[0023] The dependent variables in the experiment included takeover performance and takeover comfort. Takeover performance data was represented by takeover time, recording the time interval t from the moment the takeover request was issued to the driver pressing the takeover button with their right hand. Takeover comfort data was represented by driver heart rate changes, collected via an ECG sensor in a smart bracelet.

[0024] Conduct multiple sets of experiments to obtain sufficient sample data.

[0025] Based on experimental data, multi-dimensional state features of drivers can be extracted, including basic information and multi-dimensional real-time status. OpenCV is used to extract facial regions from camera-captured images, and driver fatigue level, emotional state, and visual occupancy are determined. Audio data collected from microphones is used to determine auditory occupancy.

[0026] Driver basic information is entered into the system by the driver and is considered known information. Specifically, this includes: ① Driver's age: Drivers are divided into elderly drivers (>=60 years old) and non-elderly drivers based on their age; ② Driver's driving experience: Drivers are divided into novice drivers (<=1 year) and non-novice drivers based on their driving experience; ③ Driver's medical history: Does the driver have any sudden illnesses such as heart disease or epilepsy? ④ Driving style: Is the driver's driving style aggressive or conservative? The method for obtaining multi-dimensional real-time driver status data is as follows: ① Visual channel occupancy status: The occupancy status is indicated by whether the eyes are looking straight ahead. The eye deflection angle is calculated by taking facial images with a camera. If the eye deflection angle is higher than the threshold, the visual channel is considered to be occupied. ② Hearing channel occupancy: Characterized by sound intensity, measured by a microphone. If the sound intensity is higher than a threshold, the hearing channel is considered to be occupied. ③ Fatigue level: Characterized by PERCLOS, facial images are captured by a camera, and those exceeding the threshold are judged as fatigued driving; ④ Emotional state: Represented by facial expressions, facial images were captured by a camera and combined with the FER2013 dataset, and classified into 7 emotions; The takeover scenario was set up and triggered during the experiment, and identified and input into the intelligent system through vehicle sensors (mounted cameras and radar) during actual driving. Takeover events mainly include: ① The vehicle in front braked suddenly ② A fixed obstacle suddenly appears in front of the road ③ A vehicle in the adjacent lane suddenly changes lanes to this lane. ④ Fog ⑤ We are about to enter a construction area. ⑥ Lane lines suddenly disappear The first three takeover events are determined by Time To Collision (TTC), which typically assesses the urgency of the takeover based on the remaining time.

[0027] ; In the formula, L represents the distance between the autonomous vehicle and other vehicles or objects, V2 represents the speed of the following vehicle, and V1 represents the speed of the preceding vehicle. If the TTC value is less than the threshold (e.g., 2 seconds), it is determined to be an emergency, triggering a takeover alert.

[0028] (II) Driver Multi-Dimensional Status Recognition The driver's multidimensional state includes the driver's inherent state and the driver's real-time state. The driver's inherent state is known information, while the driver's real-time state needs to be determined through captured image and sound data. Image data is used to determine the driver's emotional state, fatigue level, and visual occupancy; sound data is used to determine the driver's auditory occupancy.

[0029] For image data, a front-facing camera mounted in front of the driver is used to capture the driver's facial area, obtaining a video of the driver's face. OpenCV is used to read the video file and convert it into a frame sequence. The Haar cascade model in OpenCV is used to locate the facial regions and facial landmarks (such as the tip of the nose, chin, eyes, and mouth) in the video frames.

[0030] Identify emotional states: Load the model pre-trained using the FER2013 dataset, which contains 7 emotion categories (anger, disgust, fear, happiness, sadness, surprise, and neutral). Input the face region into the model to extract emotion features. Use a Softmax classifier to output the emotion category.

[0031] Visual channel occupancy determination: A head pose estimation method is used to calculate the head yaw angle and infer whether the driver is looking straight ahead. Based on the extracted facial keypoints of each frame, the OpenCV solvePnP function is used to calculate the rotation and translation vectors. The rotation vector is converted to Euler angles to obtain the head yaw angles (Yaw and Pitch). If the yaw and pitch angles are close to 0°, it indicates that the driver is looking straight ahead; if they exceed a set threshold (e.g., 15°), it is determined that the visual channel is occupied. The output result is whether the visual channel is occupied (yes / no).

[0032] Fatigue level assessment: The PERCLOS method is used to determine driver fatigue levels, which calculates the percentage of time the eyes are closed per unit of time. Based on video frames locating the eye area, the aspect ratio (EAR) of the eyes is calculated to determine whether the eyes are closed, and then the PERCLOS value is calculated.

[0033] ; In the formula, P1 to P6 are the coordinates of the key points of the eye (see details). Figure 2 Where P1 refers to the left corner of the eye; P2 refers to the position slightly to the left of the midpoint of the upper boundary of the eye; P3 refers to the position slightly to the right of the midpoint of the upper boundary of the eye; P4 refers to the right corner of the eye; P5 refers to the position slightly to the right of the midpoint of the lower boundary of the eye; and P6 refers to the position slightly to the left of the midpoint of the lower boundary of the eye.

[0034] Set an EAR threshold (e.g., 0.2). When the EAR is less than the threshold, it is considered that the eyes are closed.

[0035] ; Set a reasonable threshold (e.g., 10%) when PERCLO is detected. S When the value exceeds this threshold, the system will determine that the driver is fatigued. The output will be "Driver (Yes / No) Fatigue".

[0036] Auditory channel occupancy determination: For audio data, a microphone is used to collect sounds around the driver, acquiring audio data. A traditional mathematical algorithm is used to determine whether the auditory channel is occupied based on sound intensity. The collected data is then denoised and normalized, and the RMS (Root Mean Square) value is calculated, followed by the dB (decibels) value. If the dB value exceeds a threshold, the visual channel is considered occupied. The output result is whether the auditory channel is occupied (yes / no). The formulas for RMS and dB calculation are as follows: ; ; In the formula, x i N represents the audio samples, and N is the number of samples.

[0037] The driver's visual / auditory channel occupancy, fatigue, emotional state, age, driving experience, driving style, and medical history are combined to form a multi-dimensional state corresponding to a certain driver's takeover and a certain reminder mode.

[0038] (III) Classification of takeover performance and takeover comfort To quantify takeover performance and takeover comfort, the experimental sample data for both were categorized into four levels using cumulative frequency curves, in order to find the optimal takeover alert modality during later prediction processes.

[0039] The takeover performance is determined by the takeover time, where t refers to the time interval from the moment the takeover instruction is received until the driver presses the takeover button.

[0040] Collect driver takeover time data (t) for each instance during the experiment. Plot a cumulative frequency curve based on the experimental sample data. Referring to the classification criteria of existing studies, select 70%, 40%, and 20% cumulative frequencies as cutoffs to classify takeover performance into four categories: poor, average, good, and excellent.

[0041] Table 1 Classification of Takeover Performance Data

[0042] Related studies indicate that electroencephalogram (EEG) signals, skin conductance level (SCL), and heart rate (HR) can serve as physiological indicators during autonomous driving takeover, with elevated heart rate potentially indicating a stress response. This study uses changes in the driver's heart rate before and after takeover to represent the comfort level during that takeover.

[0043] Take over heart rate change value K ij =K 2ij -K 1ij In the formula, K ij K represents the j-th takeover by the i-th driver. 2ij K refers to the peak heart rate of the driver within 1 second after the j-th takeover request is issued by the i-th driver. 1ij This refers to the average heart rate of the driver in the 30 seconds prior to the j-th takeover request issued by the i-th driver.

[0044] Collect the heart rate change value K during each driver takeover in the experiment. ij A cumulative frequency curve was plotted based on the experimental sample data. Referring to the classification criteria for takeover performance, the cumulative frequencies of 80%, 60%, and 30% were selected as dividing lines to classify the takeover comfort level into four categories: low, lower, medium, and higher.

[0045] Table 2 Classification of Handover Comfort Data

[0046] Each takeover is assigned a takeover performance level label and a takeover comfort label using the above method, corresponding to the driver's multidimensional state at the time of the takeover and the selected alert modality, as a sample data set.

[0047] (iv) Deep Q-Network Model of Deep Reinforcement Learning Based on experimental sample data, a deep reinforcement learning model based on reward and penalty functions was established, with the training objective of optimizing takeover performance and takeover comfort. The specific steps are as follows.

[0048] Through processing of preliminary experimental data, the following eight states of the driver were obtained: visual occupancy (yes / no), auditory occupancy (yes / no), emotional state (anger, disgust, fear, happiness, sadness, surprise, neutral), fatigue (yes / no), age (yes / no elderly), driving experience (yes / no novice), medical history (yes / no history of sudden illness), and driving style (yes / no aggressive). There were also six types of takeover events, seven types of takeover alert modalities, four categories of takeover performance (poor, moderate, good, and excellent), and four categories of takeover comfort level (high, moderate, normal, and relaxed).

[0049] The above data is encoded, with "yes" data encoded as "1" and "no" data encoded as "0". The six takeover types and seven takeover alert modes are one-hot encoded, transforming them into 6-dimensional and 7-dimensional vectors respectively. Takeover performance is rated "poor", "moderate", "good", and "excellent" with scores of "0", "1", "2", and "3" respectively; takeover comfort is rated "low", "relatively low", "moderate", and "relatively high" with scores of "0", "1", "2", and "3" respectively.

[0050] The model's state features s comprise 14 features, including 8 multidimensional driver states and 6 takeover events. The action space a is constructed using 7 takeover alert modalities.

[0051] Design a reward function R.

[0052] ; ; ; In the formula, R p To take over performance bonuses, R c To take over the comfort reward, ω1 is the weight of the performance reward, and ω2 is the weight of the comfort reward.

[0053] Design a penalty function P. The penalty function is used to quantify undesirable behaviors, such as low performance or low comfort.

[0054] ; ; ; In the formula, It is an indicator function, which is 1 when the condition is true and 0 otherwise. ω3 is the weight of the performance penalty, and ω4 is the weight of the comfort penalty.

[0055] Total feedback function

[0056] The data generates a trajectory (s, a, r, s'), where s represents a 14-dimensional state vector (driver's own state, real-time state, and takeover event), a represents the seven actions taken (takeover alert modality), r represents the total feedback F, and s' represents the next state vector. Data is stored in a replay buffer. During each training iteration, a batch of data (e.g., 64 data points) is randomly sampled from the buffer for training. For each data point, if the current state is the termination state, the target Q-value is r; otherwise, the target Q-value is calculated using the following formula.

[0057] ; In the formula, r represents the immediate reward obtained after taking action a in the current state; γ is a discount factor (usually between 0.9 and 0.99), used to balance the importance of current rewards and future rewards; This represents the Q-value predicted by the target network in the next state s'.

[0058] The gradient descent method is used to define the loss function L and update the parameters of the Q network to make its predicted values ​​closer to the target Q value.

[0059] ; In the formula, Q(s,a) represents the prediction of the Q network for the current state s and action a; N represents the batch size.

[0060] Based on the parameters of the Q network, the parameters of the target network are updated periodically (e.g., every 100 steps).

[0061] Repeat the above steps iteratively until the model converges. After the model training is complete, the intelligent system can recommend the optimal takeover alert modality based on real-time data.

[0062] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A driver takeover alert modal adaptive control method based on reinforcement learning, characterized in that, include: (1) Conduct autonomous driving takeover experiments, allowing multiple drivers in different states to take over the vehicle after being reminded by different takeover reminder modalities in different driving situations, and collect data from each takeover as experimental sample data; (2) Based on the experimental sample data in step (1), analyze the key factors affecting takeover performance and takeover comfort, obtain the data of key factors, quantify takeover performance and takeover comfort, and take the key factor data other than the takeover reminder modality as feature factors; construct an experimental sample database using takeover performance, takeover comfort, takeover reminder modality and feature factors; (3) Construct a deep reinforcement learning Deep Q-Network model. The model aims to improve takeover performance and takeover comfort. Determine the optimal takeover reminder mode under the current feature factors. Use the experimental sample database in step (2) to train the model and determine the optimal parameters of the model. (4) Use the trained Deep Q-Network model to recommend the optimal takeover alert mode based on the current feature factors.

2. The driver takeover alert modal adaptive control method based on reinforcement learning according to claim 1, characterized in that, The experimental sample data in step (1) includes takeover time data, takeover heart rate data, image data, sound data, and text data, wherein: The system acquires facial video image data of the driver during the driving process, collects sound data in the driver's cab, and collects text data of the driver's multidimensional state through questionnaires. The multidimensional state of the driver includes the driver's own state and the driver's real-time state.

3. The driver takeover alert modal adaptive control method based on reinforcement learning according to claim 2, characterized in that, The driver's own status includes the driver's age, driving experience, medical history, and driving style; the driver's real-time status includes visual occupancy status, auditory occupancy status, emotional state, and fatigue level, wherein: The driver's visual occupancy status, emotional state, and fatigue level are captured by recognizing the driver's facial area; the driver's auditory occupancy status is obtained by analyzing the sound data in the cab. When the decibel level in the cab is higher than the threshold, it is considered that the hearing is occupied; otherwise, it is considered that the hearing is not occupied.

4. The driver takeover alert modal adaptive control method based on reinforcement learning according to claim 3, characterized in that, The driver's emotional state includes anger, disgust, fear, happiness, sadness, surprise, and neutrality; a CNN deep learning model is trained using the FER2013 dataset, and the trained CNN deep learning model is used to identify the driver's emotional state.

5. The driver takeover alert modal adaptive control method based on reinforcement learning according to claim 1, characterized in that, The takeover performance is characterized by takeover duration, which is the time interval from the issuance of the takeover request to the driver pressing the takeover button with his right hand. Based on the duration of the takeover, the takeover performance is divided into four categories: poor, average, good, and excellent.

6. The driver takeover alert modal adaptive control method based on reinforcement learning according to claim 1, characterized in that, The takeover comfort level is characterized by the driver's heart rate change value; the takeover comfort level is divided into four categories: low, lower, medium, and higher, based on the driver's heart rate change value.

7. The driver takeover alert modal adaptive control method based on reinforcement learning according to claim 2, characterized in that, The takeover reminder modes include visual reminders, auditory reminders, tactile reminders, dual visual and auditory reminders, dual visual and tactile reminders, dual auditory and tactile reminders, and triple visual, auditory, and tactile reminders.

8. The driver takeover alert modal adaptive control method based on reinforcement learning according to claim 2, characterized in that, The characteristic factors include driver multi-dimensional modality and takeover events, which include emergency braking of the vehicle in front, sudden lane change of the vehicle in the adjacent lane to the current lane, sudden appearance of a fixed obstacle in front of the road, fog, approaching a construction zone, and disappearance of lane markings.

9. The driver takeover alert modal adaptive control method based on reinforcement learning according to claim 1, characterized in that, The total feedback function F of the Deep Q-Network model is constructed as follows: Where R is the reward function and P is the penalty function; ; ; ; In the formula, R p To take over performance bonuses, R c To take over the comfort reward, ω1 is the weight of the performance reward, and ω2 is the weight of the comfort reward; ; ; ; In the formula, It is an indicator function, which is 1 when the condition is true and 0 otherwise; ω3 is the weight of the performance penalty and ω4 is the weight of the comfort penalty.

10. The driver takeover alert modal adaptive control method based on reinforcement learning according to claim 9, characterized in that, The training process of the Deep Q-Network model is as follows: The data generates a trajectory (s, a, r, s'), where s represents a multi-dimensional state vector consisting of the driver's own state, real-time state, and takeover event; a represents the takeover alert mode adopted; r represents the total feedback F; and s' represents the next state. Data is stored in a replay buffer, and a batch of data is randomly sampled from the buffer for training each time. The target Q-value for each data point is calculated using the following formula: ; In the formula, r represents the immediate reward obtained after taking action a in the current state; γ is a discount factor used to balance the importance of current rewards and future rewards. This represents the Q-value predicted by the target network in the next state s'; The gradient descent method is used to define the loss function L and update the parameters of the Q network to make its predicted value closer to the target Q value. ; In the formula, Q(s,a) represents the prediction of the Q network for the current state s and action a; N represents the batch size; Based on the parameters of the Q network, the parameters of the target network are updated periodically; the above steps are repeated iteratively until the model converges.

Images