A method, device, and equipment for handling takeover of an autonomous vehicle.

Abstract

This application provides a takeover handling method, device, and equipment for autonomous vehicles, relating to the field of autonomous driving technology. The method includes: acquiring data to be filtered based on the driver's physiological and eye-tracking data within the current time window; filtering and calculating the data to be filtered according to multiple preset data indicators and data types to obtain data to be predicted; obtaining the driver's takeover recovery time based on the data to be predicted using a trained time prediction model; and upon confirmation that a warning time has arrived, acquiring a warning level matching a preset warning execution rule and executing a corresponding takeover request warning strategy to achieve a takeover warning reminder for the vehicle. This application's method obtains more accurate and representative data to be predicted by filtering and calculating physiological and eye-tracking data, obtains a more accurate takeover recovery time using a trained time prediction model, and issues a warning reminder, thereby improving driving safety.

Description

Technical Field

[0001] This application relates to autonomous driving technology for vehicles, and more particularly to a takeover handling method, takeover handling device, and equipment for an autonomous vehicle. Background Technology

[0002] An automated system can continuously perform all dynamic driving tasks under its designed operating conditions. This means that as long as the vehicle is under its designed operating conditions, the automated system can take full responsibility for driving. However, when the system malfunctions or the driving environment exceeds the system's designed operating range, it is still necessary to promptly send a takeover request to the driver to notify the driver that control of the vehicle needs to be taken over to ensure driving safety.

[0003] When a takeover request is issued, the driver needs to shift their attention from non-driving tasks to driving tasks (i.e., resuming from a non-takeover state to a takeover state). During this process, the driver needs to be aware of the environment, which may include observing the surroundings, understanding road conditions, and the positions of other vehicles and pedestrians; and combining this with the driver's understanding of the autonomous vehicle's current status, which may include understanding the autonomous driving system's operating status, as well as information such as the vehicle's speed, direction, and position; then, the driver must disable the autonomous driving system and activate manual driving to perform the driving task. Therefore, it is necessary to monitor the driver's real-time takeover capability, predict the driver's takeover recovery time, and ensure vehicle safety during operation.

[0004] Existing takeover recovery time prediction models often rely on single-source data indicators such as eye-tracking metrics or driving posture to build their predictions. While these single-source data indicators can provide useful information, they may not comprehensively reflect the driver's state. For example, while eye-tracking metrics can reflect the driver's attention distribution, they cannot accurately reflect the driver's reaction time, because the driver may need some time to process the information and react after seeing a takeover request. Similarly, while driving posture can reflect the driver's driving intentions and behaviors, it may not accurately reflect the driver's attention distribution and reaction time, because the driver may be maintaining a certain posture while their attention is elsewhere. Therefore, existing takeover recovery time prediction methods can lead to prediction errors, resulting in the driver being unable to quickly regain control of the vehicle, thus posing a safety hazard. Summary of the Invention

[0005] This application provides a takeover handling method, takeover handling device, and equipment for autonomous vehicles to solve the problem of inaccurate prediction of takeover recovery time.

[0006] Firstly, this application provides a method for handling takeover of an autonomous vehicle, including:

[0007] Based on the physiological and eye movement data of the driver at all times during the current time window, the data to be filtered corresponding to the current time window is obtained. The data to be filtered and calculated according to multiple preset data indicators and the data type corresponding to each data indicator to obtain the data to be predicted corresponding to the current time window. The data to be predicted is data that meets the data indicators and the corresponding data type.

[0008] Based on the data to be predicted corresponding to the current time window, the takeover recovery time of the driver from the non-takeover state to the takeover state in the current time window is obtained through the trained time prediction model.

[0009] The autonomous driving failure time of the vehicle is obtained, and based on the takeover recovery time and the autonomous driving failure time, when the warning time is confirmed to have arrived, the warning level matching the preset warning execution rules is obtained and the corresponding request takeover warning strategy is executed to realize the takeover warning reminder of the autonomous vehicle.

[0010] In one possible design, the acquisition of multiple data metrics and the data type corresponding to each data metric includes:

[0011] Based on the acquired training dataset, a clustering algorithm is used to obtain the warning level and the warning time range corresponding to each warning level; wherein, the training dataset consists of physiological data of different data types and eye movement data of different data types, as well as the driver's takeover recovery time;

[0012] The training dataset and the warning time range corresponding to each warning level are used to perform importance identification calculations through the random forest algorithm to obtain the importance of the physiological data and eye movement data of each data type in the training set on the takeover recovery time.

[0013] Based on the importance of the physiological data and eye movement data of each data type to the takeover recovery time, the data types of physiological data and eye movement data with an importance greater than a set threshold are selected from the different data types of physiological data and eye movement data as the multiple data indicators and the data type corresponding to each data indicator.

[0014] In one possible design, obtaining the training dataset includes:

[0015] Based on a preset time window, multiple physiological data and multiple eye movement data of the driver in the non-disposal state are collected, and when a takeover request warning is issued, the takeover recovery time of the driver from the current non-disposal state to the takeover state is obtained.

[0016] The multiple physiological data and multiple eye movement data are sequentially subjected to denoising, anti-spoofing, and normalization processing to obtain multiple physiological data and multiple eye movement data to be screened.

[0017] Based on multiple preset data type dimensions, calculate the data type value for each of the physiological data to be screened, and the data type value for each of the eye movement data to be screened;

[0018] The training dataset is obtained by combining the takeover recovery time corresponding to the time window, the physiological data of the multiple data types, and the eye-tracking data of the multiple data types.

[0019] In one possible design, the step of obtaining the data to be filtered corresponding to the current time window based on the driver's physiological data and eye movement data collected at all times within the current time window includes:

[0020] Acquire physiological data of the driver at all collection points within the current time window; the physiological data includes the driver's electrocardiogram, electrodermal signal, and electromyography signal.

[0021] The electrocardiogram (ECG), electrodermal (ED) signal, and electromyography (EMG) signal are sequentially denoised and despoofed to obtain denoised and despoofed ECG, EED, and EMG signals.

[0022] The 2% and 98% percentile values ​​of the denoised and artifact-free electrocardiogram (ECG), electrodermal (ED) signals, and electromyography (EMG) signals were selected using the following formula:

[0023]

[0024] Obtain the normalized ECG signal V1 to be screened. ‘ , the skin conductance signal V2 to be screened ‘ and the electromyographic signal V3 to be screened ‘ ;

[0025] Among them, when calculating the normalized electrocardiogram signal V1 ‘ At that time, V1 is the denoised and artifact-free ECG signal, V min V is the 2nd percentile value of the electrocardiogram signal within the current time window. max The 98th percentile value of the electrocardiogram (ECG) signal within the current time window; when calculating the normalized EEG signal V2. ‘ At that time, V2 is the denoised and destigmatized skin conductance signal, V minV represents the 2nd percentile value of the electrodermal signal within the current time window. max This represents the 98th percentile of the electrodermal signal within the current time window; when calculating the normalized electromyographic signal V3... ‘ At that time, V3 is the denoised and artifact-free electromyographic signal, V min V represents the 2nd percentile value of the electromyographic signal within the current time window. max This represents the 98th percentile of the electromyographic signal within the current time window.

[0026] In one possible design, the step of obtaining the data to be filtered corresponding to the current time window based on the driver's physiological data and eye movement data collected at all times within the current time window includes:

[0027] Acquire eye movement data of the driver at all times within the current time window; the eye movement data includes the driver's blink frequency, blink duration, and fixation information;

[0028] The blink frequency, blink duration, and gaze information are sequentially subjected to denoising and artifact removal processes to obtain denoised and artifact-removed blink frequency, blink duration, and gaze information; wherein, the following formula is used:

[0029]

[0030] Obtain the gaze information F(t);

[0031] The driver's gaze position at a certain sampling time is f(t). When the driver's gaze position is in the preset non-driving area, f(t) = 1; when the driver's gaze position is in the preset driving area, f(t) = 0; T is the time window, and r is the preset sampling rate.

[0032] The 20th and 80th percentile values ​​for the denoised and destigmatized blink frequency, blink duration, and fixation information were selected using the following formula:

[0033]

[0034] Obtain the normalized blink frequency V1 to be screened. ‘ Blink duration to be selected V2 ‘ and gaze information to be filtered V3 ‘ ;

[0035] Among them, when calculating the normalized blink frequency V1 ‘ At that time, V1 is the blink frequency after noise and artifact removal, V min V represents the 20th percentile of blink frequency within the current time window. maxThe 80th percentile value of blink frequency within the current time window; when calculating the normalized blink duration V2 ‘ At that time, V2 represents the blink duration after noise and artifact removal, V min V is the 20th percentile value of the blink duration within the current time window. max The 80th percentile value of blink duration within the current time window; when calculating normalized fixation information V3. ‘ At that time, V3 represents the gaze information after denoising and artifact removal, V min V represents the 20th percentile of gaze information within the current time window. max This represents the 80th percentile of gaze information within the current time window.

[0036] In one possible design, the time prediction model is constructed from a generalized nonlinear regression equation, using the following formula:

[0037] y = f(x) i )+ε,i=1,2,…,j

[0038] Obtain the takeover recovery time y; where x i Let f(x) be the i-th data metric. i ) is a nonlinear function, and ε is a random error.

[0039] In one possible design, obtaining the takeover recovery time from the current non-takeover state to the takeover state includes:

[0040] Record the time when the takeover request warning is issued, the time when the vehicle steering wheel angle changes by 2° after the takeover request warning is issued, and the time when the vehicle brake pedal changes by 10% after the takeover request warning is issued.

[0041] If the time of the 2° change in the vehicle steering wheel angle is earlier than the time of the 10% change in the vehicle brake pedal, then the takeover recovery time is the time of the angle change minus the request time.

[0042] If the braking change time of 10% change in the vehicle's brake pedal is earlier than the angle change time of 2° change in the vehicle's steering wheel angle, then the takeover recovery time is the braking change time minus the request time.

[0043] In one possible design, the method further includes:

[0044] The time corresponding to each physiological data and each eye movement data collected by the preset device is recorded as the receiving time, and the data collection timestamp carried by each physiological data and each eye movement data is read.

[0045] If multiple data collection timestamps corresponding to the same receiving time are inconsistent, then according to the preset time synchronization rules, the timestamp to be synchronized is selected from the multiple data collection timestamps and sent to the preset device to correct the collection time of the preset device, so as to achieve collection time synchronization.

[0046] or,

[0047] At the end of a time window, the data acquisition timestamps carried by the physiological data and eye-tracking data corresponding to the last received time within the time window are obtained and read. According to the preset time synchronization rules, the timestamps to be synchronized are selected from multiple data acquisition times and sent to the preset device to correct the acquisition time of the preset device, thereby achieving acquisition time synchronization.

[0048] In one possible design, after obtaining the takeover recovery time when the driver returns to the takeover state from the current time window's non-takeover state to the takeover state, and before the confirmation warning time arrives, the method further includes:

[0049] If the takeover recovery time is less than or equal to the first time threshold, then the warning level of the current time window is obtained by querying the takeover recovery time and is a level four warning.

[0050] If the takeover recovery time is greater than the first time threshold and less than or equal to the second time threshold, then the warning level of the current time window is obtained by querying the takeover recovery time as a level three warning.

[0051] If the takeover recovery time is greater than the second time threshold and less than or equal to the third time threshold, then the warning level of the current time window is obtained by querying the takeover recovery time as a level two warning.

[0052] If the takeover recovery time is greater than the third time threshold and less than or equal to the fourth time threshold, then the warning level of the current time window, Level 1 warning, is obtained by querying based on the takeover recovery time.

[0053] In one possible design, the step of obtaining the warning level that matches the preset warning execution rules and executing the corresponding request takeover warning strategy when the warning time is confirmed includes:

[0054] Upon confirmation that the warning time has arrived, obtain the warning level of the current time window and the warning level of the previous time window;

[0055] The system selects the higher warning level from the warning levels of the current time window and the previous time window, and executes the corresponding warning strategy according to the higher warning level.

[0056] In one possible design, the step of obtaining the warning level that matches the preset warning execution rules and executing the corresponding request takeover warning strategy includes:

[0057] If the warning level is a level four warning, the warning strategy for the level four warning includes: voice prompt broadcast, displaying prompt text on the vehicle dashboard and central control screen, and strong vibration of the vehicle seats;

[0058] If the warning level is a Level 3 warning, the warning strategy for the Level 3 warning includes voice prompts, displaying prompt text on the vehicle dashboard and central control screen, and slight vibration of the vehicle seats.

[0059] If the warning level is a Level 2 warning, the warning strategy for the Level 2 warning includes voice prompts and displaying prompt text on the vehicle's dashboard and central control screen.

[0060] If the warning level is Level 1, then the warning strategy for Level 1 includes voice prompts.

[0061] In one possible design, the method further includes:

[0062] After issuing a takeover request warning, if a 10% change in the vehicle's brake pedal or a 2° change in the vehicle's steering wheel angle is detected, the warning strategy will be stopped.

[0063] Secondly, this application provides a pipe connection processing device, comprising:

[0064] The data acquisition and processing module is used to acquire the data to be filtered corresponding to the current time window based on the physiological data and eye movement data of the driver at all collection moments in the current time window, and to filter and calculate the data to be filtered according to multiple preset data indicators and the data type corresponding to each data indicator, so as to obtain the data to be predicted corresponding to the current time window, wherein the data to be predicted is data that meets the data indicators and the corresponding data type.

[0065] The takeover recovery time prediction module is used to obtain the takeover recovery time of the driver from the non-takeover state to the takeover state in the current time window based on the data to be predicted corresponding to the current time window and through a trained time prediction model.

[0066] The takeover warning processing module is used to obtain the autonomous driving failure time of the vehicle, and based on the takeover recovery time and the autonomous driving failure time, when the warning time is confirmed to have arrived, obtain the warning level that matches the preset warning execution rules and execute the corresponding request takeover warning strategy to realize the takeover warning reminder of the autonomous vehicle.

[0067] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0068] The memory stores computer-executed instructions;

[0069] The processor executes computer execution instructions stored in the memory to implement the takeover processing method.

[0070] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement a takeover processing method.

[0071] This application provides a takeover handling method, device, and equipment for autonomous vehicles. By filtering and calculating physiological and eye-tracking data, more accurate and representative data to be predicted is obtained, thereby improving the accuracy of the prediction model. Furthermore, by filtering and calculating physiological and eye-tracking data, personalized predictions can be achieved, better adapting to the characteristics of each driver. Using a trained time prediction model, the takeover recovery time can be predicted more accurately and obtained in real time. That is, based on the driver's real-time state, timely warnings are issued, thereby improving driving safety. Through preset warning level rules, appropriate warning levels are selected based on the takeover recovery time, ensuring flexible and accurate execution of warning strategies in different situations, improving the practicality and user-friendliness of the warning system. When the warning time arrives, the system can promptly alert the driver, providing sufficient reaction time and ensuring safe takeover of the vehicle in various situations. Attached Figure Description

[0072] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0073] Figure 1 A schematic flowchart of a takeover handling method for an autonomous vehicle provided in an embodiment of this application;

[0074] Figure 2 A schematic diagram of a method for obtaining multiple data indicators and the data type corresponding to each data indicator, provided in one embodiment of this application;

[0075] Figure 3 This is a schematic diagram of a method for obtaining a training dataset according to an embodiment of this application;

[0076] Figure 4 A flowchart illustrating a method for obtaining the takeover recovery time when a driver returns to a takeover state from a current non-takeover state, as provided in one embodiment of this application;

[0077] Figure 5 This is a schematic flowchart illustrating a method for synchronizing data acquisition time when data acquisition timestamps are inconsistent, provided in one embodiment of this application.

[0078] Figure 6 This is a schematic flowchart illustrating a method for synchronizing data acquisition time at the end of a time window, as provided in one embodiment of this application.

[0079] Figure 7 This is a schematic flowchart illustrating a method for obtaining an early warning level based on the takeover recovery time, as provided in one embodiment of this application.

[0080] Figure 8 A flowchart illustrating a method for obtaining an early warning level and executing a corresponding request-to-takeover early warning strategy according to an embodiment of this application;

[0081] Figure 9 A flowchart illustrating the execution of a request-to-takeover warning strategy according to an embodiment of this application;

[0082] Figure 10 This is a schematic diagram of the structure of an autonomous vehicle warning system provided in one embodiment of this application;

[0083] Figure 11 This is a schematic diagram of the structure of the pipe-handling device provided in the embodiments of this application;

[0084] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0085] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0086] In existing technologies, takeover time prediction models often rely on single-source indicators such as eye-tracking metrics or driving posture. While these models can predict driver takeover time to some extent, they fail to comprehensively reflect the driver's state and behavior because they only consider a single source of information. This results in significant uncertainty in predicting the time it takes for the driver to resume takeover from a non-takeover state, potentially leading to inaccurate predictions of how long it will take for the driver to take over, thus impacting the safety of autonomous vehicles. Furthermore, prediction errors can not only result in insufficient time for the driver to take over but also premature takeover, affecting the efficiency of the autonomous driving system.

[0087] Based on the aforementioned problems and needs, the inventive concept of this application is as follows: By integrating the driver's physiological data and eye-tracking data, a time prediction model is employed to monitor the driver's state in real time and predict the takeover recovery time. Upon confirmation that the warning time has arrived, a takeover warning strategy matching the current driver state is selected to issue a warning, reminding the driver to prepare for takeover, effectively improving the safety of autonomous vehicles. Based on this, by collecting the driver's physiological signals and eye-tracking data and combining them with time information, a takeover time prediction model is constructed to ensure that the driver has sufficient time to safely take over the vehicle under various circumstances. Simultaneously, a clustering algorithm is used to obtain various time boundary thresholds, and corresponding takeover warning levels are defined according to different takeover recovery times. Furthermore, based on the time thresholds obtained from the clustering algorithm, a random forest algorithm is used to select data indicators and data types that are more important to the takeover recovery time, thereby training the time prediction model to improve the accuracy of time prediction.

[0088] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0089] Figure 1 This is a schematic flowchart illustrating a takeover process for an autonomous vehicle provided in an embodiment of this application. Figure 1 As shown, the method includes steps S11-S13:

[0090] S11. Based on the physiological data and eye movement data of the driver at all collection moments in the current time window, obtain the data to be filtered corresponding to the current time window, and filter and calculate the data to be filtered according to multiple preset data indicators and the data type corresponding to each data indicator to obtain the data to be predicted corresponding to the current time window. The data to be predicted is data that meets the data indicators and the corresponding data type.

[0091] In this embodiment, the duration of a time window is used as the data benchmark for a single prediction. Based on the driver's physiological and eye-tracking data collected at all times within the current time window, the data to be filtered for the current time window is obtained. This data is then filtered and calculated to obtain the predictable data that meets the data indicators and the corresponding data type. Therefore, the predictable data is obtained based on the driver's physiological and eye-tracking data. Using the driver's physiological and eye-tracking data for subsequent predictions aims to monitor the driver's physical and mental state in real time, thereby improving driving safety. Prediction from a single data source leads to reduced accuracy. For example, relying solely on physiological data to predict the driver's state may overlook their mental state, such as inattention. Combined prediction using physiological and eye-tracking data provides more comprehensive and accurate information about the driver's state. Specifically, physiological data reflects the driver's physical condition. For example, a sudden increase in the driver's heart rate may indicate stress or anxiety, which could affect driving ability. Eye-tracking data reflects the driver's mental state. For example, a slowed eye movement or decreased pupil size may indicate that the driver's attention is being diverted. Combining physiological and eye movement data allows for a better understanding of the driver's state, thereby more effectively improving driving safety. The combined acquisition of physiological and eye movement data provides a strong data foundation for predicting take-off times, enabling more accurate predictions and enhancing vehicle safety.

[0092] S12, based on the data to be predicted corresponding to the current time window, obtain the takeover recovery time when the driver recovers from the non-takeover state to the takeover state in the current time window through the trained time prediction model.

[0093] In this embodiment, based on a trained time prediction model, the time required for a driver to return from autonomous driving to manual takeover is predicted by inputting the data to be predicted. The accuracy of this takeover recovery time directly relates to when a takeover request is sent to the driver, ensuring that the driver can take over the vehicle promptly and effectively in the event of system failure or exceeding the designed operating range. The data to be predicted includes physiological and eye-tracking data within the current time window. The trained time prediction model enables efficient processing and analysis of the data. The model has learned the complex relationship between physiological and eye-tracking data and takeover recovery time, capturing potential patterns and regularities among these data, ensuring its adaptability and generalization ability to various driving scenarios. By comprehensively considering the influence of physiological and eye-tracking data, the time prediction model can provide accurate takeover recovery time predictions under different driving scenarios. This is because different drivers may exhibit different physiological and perceptual characteristics under the same driving scenario. The application of the time prediction model allows for a more accurate response to the driver's state, improving the safety and reliability of vehicle operation.

[0094] S13: Obtain the autonomous driving failure time of the vehicle, and based on the takeover recovery time and autonomous driving failure time, when the warning time is confirmed to have arrived, obtain the warning level that matches the preset warning execution rules and execute the corresponding request takeover warning strategy to realize the takeover warning reminder for autonomous vehicles.

[0095] In this embodiment, upon confirming the arrival of the warning time when the autonomous driving system fails, and obtaining the warning level matching the preset warning execution rules and executing the corresponding warning strategy, a takeover request is sent to the driver, thereby reminding the driver to take over the vehicle in a timely manner and achieving safe takeover of the autonomous vehicle. First, the autonomous driving failure time of the vehicle is obtained. The failure time refers to the moment when the autonomous driving system cannot operate normally due to various reasons. For example, if radar equipment or cameras cannot correctly detect the surrounding environment, or if the control algorithm cannot correctly analyze sensor data and make decisions, then the autonomous driving system can be said to have "failed" at this point in time. Similarly, if obstacles or situations that the autonomous driving system has never encountered before appear on the road, such as suddenly appearing roadblocks or complex traffic conditions, the autonomous driving system may also "exceed its designed operating range" at this point in time. This time point is the critical point for the failure of the autonomous driving system, determining when the autonomous driving system may require driver intervention. This failure time information is usually obtained through system monitoring and fault detection, ensuring the real-time nature and reliability of the obtained time.

[0096] Next, the warning time is determined based on the takeover recovery time and the autopilot failure time. The takeover recovery time refers to the time required for the driver to return from a non-takeover state (e.g., the driver may be resting or distracted) to a takeover state (i.e., the driver can safely control the vehicle). The warning time, determined based on the takeover recovery time and the autopilot failure time, is the point in time when the warning reminder needs to be issued in advance. When the warning time arrives, the warning level matching the preset warning execution rules is obtained, and the corresponding takeover request warning strategy is executed. The warning level is determined based on the takeover recovery time, which is determined by the driver's state. That is, the warning level can be determined based on the driver's state. Alternatively, it can be determined based on both the driver's state and the state of the autopilot system. The takeover request warning strategy has different execution strategies depending on the warning level. These strategies include audible alerts, visual alerts, and vibration alerts to attract the driver's attention and prompt them to take over the vehicle in a timely manner. In summary, the system executes corresponding warning strategies based on the vehicle's autopilot failure time, the driver's takeover recovery time, and the warning time. The goal is to ensure that drivers can take over the vehicle in a timely manner when the autonomous driving system fails or exceeds its designed operating range, thereby ensuring driving safety.

[0097] This application improves the accuracy of the prediction model by filtering and calculating physiological and eye-tracking data to obtain more accurate and representative data for prediction. Furthermore, by filtering and calculating physiological and eye-tracking data, personalized predictions can be achieved, better adapting to the characteristics of each driver. Using a trained time prediction model, the takeover recovery time can be predicted more accurately and obtained in real time. This allows for timely warnings based on the driver's real-time state, thereby improving driving safety. Through preset warning level rules, the appropriate warning level can be selected based on the takeover recovery time, ensuring flexible and accurate execution of warning strategies in different situations, improving the practicality and user-friendliness of the warning system. When the warning time arrives, the system can promptly alert the driver, providing sufficient reaction time and ensuring safe vehicle takeover in various situations.

[0098] Figure 2 This is a schematic flowchart illustrating a method for obtaining multiple data indicators and the data type corresponding to each data indicator, as provided in one embodiment of this application. It is a specific description of one implementation of the data indicator and data type acquisition in step S11 above. Figure 2 As shown, it includes:

[0099] S21. Based on the acquired training dataset, a clustering algorithm is used to obtain the warning level and the warning time range corresponding to each warning level; wherein, the training dataset consists of physiological data of different types, eye-tracking data of different types, and the driver's takeover recovery time;

[0100] S22, the importance of the training dataset and the warning time range corresponding to each warning level is calculated by using the random forest algorithm to obtain the importance of the physiological data and eye movement data of each data type in the training set on the takeover recovery time.

[0101] S23, based on the importance of physiological data and eye movement data of each data type to the takeover recovery time, from physiological data and eye movement data of different data types, the data types of physiological data and eye movement data that are greater than the set importance threshold are used as multiple data indicators and the data type corresponding to each data indicator.

[0102] In this embodiment, the training dataset includes physiological and eye-tracking data of different types, along with corresponding driver take-off and recovery times. This structure allows for a comprehensive consideration of different data aspects, more fully capturing the correlation between driver state and take-off and recovery time. A clustering algorithm is used to divide the physiological and eye-tracking data in the training dataset into different clusters. This process is based on data similarity, ensuring high similarity among data points within the same cluster and low similarity between data points in different clusters. For driving behavior, this means that similar driving states and behaviors will be grouped into the same cluster. Each cluster represents a driving state. These states help determine different warning time ranges and corresponding warning levels. For example, to achieve effective clustering, classic clustering algorithms such as K-means clustering or hierarchical clustering can be used. These algorithms can group similar data points into the same cluster and help identify different driving states. Each cluster is assigned a warning level, and a corresponding warning time range is determined for each level. This means that under similar driving states, the warning level corresponding to that state can be determined.

[0103] After obtaining the clustering results, the Random Forest algorithm was used to calculate the importance of each physiological and eye-tracking data point to determine which data types have a more significant impact on takeover recovery time. Specifically, by setting importance thresholds, physiological and eye-tracking data types with significant influence were selected. This helps reduce the complexity of the model, making the final data indicators more concise and practically operable. Next, physiological and eye-tracking data types with importance exceeding the set thresholds were selected as multiple data indicators and specific data types. These data indicators represent the data features with the most significant impact on takeover recovery time in the entire training dataset. It should be noted that the data types can be set based on experience or through continuous experimentation to determine the appropriate data types for calculation. For example, four data types were used: mean, root mean square, maximum, and minimum. This selection allows the time prediction model to more accurately predict the driver's takeover recovery time, providing a more targeted basis for subsequent warnings. Furthermore, by analyzing the impact of physiological and eye-tracking data on takeover recovery time, it is possible to better adapt to the actual needs and reaction patterns of drivers. By employing clustering and random forest algorithms, warning levels and timeframes can be determined more accurately, thereby improving the precision and reliability of warnings. Furthermore, personalized warning systems can be provided based on each driver's physiological and eye-tracking data, enhancing the user experience.

[0104] In one embodiment, Figure 3 This is a flowchart illustrating a method for obtaining a training dataset according to an embodiment of this application. It is a specific description of one implementation of the training dataset obtained in step S21 above. Figure 3 As shown, it includes:

[0105] S31, based on a preset time window, collects multiple physiological data and multiple eye movement data of the driver in the non-disposal state, and when a takeover request warning is issued, obtains the takeover recovery time of the driver from the current non-disposal state to the takeover state;

[0106] S32, perform noise reduction, anti-spoofing and normalization processing on multiple physiological data and multiple eye movement data in sequence to obtain multiple physiological data and multiple eye movement data to be screened;

[0107] S33, calculate the value of each data type for each physiological data to be screened, and the value of each data type for each eye-tracking data to be screened, based on multiple preset data type dimensions;

[0108] S34. Based on the takeover recovery time corresponding to the time window, physiological data of multiple data types, and eye-tracking data of multiple data types, a training dataset is obtained.

[0109] This embodiment primarily details the acquisition and composition of the training dataset. First, within a preset time window, multiple physiological and eye-tracking data of the driver in a non-disengagement state are collected. Simultaneously, the take-over recovery time—the time it takes for the driver to return to the take-over state after a take-over request warning is issued—is recorded to obtain complete driving behavior data and corresponding take-over recovery time labels. Next, the collected physiological and eye-tracking data undergo denoising to eliminate outliers that may be caused by sensor errors or environmental interference. Denoising helps improve the accuracy and reliability of the data, ensuring the effectiveness of subsequent analysis and modeling. Simultaneously, artifact removal is performed. The main goal of artifact removal is to identify and eliminate artifacts or pseudo-signs in the data, helping to ensure that the data used during model training is realistic and reflects actual driving conditions. The denoised and artifact-removed data is then normalized, mapping the values ​​of each physiological and eye-tracking data point to similar numerical ranges. This helps eliminate the influence of different dimensions between data points, making them comparable and providing consistent input for subsequent model training. After denoising, artifact removal and normalization, multiple physiological data and multiple eye-tracking data to be screened were obtained.

[0110] Secondly, based on multiple preset data type dimensions, values ​​for each data type are calculated for both the physiological data and eye-tracking data to be screened. This aims to extract features from specific data types to express the driver's physiological and eye-tracking states in a more concrete and informative way. Finally, a training dataset is constructed based on the takeover recovery time corresponding to the time window, multiple data types of physiological data, and multiple data types of eye-tracking data. This dataset will serve as input to the model during the training phase to learn the complex relationship between driving states and physiological eye-tracking data, enabling accurate state prediction in practical applications. The above data processing flow ensures that the data used by the model is authentic, reliable, and interpretable, providing a more accurate prediction and decision-making foundation for the time prediction model. It should be further noted that in addition to the training dataset, a test dataset is also required for model training. The test dataset has the same data characteristics as the training dataset; therefore, the acquisition of the test dataset will not be elaborated upon here.

[0111] In one embodiment, a detailed description is provided of one implementation of step S11 above, which involves obtaining the data to be filtered corresponding to the current time window based on the driver's physiological data collected at all times within the current time window. This includes steps S1111-S1113:

[0112] S1111, acquire physiological data of the driver at all collection moments within the current time window; the physiological data includes the driver's electrocardiogram signal, skin conductance signal, and electromyography signal;

[0113] S1112, the electrocardiogram signal, skin conductance signal and electromyography signal are denoised and destigmatized respectively to obtain the denoised and destigmatized electrocardiogram signal, skin conductance signal and electromyography signal.

[0114] S1113, for the denoised and artifact-free ECG, ductus skinitis, and electromyography signals, the 2% and 98% percentile values ​​were selected, using the following formula:

[0115]

[0116] Obtain the normalized ECG signal V1 to be screened. ‘ , the skin conductance signal V2 to be screened ‘ and the electromyographic signal V3 to be screened ‘ Among them, when calculating the normalized electrocardiogram signal V1 ‘ At that time, V1 is the denoised and artifact-free ECG signal, V min V is the 2nd percentile value of the electrocardiogram signal within the current time window. max The 98th percentile value of the electrocardiogram (ECG) signal within the current time window; when calculating the normalized EEG signal V2. ‘ At that time, V2 is the denoised and destigmatized skin conductance signal, V min V represents the 2nd percentile value of the electrodermal signal within the current time window. max This represents the 98th percentile of the electrodermal signal within the current time window; when calculating the normalized electromyographic signal V3... ‘ At that time, V3 is the denoised and artifact-free electromyographic signal, V min V represents the 2nd percentile value of the electromyographic signal within the current time window. max This represents the 98th percentile of the electromyographic signal within the current time window.

[0117] In this embodiment, specific acquisition directions are given for physiological data, such as electrocardiogram (ECG), electrodermal (EDA), and electromyography (EMG) signals. ECG, EDA, and EMG signals of the driver are acquired using contact devices, such as watchband-type devices. These signals may be affected by factors such as vehicle vibration and electromagnetic interference, thus requiring denoising. The purpose of denoising is to eliminate unnecessary noise, making the signal purer and more reliable. Artifact removal is also necessary. Specifically, MATLAB toolboxes are used to remove noise and artifact segments from the ECG signal; a fifth-order Butterworth low-pass filter is used to filter the EDA signal, and interpolation is used to remove EDA signal artifacts (i.e., artifact removal); MATLAB toolboxes are used to remove motion artifacts from the EMG signal, eliminating the influence of low-frequency human movement (range) on the EMG signal. Next, normalization is applied to map the denoised and artifact-removed ECG, EDA, and EMG signals to similar numerical ranges. This eliminates the dimensional influence between different signals, making them comparable. The specific formula is shown above. When calculating the normalized ECG signal V1... ‘ At that time, V1 is the denoised and artifact-free ECG signal, V min V is the 2nd percentile value of the electrocardiogram signal within the current time window. max The 98th percentile value of the electrocardiogram (ECG) signal within the current time window; when calculating the normalized EEG signal V2. ‘ At that time, V2 is the denoised and destigmatized skin conductance signal, V min V represents the 2nd percentile value of the electrodermal signal within the current time window. max This represents the 98th percentile of the electrodermal signal within the current time window; when calculating the normalized electromyographic signal V3... ‘ At that time, V3 is the denoised and artifact-free electromyographic signal, V min V represents the 2nd percentile value of the electromyographic signal within the current time window. max This represents the 98th percentile value of the electromyographic signals within the current time window. The resulting data to be screened has a uniform numerical range, which is helpful for subsequent analysis.

[0118] In summary, denoising and artifact removal processes eliminate noise and artifact signals that could affect data accuracy, ensuring that the physiological data used truly reflect driving conditions. Normalization eliminates dimensional differences between different signals, giving them a consistent numerical range and enhancing data comparability. Furthermore, these preprocessing steps are used in both modeling and subsequent data analysis, ensuring the high quality and reliability of the physiological data used, providing a reliable input foundation for predicting time based on driving conditions.

[0119] In another embodiment, a specific description is provided of one implementation of step S11 above, which involves obtaining the data to be filtered corresponding to the current time window based on the driver's eye-tracking data collected at all times within the current time window. This includes steps S1121-S1123:

[0120] S1121, acquire eye movement data of the driver at all collection moments within the current time window; wherein, eye movement data includes the driver's blink frequency, blink duration and fixation information;

[0121] S1122, the blink frequency, blink duration and gaze information are denoised and destigmatized in sequence to obtain the denoised and destigmatized blink frequency, blink duration and gaze information.

[0122] S1123, for the denoised and destigmatized blink frequency, blink duration, and fixation information, the 20th and 80th percentile values ​​are selected, respectively, using the following formula:

[0123]

[0124] Obtain the normalized blink frequency V1 to be screened. ‘ Blink duration to be selected V2 ‘ and gaze information to be filtered V3 ‘ Among them, when calculating the normalized blink frequency V1 ‘ At that time, V1 is the blink frequency after noise and artifact removal, V min V represents the 20th percentile of blink frequency within the current time window. max The 80th percentile value of blink frequency within the current time window; when calculating the normalized blink duration V2 ‘ At that time, V2 represents the blink duration after noise and artifact removal, V min V is the 20th percentile value of the blink duration within the current time window. max The 80th percentile value of blink duration within the current time window; when calculating normalized fixation information V3. ‘ At that time, V3 represents the gaze information after denoising and artifact removal, V min V represents the 20th percentile of gaze information within the current time window. max This represents the 80th percentile of gaze information within the current time window.

[0125] In this embodiment, eye movement data of the driver within the current time window is acquired, including blink frequency, blink duration, and fixation information. For example, eye movement data of the driver is collected using a non-contact dual-camera eye tracker. To improve the quality and accuracy of the data, denoising and artifact removal are performed, and normalization is applied to obtain the blink frequency, blink duration, and fixation information to be filtered. Specifically, blink frequency is the number of blinks per minute, and blink duration is the time from the start to the end of a blink. A blink detection algorithm is used to automatically remove noise and artifact segments. Denoising and artifact removal are performed on the blink frequency, blink duration, and fixation information, making the processed eye movement data more reflective of the driver's true eye movement behavior and providing more accurate input for subsequent analysis. Subsequently, by selecting the 20th and 80th percentile values, a normalization formula is applied to normalize the denoised and artifact-removed blink frequency, blink duration, and fixation information. Similar to the physiological data processing described above, this series of processing operations is designed to ensure that the acquired eye-tracking data is more accurate, authentic, and comparable, providing more reliable data support for subsequent analysis, modeling, and decision-making.

[0126] The driver's gaze information typically refers to the location and duration of the driver's gaze while driving. This information can be obtained using eye-tracking technology. An illustrative explanation of gaze information is provided here. Specifically, it uses the following formula:

[0127]

[0128] The process involves obtaining gaze information F(t). The vehicle interior is divided into driving and non-driving zones. The driver's gaze position at a given sampling moment is f(t). When the driver's gaze is in the preset non-driving zone, f(t) = 1; when the driver's gaze is in the preset driving zone, f(t) = 0. T is the time window, and r is the preset sampling rate. According to the formula, accumulating gaze positions using an integral method yields comprehensive gaze information within a time window. Secondly, binarying the gaze positions in the driving and non-driving zones helps to further quantify the driver's attention distribution. For example, whether the driver is looking at the road, checking the dashboard, or observing the rearview mirror. Furthermore, gaze information can be used to analyze the driver's driving behavior and state, such as whether the driver is distracted or fatigued. Gaze information can more precisely describe the driver's attention to the driving and non-driving zones at different time periods, providing crucial data for understanding driver behavior and state, and playing a vital role in improving driving safety.

[0129] In one specific embodiment, a concrete implementation method for constructing the time prediction model is provided. Based on the above embodiment, this is a specific implementation of the time prediction model trained in step S12. The time prediction model is constructed from a generalized nonlinear regression equation, using the following formula:

[0130] y = f(x) i )+ε,i=1,2,…,j

[0131] Obtain the takeover recovery time y; where x i Let f(x) be the i-th data metric. i ) is a nonlinear function, and ε is a random error.

[0132] In this embodiment, a nonlinear regression model is employed. Unlike traditional linear regression models, nonlinear regression models are more flexible and can better fit complex data relationships. This is particularly important for driving behavior, a problem involving multiple factors and exhibiting nonlinear relationships. As the formula shows, this model can output an estimate of the takeover recovery time based on the various input data indicators, combined with a nonlinear function and a random error term. This is achieved by introducing a nonlinear function f(x)... i This approach enhances the model's expressive power, enabling it to more accurately capture the complex relationships between various data indicators. The inclusion of random error means it can handle random fluctuations in the data, thus improving the model's robustness. During the model training phase, the model is trained using existing training datasets. The goal of this training process is to adjust the parameters in the nonlinear function so that the model can better fit known data and exhibit better generalization ability when facing new data. The model continuously optimizes itself to adapt to real-world situations by learning patterns and trends in the data. In the application phase, the trained model can be used to predict takeover recovery time, leading to a better understanding of the relationship between driver state and behavior. In summary, by constructing a time prediction model using a generalized nonlinear regression equation and through multi-stage processing and training, the model can more accurately reflect the complex relationships between physiological and eye-tracking data, providing strong support for predicting takeover recovery time. This leads to a more comprehensive understanding of driving behavior and improves driving safety.

[0133] Figure 4 This is a flowchart illustrating a method for obtaining the takeover recovery time when a driver returns to a takeover state from a current non-takeover state, as provided in one embodiment of this application. It is a specific manifestation of the takeover recovery time definition in step S11 above. Figure 4 As shown, it includes:

[0134] S41, record the request time when the takeover request warning is issued, the angle change time when the vehicle steering wheel angle changes by 2° after the takeover request warning is issued, and the braking change time when the vehicle brake pedal changes by 10% after the takeover request warning is issued.

[0135] S42, if the time of the 2° change in the vehicle steering wheel angle is earlier than the time of the 10% change in the vehicle brake pedal angle, then the takeover recovery time is the angle change time minus the request time.

[0136] S43, if the braking change time of 10% change of the vehicle's brake pedal is earlier than the angle change time of 2° change of the vehicle's steering wheel angle, then the takeover recovery time is the braking change time minus the request time.

[0137] In this embodiment, during the takeover request warning process, key time points and vehicle status information need to be recorded for subsequent calculation of the takeover recovery time. This mainly involves two key indicators: changes in the vehicle's steering wheel angle and changes in the vehicle's brake pedal. Recording this information allows for a more accurate understanding of the driver's response speed to the takeover request. First, the request time when the takeover request warning is issued is recorded. This time point marks the moment the system sends the takeover request to the driver and is the starting point for subsequent calculations. Second, the time for a 2° change in the vehicle's steering wheel angle is recorded. This data represents the driver's response time to turning the steering wheel after the takeover request warning is issued. The change in steering wheel angle reflects the driver's perception and decision-making process regarding the takeover request from the system. Recording this time allows for understanding the driver's operational speed and sensitivity to the warning when facing a takeover request. Simultaneously, the time for a 10% change in the vehicle's brake pedal is also recorded. This data reflects another way the driver responds to the takeover request: reacting through the brake pedal. Changes in the brake pedal typically indicate that the driver has taken braking measures when facing the warning. Recording this time allows for understanding the driver's braking reaction speed in emergency situations.

[0138] Next, a comparison is needed to determine the takeover recovery time. If the steering wheel angle change of 2° occurs earlier than the brake pedal change of 10%, then the takeover recovery time is the steering wheel angle change time minus the request time. This is because the faster steering wheel angle change indicates a quicker driver response. Conversely, if the brake pedal change of 10% occurs earlier than the steering wheel angle change of 2°, then the takeover recovery time is the brake change time minus the request time. This is because the brake pedal responds faster, indicating a quicker braking action by the driver. This recording and comparison process helps to gain deeper insights into driver behavior patterns and decision-making processes during takeover requests. Furthermore, analyzing this data can better optimize the takeover strategy of the autonomous driving system, improving system safety and driver comfort. This also provides a useful reference for future driver assistance systems and autonomous driving technologies.

[0139] Figure 5 This is a schematic flowchart illustrating a method for synchronizing data acquisition time when data acquisition timestamps are inconsistent, provided in one embodiment of this application. Based on the above embodiment, as follows... Figure 5 As shown, the method also includes:

[0140] S51, record the time corresponding to each physiological data and each eye movement data collected by the preset device as the receiving time, and read the data collection timestamp carried by each physiological data and each eye movement data;

[0141] S52, if multiple data acquisition timestamps corresponding to the same reception time are inconsistent, then according to the preset time synchronization rules, the timestamp to be synchronized is selected from the multiple data acquisition timestamps and sent to the preset device to correct the acquisition time of the preset device, thereby achieving acquisition time synchronization.

[0142] In this embodiment, the time corresponding to each physiological data and eye-tracking data collected by the preset device is recorded as the reception time. This reception time marks the moment the data arrives at the system and is the basis for subsequent time synchronization. Next, the data acquisition timestamp carried by each physiological data and eye-tracking data is read. The data acquisition timestamp is the time information recorded by the preset device when collecting data, used to identify the exact moment of data acquisition. Next, multiple data acquisition timestamps corresponding to the same reception time may be inconsistent. This may be due to differences in clock precision between different sensors or delays in data transmission. To solve this problem, preset time synchronization rules are set. According to the preset time synchronization rules, timestamps to be synchronized are selected from multiple data acquisition timestamps. This timetamp to be synchronized is a representative timestamp selected by the rules and can be used as a reference for the acquisition time. The purpose of this step is to eliminate time differences between different data and ensure that they are consistent in time. Finally, the selected timestamps to be synchronized are sent to the preset device to correct its acquisition time. This operation achieves the synchronization of acquisition time, ensuring the consistency of different physiological data and eye-tracking data in time. Ensuring accurate alignment of data in time can improve the accuracy and reliability of analysis.

[0143] Figure 6 This is a flowchart illustrating a method for synchronizing data acquisition time at the end of a time window, as provided in one embodiment of this application. Based on the above embodiment, as follows... Figure 6 As shown, the method also includes:

[0144] S61, record the time corresponding to each physiological data and each eye movement data collected by the preset device as the receiving time, and read the data collection timestamp carried by each physiological data and each eye movement data;

[0145] S62, at the end of a time window, acquire and read the data acquisition timestamps carried by the physiological data and eye-tracking data corresponding to the last received time within the time window, filter and obtain the timestamps to be synchronized from multiple data acquisition times according to the preset time synchronization rules, and send the timestamps to be synchronized to the preset device to correct the acquisition time of the preset device, thereby achieving acquisition time synchronization.

[0146] In this embodiment, similar to step S51 above, the receiving time is the time corresponding to each physiological data and eye-tracking data collected by the preset device, and the data acquisition timestamp carried by each physiological data and eye-tracking data is read. Subsequently, when a time window ends, the data acquisition timestamp carried by the physiological data and eye-tracking data corresponding to the last receiving time within the time window is obtained and read. The purpose of this step is to obtain the last set of data at the end of the time window for subsequent time synchronization. Next, according to preset time synchronization rules, a timestamp to be synchronized is selected from multiple data acquisition timestamps. This timestamp to be synchronized is a representative timestamp selected by the rules and serves as a reference for the acquisition time. The purpose of this step is to synchronize the time of data within a time window, ensuring that they are consistent in time. Finally, the selected timestamps to be synchronized are sent to the preset device to correct its acquisition time. This achieves the synchronization of acquisition time, ensuring the consistency of data collected in different time windows in time. This helps improve the comparability of data and provides a reliable foundation for subsequent data processing and analysis.

[0147] In one specific embodiment, Figure 7 This is a schematic flowchart illustrating a method for obtaining an early warning level based on takeover recovery time, as provided in one embodiment of this application. Based on the above embodiment, as... Figure 7 The diagram illustrates the specific relationship between takeover recovery time and early warning levels, including steps S71-S74:

[0148] S71, if the takeover recovery time is less than or equal to the first time threshold, then the warning level of the current time window is obtained by querying the takeover recovery time and is a level four warning.

[0149] S72, if the takeover recovery time is greater than the first time threshold and less than or equal to the second time threshold, then the warning level of the current time window is obtained by querying the takeover recovery time and is a level three warning.

[0150] S73, if the takeover recovery time is greater than the second time threshold and less than or equal to the third time threshold, then the warning level of the current time window is obtained by querying the takeover recovery time and is a level two warning.

[0151] S74. If the takeover recovery time is greater than the third time threshold and less than or equal to the fourth time threshold, then the warning level of the current time window, Level 1 warning, is obtained by querying the takeover recovery time.

[0152] In this embodiment, the relationship between the takeover recovery time and a first time threshold is determined. If the takeover recovery time is less than or equal to the first time threshold, the warning level for the current time window is determined to be a Level 4 warning. This means that in this case, the takeover recovery time is very short, indicating a relatively urgent situation, hence the relatively high warning level. Secondly, if the takeover recovery time is greater than the first time threshold but less than or equal to the second time threshold, the warning level for the current time window is determined to be a Level 3 warning. This indicates that the vehicle's takeover recovery time is moderate, thus the warning level can be lowered. Next, when the takeover recovery time is greater than the second time threshold but less than or equal to the third time threshold, the warning level for the current time window is determined to be a Level 2 warning. At this point, the vehicle's takeover recovery time is relatively long, and since the warning is issued in advance, it means the driver has more time to recover, thus the warning level is lowered again. Finally, when the takeover recovery time is greater than the third time threshold but less than or equal to the fourth time threshold, the warning level for the current time window is determined to be a Level 1 warning. This indicates that the vehicle's takeover recovery time is relatively long, the warning is issued much earlier, and the warning level is the lowest at this time. It should be noted that the time thresholds are determined using a clustering algorithm. Clustering algorithms can effectively divide takeover recovery times into different intervals, resulting in high similarity among the data within each interval. This facilitates more refined and accurate analysis based on the actual situation. The hierarchical division of takeover recovery times provides a basis for subsequent warning level determination. By fully utilizing the data analysis capabilities of clustering algorithms, warning levels become more reasonable and reliable. This has positive implications for improving the safety of autonomous vehicles and preventing potential hazards.

[0153] Furthermore, Figure 8 This is a flowchart illustrating a method for obtaining an early warning level and executing a corresponding request-to-takeover early warning strategy, provided in one embodiment of this application. Based on the above embodiment, as... Figure 8 The diagram illustrates a specific implementation method for obtaining the warning level based on the warning execution rules in step S13, including steps S81-S82:

[0154] S81, upon confirming the arrival of the warning time, obtain the warning level of the current time window and the warning level of the previous time window;

[0155] S82: Select the higher warning level from the warning levels of the current time window and the warning levels of the previous time window, and execute the corresponding warning strategy according to the higher warning level.

[0156] In this embodiment, the warning level of the current time window and the warning level of the previous time window are obtained to determine the warning level corresponding to the driver's takeover recovery time within the two most recent time windows. Next, the higher warning level is selected from the warning levels of the current and previous time windows. Then, the corresponding warning strategy is executed based on the higher warning level. Different warning levels may correspond to different levels of urgency and handling methods. For example, if the warning level is level four, it may mean that the driver's takeover recovery time is very short. In this case, it may be necessary to alert the driver through audible and visual alarms, seat vibration, or a human-machine interface to make them more alert. Conversely, if the warning level is level one, it may mean that the driver's takeover recovery time is longer, and a more lenient warning strategy can be adopted, such as displaying warning information through a human-machine interface.

[0157] Here's an example to illustrate the selection of warning levels. Suppose that at a certain moment, the warning level for the previous time window was Level 2, while the warning level for the current time window is Level 4. This might mean that the driver's takeover recovery time is decreasing, and the driver's state may be becoming more tense. In this case, the higher warning level, Level 4, should be selected, and the corresponding warning strategy should be implemented to alert the driver. In general, by comparing the warning levels within the two most recent time windows, the current driver's state can be determined, and an appropriate warning strategy can be selected based on this. This more accurately reflects the driver's actual state, thereby improving the effectiveness of the warning system and enhancing driving safety. Simultaneously, this also helps drivers better understand the warning information, thus enabling them to react more effectively.

[0158] In one specific embodiment, Figure 9 This is a flowchart illustrating the execution of a request-to-takeover warning strategy according to one embodiment of this application. Based on the above embodiment, as... Figure 9 The diagram illustrates the specific implementation of the takeover warning strategy in steps S13 and S82, including steps S91-S94:

[0159] S91, if the warning level is level four, the warning strategy for level four warning includes: voice prompt broadcast, displaying prompt text on the vehicle's instrument panel and central control screen, and strong vibration of the vehicle's seats;

[0160] S92, if the warning level is a level three warning, the warning strategy for a level three warning includes voice prompts, displaying prompt text on the vehicle's dashboard and central control screen, and slight vibration of the vehicle's seats;

[0161] S93, if the warning level is Level II, the warning strategy for Level II includes voice prompts and displaying prompt text on the vehicle's instrument panel and central control screen;

[0162] S94, if the warning level is Level 1, the warning strategy for Level 1 includes voice prompts.

[0163] In this embodiment, corresponding warning strategies are adopted when faced with different levels of warnings to ensure that the warning information is delivered to the driver accurately and timely. Specifically, for a level four warning, a clear and concise voice prompt is issued to the driver through the vehicle's built-in voice system to convey the emergency situation, such as the voice broadcast: "Please take over immediately"; relevant text prompts are displayed on the vehicle's dashboard and central control screen to provide a more intuitive information display, such as the text "Please take over immediately"; and strong vibrations are implemented through the vehicle seat to arouse the driver's strong perception and enhance alertness. Since the takeover recovery time corresponding to a level four warning is relatively short, that is, the advance warning time is relatively short, it is necessary for the driver to pay close attention and react quickly. Through multi-channel prompts of voice, display, and seat vibration, the driver's perception of the emergency is improved to maximize the safety of the driver and the vehicle.

[0164] Meanwhile, similar to Level 4 warnings, Level 3 warnings utilize a voice system to deliver clear warning information to the driver and display text prompts on the vehicle's visual interface to help the driver better understand the warning content. The difference is that a slight seat vibration is used to deliver a relatively gentle reminder, guiding the driver to take appropriate action. The takeover recovery time for Level 3 warnings is longer than for Level 4 warnings, and the advance warning time is also longer; therefore, the intensity of the seat reminder is reduced. For Level 2 warnings, similar to the above warnings, a concise voice system is used to inform the driver of the current situation, and relevant text prompts are displayed on the vehicle's instrument panel and central control screen to provide necessary information. There is no seat vibration reminder at this time. The takeover recovery time for Level 2 warnings is relatively long, meaning the advance warning time is also longer. Information is mainly provided through voice and display to guide the driver to pay attention and make appropriate adjustments. For Level 1 warnings, a concise voice prompt is sufficient to convey information to the driver. Level 1 warnings have the longest takeover recovery time, which again means the advance warning time is the longest. Therefore, a slow and gentle voice prompt can be used to allow the driver sufficient time to recover without startling them with a sudden warning. By employing different levels of warning strategies, drivers are provided with appropriate alerts based on their individual conditions, enabling them to drive more safely and efficiently in various driving scenarios. This helps improve the usability of the warnings and the user experience, while ensuring that there are suitable responses to various situations.

[0165] Based on the above embodiments, a specific explanation is given on the implementation of stopping the execution of the early warning strategy, and the method further includes:

[0166] After issuing a takeover request warning, if a 10% change in the vehicle's brake pedal or a 2° change in the vehicle's steering wheel angle is detected, the warning strategy will be stopped.

[0167] In this embodiment, after a takeover request warning is issued, the warning strategy is stopped when a change in the vehicle's brake pedal reaches 10% or a change in the vehicle's steering wheel angle reaches 2°. This mechanism is designed to respond more accurately to driver actions, reduce unnecessary warning duration, and minimize driver interference. Specifically, one scenario involves real-time monitoring of brake pedal changes with a set threshold. When a 10% change in brake pedal position is detected, this change is immediately identified and recorded. Changes in brake pedal position typically indicate that the driver may have taken active steps to control the vehicle, such as encountering a sudden traffic situation or needing to slow down and stop. Another scenario involves real-time monitoring of steering wheel angle changes with a set threshold. Once a 2° change in steering wheel angle is detected, this change is immediately detected and recorded. Changes in steering wheel angle typically indicate that the driver may be actively adjusting the vehicle's direction, such as to avoid obstacles, change lanes, or make other driving decisions. Once a change in brake pedal position or steering wheel angle is detected and reaches the set threshold, the currently executed warning strategy is immediately terminated, and no further warning signals are issued to the driver. To respect the driver's operation, the warning strategy will no longer be executed to avoid interfering with the driver's normal operation, thereby improving the humanization and applicability of the warning execution and providing the driver with a smoother and more natural driving experience.

[0168] Based on the above embodiments, this application collects physiological and eye-tracking data from drivers to monitor their physiological condition, attention level, and fatigue level in real time, thereby gaining a comprehensive understanding of their driving status. Based on a preset time window, the physiological and eye-tracking data are analyzed and preprocessed to obtain a training dataset. Clustering and random forest algorithms are applied to determine warning levels, time thresholds, and corresponding warning strategies, providing a more driver-friendly warning mechanism. Furthermore, different warning mechanisms can be flexibly customized for different drivers. Regarding the acquisition of takeover recovery time, a predicted value is obtained through model training, providing more accurate prediction capabilities. Different warning methods are used for different warning levels, such as voice prompts, displayed text, and vehicle seat vibration, to improve the targeted nature and user-friendliness of the warning effect. By detecting changes in the vehicle's brake pedal and steering wheel, the warning strategy is adjusted in real time to avoid continuous and unnecessary warnings to the driver, creating a smoother and more natural driving experience and improving the adaptability and comfort of the warning strategy. Simultaneously, a time synchronization mechanism ensures that the collected data has a consistent time reference, improving the accuracy and reliability of the data. In summary, by integrating multi-source data, predictive models, and early warning strategies, as well as coordinating with driver behavior, autonomous driving systems have improved safety, humanization, and user experience, making them more adaptable to different driving environments and driver needs.

[0169] Figure 10 This is a schematic diagram of the structure of an autonomous vehicle warning system provided in one embodiment of this application. Figure 10 As shown, the vehicle warning system includes a data acquisition subsystem 101, a data processing subsystem 102, and a takeover warning subsystem 103.

[0170] The data acquisition subsystem 101 includes: a physiological data acquisition device 1011 responsible for collecting physiological data of the driver, including electrocardiogram signals, skin conductance signals and electromyogram signals; an eye movement data acquisition device 1012 responsible for collecting eye movement data of the driver, including blink frequency, blink duration and fixation information; and a wireless transceiver 1013 responsible for wirelessly transmitting the collected physiological data and eye movement data to the data processing subsystem 102.

[0171] The data processing subsystem 102 includes: a wireless transceiver 1021 that receives physiological data and eye-tracking data transmitted from the data acquisition subsystem; a data synchronization module 1022 that processes the received data, performs time synchronization, and ensures the consistency of the data's time stamps; a physiological data preprocessing module 1023 that performs denoising, anti-spoofing, and normalization processing on the acquired physiological data; an eye-tracking data preprocessing module 1024 that performs denoising, anti-spoofing, and normalization processing on the acquired eye-tracking data; and a takeover time prediction module 1025 that uses the processed data to perform calculations and analysis to predict the driver's takeover recovery time.

[0172] The takeover warning subsystem 103 includes: a voice module 1031 for providing voice prompts based on the output of the takeover prediction module, including broadcasting corresponding warning information; a display module 1032 for displaying prompt text related to the prediction results on the vehicle dashboard and central control screen to attract the driver's attention; and a vibration module 1033 for applying vibration to the vehicle seat to provide vibration feedback of different intensities to enhance the transmission of warning information.

[0173] In this embodiment of the invention, electronic devices or main control devices can be divided into functional modules according to the above method examples. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional module. It should be noted that the module division in this embodiment of the invention is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0174] Figure 11 This is a schematic diagram of the pipe-handling device provided in an embodiment of this application. Figure 11 As shown, the pipe handling device 11 includes:

[0175] The data acquisition and processing module 111 is used to acquire the data to be filtered corresponding to the current time window based on the physiological data and eye movement data of the driver at all collection moments in the current time window, and to filter and calculate the data to be filtered according to multiple preset data indicators and the data type corresponding to each data indicator, so as to obtain the data to be predicted corresponding to the current time window. The data to be predicted is data that meets the data indicators and the corresponding data type.

[0176] The takeover recovery time prediction module 112 is used to obtain the takeover recovery time when the driver recovers from the non-takeover state to the takeover state in the current time window based on the data to be predicted corresponding to the current time window and through the trained time prediction model.

[0177] The takeover warning processing module 113 is used to obtain the autonomous driving failure time of the vehicle, and based on the takeover recovery time and the autonomous driving failure time, when the warning time is confirmed to have arrived, obtain the warning level that matches the preset warning execution rules and execute the corresponding request takeover warning strategy to realize the takeover warning reminder of the autonomous vehicle.

[0178] The pipe connection processing device provided in this embodiment can execute the pipe connection processing method of the above embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0179] In the specific implementation of the aforementioned takeover processing method, each module can be implemented as a processor, which can execute computer execution instructions stored in memory, thereby enabling the processor to execute the aforementioned takeover processing method.

[0180] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 12 As shown, the electronic device 12 includes at least one processor 121 and a memory 122. The electronic device 12 also includes a communication component 123. The processor 121, the memory 122, and the communication component 123 are connected via a bus 124.

[0181] In the specific implementation process, at least one processor 121 executes computer execution instructions stored in memory 122, causing at least one processor 121 to execute the takeover processing method executed on the electronic device side as described above.

[0182] The specific implementation process of processor 121 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0183] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0184] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage.

[0185] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0186] The above description of the functions implemented by electronic devices and main control devices has introduced the solutions provided by the embodiments of the present invention. It is understood that, in order to implement the above functions, the electronic device or main control device includes hardware structures and / or software modules corresponding to the execution of each function. By combining the units and algorithm steps of the various examples described in the embodiments of the present invention, the embodiments of the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of the embodiments of the present invention.

[0187] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described takeover processing method.

[0188] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0189] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in an electronic device or a host device.

[0190] This application also provides a computer program product, comprising: a computer program stored in a readable storage medium, wherein at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the scheme provided in any of the above embodiments.

[0191] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0192] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for handling takeover of an autonomous vehicle, characterized in that, include: Based on the physiological and eye movement data of the driver at all times during the current time window, the data to be filtered corresponding to the current time window is obtained. The data to be filtered and calculated according to multiple preset data indicators and the data type corresponding to each data indicator to obtain the data to be predicted corresponding to the current time window. The data to be predicted is data that meets the data indicators and the corresponding data type. Based on the data to be predicted corresponding to the current time window, the takeover recovery time of the driver from the non-takeover state to the takeover state in the current time window is obtained through the trained time prediction model. The autonomous driving failure time of the vehicle is obtained, and based on the takeover recovery time and the autonomous driving failure time, when the warning time is confirmed to have arrived, the warning level matching the preset warning execution rules is obtained and the corresponding request takeover warning strategy is executed to realize the takeover warning reminder of the autonomous vehicle.

2. The method according to claim 1, characterized in that, The acquisition of multiple data metrics and the data type corresponding to each data metric includes: Based on the acquired training dataset, a clustering algorithm is used to obtain the warning level and the warning time range corresponding to each warning level; wherein, the training dataset consists of physiological data of different data types and eye movement data of different data types, as well as the driver's takeover recovery time; The training dataset and the warning time range corresponding to each warning level are used to perform importance identification calculations through the random forest algorithm to obtain the importance of the physiological data and eye movement data of each data type in the training set on the takeover recovery time. Based on the importance of the physiological data and eye movement data of each data type to the takeover recovery time, the data types of physiological data and eye movement data with an importance greater than a set threshold are selected from the different data types of physiological data and eye movement data as the multiple data indicators and the data type corresponding to each data indicator.

3. The method according to claim 2, characterized in that, Obtaining the training dataset includes: Based on a preset time window, multiple physiological data and multiple eye movement data of the driver in the non-disposal state are collected, and when a takeover request warning is issued, the takeover recovery time of the driver from the current non-disposal state to the takeover state is obtained. The multiple physiological data and multiple eye movement data are sequentially subjected to denoising, anti-spoofing, and normalization processing to obtain multiple physiological data and multiple eye movement data to be screened. Based on multiple preset data type dimensions, calculate the data type value for each of the physiological data to be screened, and the data type value for each of the eye movement data to be screened; The training dataset is obtained by combining the takeover recovery time corresponding to the time window, the physiological data of the multiple data types, and the eye-tracking data of the multiple data types.

4. The method according to claim 1, characterized in that, The process involves obtaining the data to be filtered corresponding to the current time window based on the driver's physiological and eye-tracking data collected at all times within the current time window, including: Acquire physiological data of the driver at all collection points within the current time window; the physiological data includes the driver's electrocardiogram, electrodermal signal, and electromyography signal. The electrocardiogram (ECG), electrodermal (ED) signal, and electromyography (EMG) signal are sequentially denoised and despoofed to obtain denoised and despoofed ECG, EED, and EMG signals. The 2% and 98% percentile values ​​of the denoised and artifact-free electrocardiogram (ECG), electrodermal (ED) signals, and electromyography (EMG) signals were selected using the following formula: Normalized electrocardiogram (ECG) signal V'1, electrodermal conductance (EDA) signal V'2, and electromyography (EMG) signal V'3 were obtained respectively. wherein, when the normalized electrocardio signal V'1 is calculated, V1 is the electrocardio signal after denoising and deartifacting, V min is the 2% percentile value of the electrocardio signal in the current time window, V max is the 98% percentile value of the electrocardio signal in the current time window; or, When calculating the normalized skin conductance signal V'2, V2 is the denoised and destigmatized skin conductance signal. min V represents the 2nd percentile value of the electrodermal signal within the current time window. max This represents the 98th percentile value of the electrodermal signal within the current time window. or, When calculating the normalized electromyographic signal V'3, V3 is the denoised and artifact-free electromyographic signal. min V represents the 2nd percentile value of the electromyographic signal within the current time window. max This represents the 98th percentile of the electromyographic signal within the current time window.

5. The method according to claim 1, characterized in that, The process involves obtaining the data to be filtered corresponding to the current time window based on the driver's physiological and eye-tracking data collected at all times within the current time window, including: Acquire eye movement data of the driver at all times within the current time window; the eye movement data includes the driver's blink frequency, blink duration, and fixation information; The blink frequency, blink duration, and gaze information are sequentially subjected to denoising and artifact removal processes to obtain denoised and artifact-removed blink frequency, blink duration, and gaze information; wherein, the following formula is used: Obtain the gaze information F(t); The driver's gaze position at a certain sampling time is f(t). When the driver's gaze position is in the preset non-driving area, f(t) = 1; when the driver's gaze position is in the preset driving area, f(t) = 0; T is the time window, and r is the preset sampling rate. The 20th and 80th percentile values ​​for the denoised and destigmatized blink frequency, blink duration, and fixation information were selected using the following formula: The normalized blink frequency V'1, blink duration V'2, and fixation information V'3 to be selected are obtained respectively. Wherein, when calculating the normalized blink frequency V'1, V1 is the blink frequency after noise and artifact removal, V min V represents the 20th percentile of blink frequency within the current time window. max This represents the 80th percentile of blink frequency within the current time window. or, When calculating the normalized blink duration V'2, V2 is the blink duration after denoising and artifact removal. min V is the 20th percentile value of the blink duration within the current time window. max The 80th percentile value of the blink duration within the current time window; or, When calculating the normalized gaze information V'3, V3 is the gaze information after denoising and artifact removal. min V represents the 20th percentile of gaze information within the current time window. max This represents the 80th percentile of gaze information within the current time window.

6. The method according to claim 1, characterized in that, The time prediction model is constructed from a generalized nonlinear regression equation, using the following formula: y=f(x i )+ε,i=1,2,…,j Obtain the takeover recovery time y; where x i Let f(x) be the i-th data metric. i ) is a nonlinear function, and ε is a random error.

7. The method according to claim 3, characterized in that, The step of obtaining the takeover recovery time when the driver returns to the takeover state from the current non-takeover state includes: Record the time when the takeover request warning is issued, the time when the vehicle steering wheel angle changes by 2° after the takeover request warning is issued, and the time when the vehicle brake pedal changes by 10% after the takeover request warning is issued. If the time of the 2° change in the vehicle steering wheel angle is earlier than the time of the 10% change in the vehicle brake pedal, then the takeover recovery time is the time of the angle change minus the request time. or, If the braking change time of 10% change in the vehicle's brake pedal is earlier than the angle change time of 2° change in the vehicle's steering wheel angle, then the takeover recovery time is the braking change time minus the request time.

8. The method according to claim 1, characterized in that, The method further includes: The time corresponding to each physiological data and each eye movement data collected by the preset device is recorded as the receiving time, and the data collection timestamp carried by each physiological data and each eye movement data is read. If multiple data collection timestamps corresponding to the same receiving time are inconsistent, then according to the preset time synchronization rules, the timestamp to be synchronized is selected from the multiple data collection timestamps and sent to the preset device to correct the collection time of the preset device, so as to achieve collection time synchronization. or, At the end of a time window, the data acquisition timestamps carried by the physiological data and eye-tracking data corresponding to the last received time within the time window are obtained and read. According to the preset time synchronization rules, the timestamps to be synchronized are selected from multiple data acquisition times and sent to the preset device to correct the acquisition time of the preset device, thereby achieving acquisition time synchronization.

9. The method according to claim 1, characterized in that, After obtaining the takeover recovery time when the driver returns to the takeover state from the non-takeover state in the current time window and before the confirmation warning time arrives, the method further includes: If the takeover recovery time is less than or equal to the first time threshold, then the warning level of the current time window is obtained by querying the takeover recovery time and is a level four warning. or, If the takeover recovery time is greater than the first time threshold and less than or equal to the second time threshold, then the warning level of the current time window is obtained by querying the takeover recovery time as a level three warning. or, If the takeover recovery time is greater than the second time threshold and less than or equal to the third time threshold, then the warning level of the current time window is obtained by querying the takeover recovery time as a level two warning. or, If the takeover recovery time is greater than the third time threshold and less than or equal to the fourth time threshold, then the warning level of the current time window, Level 1 warning, is obtained by querying based on the takeover recovery time.

10. The method according to claim 9, characterized in that, The step of obtaining the warning level that matches the preset warning execution rules and executing the corresponding request takeover warning strategy when the warning time is confirmed includes: Upon confirmation that the warning time has arrived, obtain the warning level of the current time window and the warning level of the previous time window; The system selects the higher warning level from the warning levels of the current time window and the previous time window, and executes the corresponding warning strategy according to the higher warning level.

11. The method according to any one of claims 1 or 10, characterized in that, The step of obtaining the warning level that matches the preset warning execution rules and executing the corresponding request takeover warning strategy includes: If the warning level is a level four warning, the warning strategy for the level four warning includes: voice prompt broadcast, displaying prompt text on the vehicle dashboard and central control screen, and strong vibration of the vehicle seats; or, If the warning level is a Level 3 warning, the warning strategy for the Level 3 warning includes voice prompts, displaying prompt text on the vehicle dashboard and central control screen, and slight vibration of the vehicle seats. or, If the warning level is a Level 2 warning, the warning strategy for the Level 2 warning includes voice prompts and displaying prompt text on the vehicle's dashboard and central control screen. or, If the warning level is Level 1, then the warning strategy for Level 1 includes voice prompts.

12. The method according to any one of claims 1-10, characterized in that, The method further includes: After issuing a takeover request warning, if a 10% change in the vehicle's brake pedal or a 2° change in the vehicle's steering wheel angle is detected, the warning strategy will be stopped.

13. A pipe-connecting processing device, characterized in that, include: The data acquisition and processing module is used to acquire the data to be filtered corresponding to the current time window based on the physiological data and eye movement data of the driver at all collection moments in the current time window, and to filter and calculate the data to be filtered according to multiple preset data indicators and the data type corresponding to each data indicator, so as to obtain the data to be predicted corresponding to the current time window, wherein the data to be predicted is data that meets the data indicators and the corresponding data type. The takeover recovery time prediction module is used to obtain the takeover recovery time of the driver from the non-takeover state to the takeover state in the current time window based on the data to be predicted corresponding to the current time window and through a trained time prediction model. The takeover warning processing module is used to obtain the autonomous driving failure time of the vehicle, and based on the takeover recovery time and the autonomous driving failure time, when the warning time is confirmed to have arrived, obtain the warning level that matches the preset warning execution rules and execute the corresponding request takeover warning strategy to realize the takeover warning reminder of the autonomous vehicle.

14. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 12.

Images