Automatic driving takeover displacement prediction method and device, vehicle, medium and product

Abstract

The application discloses an automatic driving takeover displacement prediction method and device, a vehicle, a medium and a product. The method comprises the following steps: when an automatic driving system detects that a driver needs to take over, acquiring driver physiological attribute data and vehicle operation data; performing feature extraction processing on the physiological attribute data and the vehicle operation data to obtain physiological attribute features and vehicle operation features; performing feature fusion processing on the physiological attribute features and the vehicle operation features to obtain to-be-predicted fusion data; inputting the to-be-predicted fusion data into a pre-trained supervised machine learning model to perform takeover displacement prediction and obtain a takeover displacement prediction result; and the takeover displacement prediction result comprises vehicle displacement prediction data within a period of time after the driver takes over. The application realizes driver emotional state recognition in different driving scenarios.

Description

Technical Field

[0001] This application relates to the field of emotion recognition technology, and in particular to autonomous driving takeover displacement prediction methods, devices, vehicles, media, and products. Background Technology

[0002] In today's world, automobiles are the primary mode of transportation. Every year, a large number of people die in traffic accidents. Research shows that autonomous driving technology can effectively reduce traffic accidents caused by human error. Level 3 autonomous vehicles represent a transitional stage in the development of fully intelligent vehicles. They require drivers to monitor the vehicle and traffic conditions. When encountering vehicle malfunctions or complex traffic scenarios, the system needs to switch from autonomous driving to manual driving; the process of the driver taking over the vehicle is crucial.

[0003] At present, research mainly focuses on algorithm optimization and vehicle perception systems in autonomous driving technology. Research on the role of the driver in autonomous driving systems, vehicle displacement prediction after driver takeover, and correlation analysis of vehicle and driver physiological data is relatively scarce.

[0004] As autonomous driving technology matures, there is an urgent need for a method to predict L3 autonomous driving takeover displacement by integrating driver physiological information and vehicle data, in order to plan routes in advance, avoid risks, and meet the requirements of autonomous driving safety and application effectiveness. Summary of the Invention

[0005] The main objective of this application is to provide a method, device, vehicle, medium, and product for predicting the displacement of autonomous driving takeover, aiming to solve the technical problem that autonomous driving takeover displacement prediction is impossible in related technologies.

[0006] To achieve the above objectives, in a first aspect, this application provides a method for predicting displacement during autonomous driving takeover, the method comprising: When the autonomous driving system detects that the driver needs to take over, it acquires the driver's physiological attribute data and vehicle operation data; Feature extraction processing is performed on physiological attribute data and vehicle operation data to obtain physiological attribute features and vehicle operation features; Physiological attribute features and vehicle operation features are fused to obtain the fused data to be predicted. The fused data to be predicted is input into a pre-trained supervised machine learning model to perform takeover displacement prediction and obtain the takeover displacement prediction results; the takeover displacement prediction results include vehicle displacement prediction data for a period of time after the driver takes over.

[0007] In one embodiment, the training method for a supervised machine learning model includes: Physiological attribute training data and corresponding vehicle operation training data of effective participants were collected during the experiment; effective participants included subjects with different sleep durations. Analyze the vehicle displacement data of the effective participants after the driver takeover process to determine the vehicle displacement data after the takeover process; After feature extraction processing of physiological attribute training data and vehicle operation training data respectively, they are fused together. The vehicle displacement data after the takeover process is used as the label to construct a training sample set. Based on the training sample set, a supervised machine learning model is trained using a supervised regression method that involves forward propagation, loss calculation, and backpropagation to update parameters, thus obtaining a pre-trained supervised machine learning model.

[0008] In one embodiment, the features of the physiological attribute training data include the time-domain features, frequency-domain features, and nonlinear features of the physiological attribute training data; Physiological attributes and vehicle operation characteristics are fused using a rectangular stitching method.

[0009] In one embodiment, the steps of performing feature extraction processing on physiological attribute training data and vehicle operation training data respectively include: Determine the mean and root mean square of the physiological attribute data; Determine data kurtosis and data skewness based on the data mean; Based on the mean and root mean square of the data, determine the waveform factor and peak factor; Determine the temporal characteristics of physiological attribute training data.

[0010] In one embodiment, the steps of performing feature extraction processing on physiological attribute training data and vehicle operation training data respectively include: The physiological attribute data is divided into multiple overlapping segments of a preset length; overlapping segments indicate that there is partial overlap between adjacent data segments. After windowing the overlapping segments; Calculate the Fourier transform of each windowed overlapping segment to determine the power spectral density of each overlapping segment. Based on the power spectral density of each overlapping segment, the frequency domain characteristics of the physiological attribute training data are determined.

[0011] In one embodiment, the steps of performing feature extraction processing on physiological attribute training data and vehicle operation training data respectively include: The physiological attribute training data is serialized and sequentially assembled into a multidimensional vector. Calculate the approximate entropy, sample entropy, and permutation entropy values ​​of the multidimensional vector to determine the nonlinear characteristics of the physiological attribute training data.

[0012] Secondly, to achieve the above objectives, this application further provides an automatic driving takeover displacement prediction device, the device comprising: The data acquisition module is used to acquire driver physiological attribute data and vehicle operation data when the autonomous driving system detects that driver takeover is required; The feature extraction module is used to perform feature extraction processing on physiological attribute data and vehicle operation data to obtain physiological attribute features and vehicle operation features. The feature fusion module is used to perform feature fusion processing on physiological attribute features and vehicle operation features to obtain the fused data to be predicted; The fused data to be predicted is input into a pre-trained supervised machine learning model to perform takeover displacement prediction and obtain the takeover displacement prediction results; the takeover displacement prediction results include vehicle displacement prediction data for a period of time after the driver takes over.

[0013] Thirdly, to achieve the above objectives, this application further provides a vehicle, the device including: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the above-described autonomous driving takeover displacement prediction method.

[0014] Fourthly, to achieve the above objectives, this application further provides a storage medium, characterized in that the storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium, wherein when the computer program is executed by a processor, the steps of the above-mentioned automatic driving takeover displacement prediction method are implemented.

[0015] Fifthly, to achieve the above objectives, this application further provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the above-described automatic driving takeover displacement prediction method.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: This application, by integrating driver physiological attribute data and vehicle operation data when an autonomous driving system detects a takeover request, predicts vehicle displacement within a certain period after the driver takes over. This enables early perception and risk prediction of the vehicle's movement trend after takeover, improving the accuracy and reliability of takeover behavior analysis compared to methods that rely solely on vehicle state analysis. Furthermore, by extracting temporal, frequency, and nonlinear features from the physiological attribute data and combining them with vehicle operation characteristics for fusion modeling, the model's ability to represent driver fatigue, cognitive state, and differences in takeover capability is enhanced, thereby improving the accuracy of takeover displacement prediction. In addition, training the supervised machine learning model using supervised regression effectively reduces vehicle displacement prediction errors, providing technical support for autonomous driving systems to plan vehicle trajectories in advance, assist in risk avoidance, and improve the safety of L3 autonomous driving takeovers. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the autonomous driving takeover displacement prediction method in the embodiments of this application.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] The main solution of this application embodiment is as follows: when the autonomous driving system detects that the driver needs to take over, it acquires the driver's physiological attribute data and vehicle operation data, performs feature extraction processing on the physiological attribute data and vehicle operation data to obtain corresponding physiological attribute features and vehicle operation features, and performs fusion processing on the physiological attribute features and vehicle operation features to obtain fused data to be predicted; then, the fused data to be predicted is input into a pre-trained supervised machine learning model to predict the takeover displacement, and outputs the vehicle displacement prediction data for a period of time after the driver takes over.

[0024] Specifically, this application provides a method for predicting displacement during autonomous driving takeover, referring to... Figure 1 In this embodiment, the autonomous driving takeover displacement prediction method includes steps S10 to S40: Step S10: When the autonomous driving system detects that the driver needs to take over, it acquires the driver's physiological attribute data and vehicle operation data. Step S20: Perform feature extraction processing on physiological attribute data and vehicle operation data to obtain physiological attribute features and vehicle operation features; Step S30: Perform feature fusion processing on physiological attribute features and vehicle operation features to obtain the fused data to be predicted; Step S40: Input the fused data to be predicted into the pre-trained supervised machine learning model to perform the nozzle displacement prediction and obtain the nozzle displacement prediction result.

[0025] The takeover displacement prediction results include vehicle displacement prediction data for a period of time after the driver takes over.

[0026] In one feasible implementation, the training methods for supervised machine learning models include: Physiological attribute training data and corresponding vehicle operation training data of effective participants were collected during the experiment; the effective participants included subjects with different sleep durations.

[0027] The vehicle displacement data after the effective participant takes over the driving process is analyzed to determine the vehicle displacement data after the takeover process.

[0028] After feature extraction processing of the physiological attribute training data and vehicle operation training data respectively, they are fused together, and the vehicle displacement data after the takeover process is used as the label to construct a training sample set.

[0029] Based on the training sample set, the supervised machine learning model is trained using a supervised regression method of forward propagation—loss calculation—parameter update through backpropagation, thus obtaining the pre-trained supervised machine learning model.

[0030] The features of the physiological attribute training data include the time domain features, frequency domain features, and nonlinear features of the physiological attribute training data.

[0031] The physiological attribute features and the vehicle operation features are fused together using a rectangular stitching method.

[0032] The steps of performing feature extraction processing on the physiological attribute training data and the vehicle operation training data respectively include: Determine the mean and root mean square of the physiological attribute data; Based on the data mean, determine the data kurtosis and data skewness; Based on the mean and root mean square of the data, the waveform factor and peak factor are determined; Determine the temporal characteristics of the physiological attribute training data.

[0033] The physiological attribute data is divided into multiple overlapping segments of a preset length; the overlapping segments indicate that there is partial overlap between adjacent data segments. After windowing processing of each of the aforementioned overlapping segments; Calculate the Fourier transform of each windowed overlapping segment to determine the power spectral density of each overlapping segment; The frequency domain characteristics of the physiological attribute training data are determined based on the power spectral density of each overlapping segment.

[0034] The physiological attribute training data is serialized and sequentially assembled into a multidimensional vector. Calculate the approximate entropy, sample entropy, and permutation entropy values ​​of the multidimensional vector to determine the nonlinear characteristics of the physiological attribute training data.

[0035] Specifically, the autonomous driving takeover displacement prediction method provided in this application first acquires driver physiological attribute data and vehicle operation data simultaneously when the autonomous driving system detects the need for driver takeover. The driver physiological attribute data characterizes the driver's physiological state at the moment of takeover, while the vehicle operation data characterizes the motion and control state of the autonomous vehicle at the time of takeover triggering, such as vehicle speed and vehicle position.

[0036] Furthermore, feature extraction processing is performed on the physiological attribute data and vehicle operation data respectively to obtain physiological attribute features and vehicle operation features. The physiological attribute features preferably include time-domain features, frequency-domain features, and nonlinear features, used to characterize the driver's fatigue level, arousal level, tension state, and physiological fluctuation patterns from different perspectives. Vehicle operation features are used to reflect the trend of vehicle operation changes during takeover. Specifically, time-domain features describe the amplitude variation of physiological attribute data over time, and may include indicators such as data mean, root mean square (RMS), kurtosis, skewness, waveform factor, and peak factor. The mean and RMS reflect the overall level and energy distribution of the signal, kurtosis and skewness reflect the concentration and asymmetry of the signal waveform, and waveform factor and peak factor characterize the peak value variation characteristics of the signal. Through these time-domain features, the instantaneous changes of the driver's physiological signals during takeover can be more intuitively characterized, thus providing basic feature support for takeover displacement prediction. Frequency domain features are used to describe the distribution patterns of physiological attribute data along the frequency dimension. Specifically, this involves dividing the physiological attribute data into multiple overlapping segments of a predetermined length, windowing each overlapping segment, calculating the Fourier transform, and determining the power spectral density. By analyzing the power spectral density, the energy distribution characteristics of the signal at different frequency components can be obtained, thereby characterizing the periodicity and stability of changes in the driver's physiological state. This method helps to reveal the intrinsic changing patterns of the driver's physiological signals during takeover from a frequency domain perspective, improving the comprehensiveness of feature extraction. Nonlinear features are used to characterize the complexity and randomness of physiological attribute data. Specifically, this can be achieved by serializing the physiological attribute training data and sequentially assembling it into a multidimensional vector, then calculating the approximate entropy, sample entropy, and permutation entropy values ​​of the multidimensional vector. Approximate entropy and sample entropy are used to quantify the complexity and regularity of the time series, while permutation entropy characterizes the ordered arrangement and dynamic complexity of the sequence.

[0037] After feature extraction, physiological attribute features and vehicle operation features are fused using a rectangular concatenation method to obtain fused data to be predicted. This fused data is then input into a pre-trained supervised machine learning model for takeover displacement prediction, outputting vehicle displacement prediction data for a period of time after the driver takes over. In the model training method, physiological attribute training data and corresponding vehicle operation training data of subjects with different sleep durations during the experiment are used as the foundation. Vehicle displacement data after the takeover process is used as the label to construct the training sample set. Model training is completed using a supervised regression method based on forward propagation, loss calculation, and backpropagation to update parameters. This method enables the model to learn the mapping relationship between physiological state, vehicle state, and takeover displacement, thereby achieving accurate prediction of the vehicle's movement trend after takeover.

[0038] For example, data was collected, and 28 valid participants were selected, including 22 women and 6 men, with a mean age of 23.25 and a variance of 7.24. Their sleep information from the previous night was recorded using a sleep tracker (smart bracelet), and they were divided into those experiencing partial sleep deprivation (sleep time less than 6 hours) and those experiencing sufficient sleep (sleep time more than 8 hours). Among them, there were 9 participants experiencing partial sleep deprivation and 19 participants experiencing sufficient sleep.

[0039] After grouping participants, an experimental scenario was set up to simulate a takeover by an autonomous vehicle, and physiological and vehicle data were collected during the experiment. The BioPac Student Lab 3.7.7 software and BioPac MP36 hardware were used to collect participants' physiological data, such as heart rate, blood pressure, skin conductance, electroencephalography (EEG), and respiratory rate. Data preprocessing includes checking the integrity of vehicle and driver physiological data, filling in missing data, handling abnormal and duplicate vehicle data, and processing driver electrodermatology data; wavelet denoising is used to process driver heart rate data. Code is designed to import driver physiological and vehicle data into Python and clean the dataset.

[0040] The takeover process is described based on the dataset, and the data generated during the takeover process is summarized to determine the training dataset.

[0041] This example uses driver physiological data during driver takeover, specifically: RR Interval: The time interval between two consecutive heartbeats.

[0042] RR Interval = T 当前R波峰值时间戳 T 前一个R波峰值时间戳 Heart rate standard deviation in, , Takeover response time TOT = T r T c Wherein: Tr is the time when the system issues a takeover warning; Tc is the time when the driver takes over.

[0043] Takeover Control Time (TCT) = T R T c Wherein: T RThe system issues a takeover alert at the moment Tc indicates that the driver has taken control of the vehicle. The physiological signal features of the driver are extracted. These features include time-domain features, frequency-domain features, and nonlinear features. The relevant calculations are as follows: Temporal characteristics: Mean: Variance: Root Mean Square (RMS) Within the time-domain analysis framework of electrodermal signals, the corresponding formulas for calculating kurtosis and skewness are as follows: Where K represents kurtosis and S represents skewness. Crest Factor: Where B represents the waveform factor, F represents the peak factor, and N is the number of samples. is the mean squared error, i = 1, 2, 3...N, and is the sample value of the nth sampling point. Frequency domain characteristics: Power spectral density: The specific calculation steps are as follows: Segmentation: The original signal x[n] is divided into multiple overlapping segments, with an overlap of 50%, and each segment has a length of N. For example, for a signal x[n] of length L, it is divided into K overlapping segments, each of length N, with an overlap length of D (usually D=N / 2). A window function is introduced to reduce the spectral leakage w[n] introduced during signal truncation; each segment is multiplied by the window function. Calculation of the Fourier Transform for each segment: The Discrete Fourier Transform (DFT) is calculated for each windowed segment. Calculate the power spectral density: Where U is the normalization factor of the window function: Averaged power spectral density: Centroid frequency (CF) is the overall frequency characteristic of a system during vibration or wave motion, i.e., the average frequency of the signal. It is derived by weighting the average frequency, with the weights corresponding to the power spectral density values.

[0044] Where fk is the frequency, and P(fk) is the corresponding power spectral density value. Mean square frequency: the average frequency distribution in the frequency domain.

[0045] Among them, f k It is the k-th frequency component. is the power spectral density value of that frequency component, and N is the signal length. Root mean square frequency (RMSF): The arithmetic square root mean of the power spectral density of each frequency component of an electrodermal signal over a specific time period. Its value represents the equivalent center frequency of the signal power distribution. The RMSF value can effectively distinguish the energy distribution characteristics of different frequency bands, providing a quantitative indicator for the analysis of signal frequency domain characteristics.

[0046] Frequency variance (VF): This is obtained by calculating the weighted average of the squared deviations of each frequency component from the mean. This parameter is used to characterize the dispersion of the signal's spectral distribution. Nonlinear characteristics: Approximate Entropy (AE): A nonlinear statistic used to quantify the complexity of time series data, characterizing the randomness and regularity of signals. The algorithm principle of approximate entropy is as follows: Given the sequence x(1), x(2), x(N) are arranged in order to form an m-dimensional vector, that is In the formula: 1≤i≤N-m+1 Define the distance between vectors ( and vector () as ,Right now In the formula: 1≤k≤m-1; 1≤i; j≤N-m+1; i≠j Given a threshold r (r>0), statistics are performed for each value i. The number of vectors <r and its ratio to the total number of vectors N-m+1, i.e. Pick The logarithm of and its average over all i, i.e. Repeat the above process for the vector at point m+1 to obtain Then the approximate entropy is Sample Entropy (SE): An optimized and improved version of approximate entropy, used to quantify the complexity of time series data. The algorithm principle is as follows: The first and second steps are the same as the first and second steps of calculating the approximate entropy.

[0047] Given a threshold r (r>0), statistics are performed for each value i. The number of vectors and its ratio to the total number of vectors N-m+1 are denoted as : In the formula: calculate its average value for all terms i. For the vector at point m+1, the same applies. In the formula, 1 < j < Nm, i ≠ j. Find its average value for all i. The sample entropy of this sequence is Permutation Entropy (PE): A metric for measuring the complexity of a time series, focusing on the statistical properties of the time series. Reconstructing the phase space yields the following matrix: The permutation entropy is: The fused feature values ​​are input into a supervised machine learning model to predict vehicle position coordinates. The model performance is analyzed using six key evaluation metrics: coefficient of determination (R²), mean absolute error (MAE), mean square error (MSE), root mean square error (RMSE), Akaike information criterion (AIC), and Bayesian information criterion (BIC).

[0048] The merged dataset is then input into five representative supervised machine learning models, such as decision tree model, gradient boosting model, random forest model, support vector machine model, and neural network model, to predict vehicle position coordinates. The specific formulas for the neural network model include: Neuron output formula: in, Representative input and The inner product; Representative activation function Forward propagation formula: in It is the activation value of the Lth layer; This represents the weight matrix of the Lth layer; This represents the bias vector of the Lth layer.

[0049] Common loss functions include mean squared error and cross-entropy loss, and their formulas are as follows: in, Represents the actual label; This represents the predicted value.

[0050] Backpropagation formula: This example continues to provide an autonomous vehicle equipped with a Level 3 autonomous driving takeover control system. The vehicle is tested in multiple different scenarios, such as urban and rural areas, and the takeover displacement is predicted through a neural network model.

[0051] The neural network model has an R² score of 0.78, which is relatively close to the optimal model. However, it has a significant advantage in the five metrics of MSE, RMSE, MAE, AIC, and BIC. This model is superior in reducing prediction errors and controlling model complexity.

[0052] This invention improves prediction accuracy by integrating driver physiological information and vehicle data to build a neural network model, thus ensuring the safety and reliability of the system. High-precision displacement prediction helps autonomous driving systems plan routes in advance, avoid risks, reduce accident rates, and promote the development of autonomous driving technology.

[0053] In another embodiment, this application further provides an autonomous driving takeover displacement prediction device, the device comprising: The data acquisition module is used to acquire driver physiological attribute data and vehicle operation data when the autonomous driving system detects that driver takeover is required; The feature extraction module is used to perform feature extraction processing on physiological attribute data and vehicle operation data to obtain physiological attribute features and vehicle operation features. The feature fusion module is used to perform feature fusion processing on physiological attribute features and vehicle operation features to obtain the fused data to be predicted; The fused data to be predicted is input into a pre-trained supervised machine learning model to perform takeover displacement prediction and obtain the takeover displacement prediction results; the takeover displacement prediction results include vehicle displacement prediction data for a period of time after the driver takes over.

[0054] The autonomous driving takeover displacement prediction device provided in this application, employing the autonomous driving takeover displacement prediction method in the above embodiments, can solve the technical problem of the inability to perform autonomous driving takeover displacement prediction in related technologies. Compared with related technologies, the beneficial effects of the autonomous driving takeover displacement prediction device provided in this application are the same as those of the autonomous driving takeover displacement prediction method provided in the above embodiments, and other technical features in the autonomous driving takeover displacement prediction device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0055] This application provides a vehicle, the vehicle including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the autonomous driving takeover displacement prediction method in the above embodiments.

[0056] The vehicle provided in this application, employing the autonomous driving takeover displacement prediction method described in the above embodiments, can solve the technical problem in related technologies where autonomous driving takeover displacement prediction is impossible. Compared with related technologies, the beneficial effects of the vehicle provided in this application are the same as those of the autonomous driving takeover displacement prediction method provided in the above embodiments, and other technical features of the vehicle are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0057] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0058] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0059] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the autonomous driving takeover displacement prediction method in the above embodiments.

[0060] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0061] The aforementioned computer-readable storage medium may be included in the vehicle or may exist independently and not installed in the vehicle.

[0062] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0063] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0064] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0065] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described autonomous driving takeover displacement prediction method, thereby solving the technical problem in related technologies where autonomous driving takeover displacement prediction is not possible. Compared with related technologies, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the autonomous driving takeover displacement prediction method provided in the above embodiments, and will not be repeated here.

[0066] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described autonomous driving takeover displacement prediction method.

[0067] The computer program product provided in this application can solve the technical problem of the inability to predict the displacement during autonomous driving takeover in related technologies. Compared with related technologies, the beneficial effects of the computer program product provided in this application are the same as those of the autonomous driving takeover displacement prediction method provided in the above embodiments, and will not be repeated here.

[0068] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for predicting displacement during autonomous driving takeover, characterized in that, The method includes: When the autonomous driving system detects that the driver needs to take over, it acquires the driver's physiological attribute data and vehicle operation data; Feature extraction processing is performed on the physiological attribute data and the vehicle operation data to obtain physiological attribute features and vehicle operation features; The physiological attribute features and the vehicle operation features are fused together to obtain the fused data to be predicted. The fused data to be predicted is input into a pre-trained supervised machine learning model to perform takeover displacement prediction and obtain the takeover displacement prediction result; the takeover displacement prediction result includes vehicle displacement prediction data for a period of time after the driver takes over.

2. The autonomous driving takeover displacement prediction method as described in claim 1, characterized in that, The training methods for the supervised machine learning models include: Physiological attribute training data and corresponding vehicle operation training data of effective participants were collected during the experiment; the effective participants included subjects with different sleep durations. Analyze the vehicle displacement data of the effective participants after the driving takeover process to determine the vehicle displacement data after the takeover process. After feature extraction processing of the physiological attribute training data and vehicle operation training data respectively, they are fused together, and the vehicle displacement data after the takeover process is used as the label to construct a training sample set. Based on the training sample set, the supervised machine learning model is trained using a supervised regression method of forward propagation—loss calculation—parameter update through backpropagation, thus obtaining the pre-trained supervised machine learning model.

3. The autonomous driving takeover displacement prediction method as described in claim 2, characterized in that, The features of the physiological attribute training data include the time domain features, frequency domain features, and nonlinear features of the physiological attribute training data; The physiological attribute features and the vehicle operation features are fused together using a rectangular stitching method.

4. The autonomous driving takeover displacement prediction method as described in claim 3, characterized in that, The steps of performing feature extraction processing on the physiological attribute training data and the vehicle operation training data respectively include: Determine the mean and root mean square of the physiological attribute data; Based on the data mean, determine the data kurtosis and data skewness; Based on the mean and root mean square of the data, the waveform factor and peak factor are determined; Determine the temporal characteristics of the physiological attribute training data.

5. The autonomous driving takeover displacement prediction method as described in claim 4, characterized in that, The steps of performing feature extraction processing on the physiological attribute training data and the vehicle operation training data respectively include: The physiological attribute data is divided into multiple overlapping segments of a preset length; the overlapping segments indicate that there is partial overlap between adjacent data segments. After windowing processing of each of the aforementioned overlapping segments; Calculate the Fourier transform of each windowed overlapping segment to determine the power spectral density of each overlapping segment; The frequency domain characteristics of the physiological attribute training data are determined based on the power spectral density of each overlapping segment.

6. The autonomous driving takeover displacement prediction method as described in claim 5, characterized in that, The steps of performing feature extraction processing on the physiological attribute training data and the vehicle operation training data respectively include: The physiological attribute training data is serialized and sequentially assembled into a multidimensional vector. Calculate the approximate entropy, sample entropy, and permutation entropy values ​​of the multidimensional vector to determine the nonlinear characteristics of the physiological attribute training data.

7. An automatic driving takeover displacement prediction device, characterized in that, The device includes: The data acquisition module is used to acquire driver physiological attribute data and vehicle operation data when the autonomous driving system detects that driver takeover is required; The feature extraction module is used to perform feature extraction processing on the physiological attribute data and the vehicle operation data to obtain physiological attribute features and vehicle operation features; The feature fusion module is used to perform feature fusion processing on the physiological attribute features and the vehicle operation features to obtain the fused data to be predicted. The fused data to be predicted is input into a pre-trained supervised machine learning model to perform takeover displacement prediction and obtain the takeover displacement prediction result; the takeover displacement prediction result includes vehicle displacement prediction data for a period of time after the driver takes over.

8. A vehicle, characterized in that, The vehicle includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the autonomous driving takeover displacement prediction method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the autonomous driving takeover displacement prediction method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the autonomous driving takeover displacement prediction method as described in any one of claims 1 to 6.

Images