Automatic driving takeover control method and device, intelligent driving system and vehicle

Abstract

The invention relates to the technical field of intelligent driving, for example, relates to an automatic driving takeover control method and device, an automatic driving system, a vehicle and a readable storage medium. The control method comprises the steps that in the automatic driving process of a vehicle, under the condition that an automatic driving system is triggered to send a driving take-over request, running state data at the current moment and a historical running state data sequence in a preset time window are obtained; inputting the vehicle running state data and the historical running state data sequence into a driver takeover capability judgment model obtained through pre-training, and outputting a prediction result; if the prediction result is that the vehicle can be safely taken over, the driver takes over the vehicle; and when the prediction result is that the vehicle cannot be safely taken over, triggering a safe driving system to take over the vehicle. The driver takeover capability judgment model is adopted to predict the takeover capability, and the safe driving system is triggered to start the safe vehicle control strategy based on the prediction result that safe takeover cannot be achieved, so that the accident rate is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving, for example, to an automatic driving takeover control method and device, an automatic driving system, a vehicle, and a readable storage medium. BACKGROUND

[0002] In the case of complex or variable traffic scenarios and driving risks, the intelligent driving technology issues a takeover request for driving right, and needs to hand over the driving right to the driver for driving. In order to improve the safety of the driver taking over the vehicle, in the related technology, the image related to the driver and the environmental information of the vehicle are acquired, the takeover level is determined, and the corresponding takeover response required to be executed is determined according to the takeover level.

[0003] In the process of implementing the embodiments of the present disclosure, it is found that at least the following problems exist in the related technology: The data in the related technology only depends on the current image of the driver and the environmental information of the vehicle, and the single-time data reduces the accuracy of determining the takeover level, and is prone to misjudgment or omission. Moreover, after obtaining the driver state features and scene features, the takeover level output by the intelligent takeover prompt model can be obtained after inputting the intelligent takeover prompt model. The intelligent takeover prompt model outputs the takeover level in the following ways: directly outputting the takeover level with the maximum matching probability, or outputting the matching probability corresponding to each takeover level. For example, the matching probability of the takeover level 1 is 0, the matching probability of the takeover level 2 is 0, the matching probability of the takeover level 3 is 45%, the matching probability of the takeover level 4 is 40%, and the matching probability of the takeover level 5 is 15%. Different takeover responses are performed for different takeover levels. The takeover response corresponding to the takeover level 1 is non-prompt, the takeover response corresponding to the takeover level 2 is visual prompt, the takeover response corresponding to the takeover level 3 is visual and auditory prompt, and the takeover responses corresponding to the takeover level 4 and the takeover level 5 are active takeover.

[0004] It can be seen that, in the related technology, the takeover level is divided into multiple levels by the fixed probability matching rule when the vehicle issues a takeover request, which results in low accuracy of the takeover level. Moreover, the takeover level is divided into multiple levels with different matching probabilities, which results in the need for the system to process multiple takeover responses of the levels, and the takeover responses mainly include prompt and active takeover of the vehicle system. The prompt itself has uncertainty, which results in low success probability of the takeover result and high safety risk. The active takeover still adopts the automatic driving of the vehicle system, that is, the risk of the automatic driving does not change before and after the takeover determination, which results in low driving safety of the vehicle. As can be seen, the takeover control in the related technology has poor accuracy in the determination of the takeover ability of the driver and the prediction of the takeover result, and has great safety hazards in the takeover control. SUMMARY

[0005] The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosed embodiments. The summary is not an extensive overview of the disclosure, and is not intended to identify key / critical elements or to delineate the scope of the embodiments. Its sole purpose is to present some concepts of the embodiments in a simplified form as a prelude to the more detailed description that is presented later.

[0006] The embodiments of the present disclosure provide an automatic driving takeover control method and device, an automatic driving system, a vehicle and a readable storage medium, so as to improve the accuracy of takeover ability determination, and further improve the driving safety.

[0007] In some embodiments, an automatic driving takeover control method is provided, including: in a case where a vehicle triggers an automatic driving system to issue a driving takeover request during automatic driving, obtaining running state data at a current time and a historical running state data sequence within a preset time window; inputting the vehicle running state data and the historical running state data sequence into a pre-trained driver takeover ability judgment model, the driver takeover ability judgment model being configured to map a prediction result for representing a driver's takeover ability based on the input data, the prediction result including being able to safely take over or being unable to safely take over; in a case where the prediction result is being able to safely take over, taking over the vehicle by the driver, the being able to safely take over indicating that the driver can successfully take over the vehicle and keep the vehicle in a safe driving state; and in a case where the prediction result is being unable to safely take over, triggering a safe driving system to take over the vehicle, the being unable to safely take over indicating that the driver cannot take over the vehicle or the vehicle will be in an abnormal state after the driver takes over the vehicle.

[0008] The automatic driving takeover control method provided by the present disclosure obtains running state data at a current time and a historical running state data sequence within a preset time window at the current time when a takeover request is triggered. The running state data at the current time and the historical running state data sequence are input into a pre-trained driver takeover ability judgment model. When the prediction result output by the driver takeover ability judgment model is being able to safely take over, the vehicle control right is given to the driver. When the prediction result output by the driver takeover ability judgment model is being unable to safely take over, the safe driving system is activated immediately, and the vehicle is taken over by the safe driving system.

[0009] Thus, the control method provided by the present disclosure simultaneously acquires the running state data at the current time and the historical running state data sequence within the preset time window before the current time at the current time when the takeover request is triggered. That is, the present application simultaneously obtains the running state data in the continuous time period according to the current running state data and the historical running state data sequence before the current time, and the running state data in the continuous time period can accurately reflect the change trend of the running state of the vehicle and the change trend of the state of the driver. Further, the current running state data and the historical running state data sequence before the current time are synchronously input into the pre-trained driver takeover ability judgment model to predict the takeover ability of the driver. That is, compared with the related art in which the state of the driver and the state of the vehicle are determined for takeover by using single-time data, the present disclosure can greatly improve the accuracy and reliability of the prediction of the takeover ability of the driver by combining the running state data in the continuous time period and the pre-trained driver takeover ability judgment model, and reduce the risk of misjudgment or omission of the state of the driver or the state of the vehicle by single data.

[0010] Further, in the scheme adopted by the present application, the current running state data and the historical running state data sequence before the current time are synchronously input into the pre-trained driver takeover ability judgment model, and the prediction result output includes two kinds, one is that the driver can safely take over, and the other is that the driver cannot safely take over. That is, the scheme provided by the present application is a binary classification prediction model with takeover success or failure as the output, which directly gives an explicit determination result, and according to the two results, corresponding processing logic is given, and a control closed loop from triggering a request, result prediction to execution operation is realized, and the safety of driving control is improved. Among them, for the prediction result that the driver can safely take over, the driving right is given to the driver, that is, according to the input data, it is predicted that the driver can take over the vehicle, and after taking over the vehicle, the vehicle can be safely driven. For the prediction result that the driver cannot safely take over, that is, according to the input data, it is predicted that the driver cannot take over the vehicle, or even if the driver takes over the vehicle, it is predicted that the driver will take over the vehicle according to the state of the driver or the running state of the vehicle, and the probability of vehicle abnormality after the driver takes over the vehicle is high, then the safe driving system is immediately activated, and the vehicle is taken over by the safe driving system. In this way, in the case that the driver cannot take over, the safe driving system is used to intervene in the vehicle control in advance, fully utilizes the time before the accident and collision to control the vehicle, leaves more sufficient time for active collision avoidance control, and reduces the accident rate.

[0011] Compared with the prior art, the prediction result is multiple takeover levels, and the prompt or self-driving system takeover is performed for different takeover levels. The prediction result output by the pre-trained driver takeover ability judgment model is a binary classification prediction result of being able to take over or not being able to take over. Being able to take over means that the driver can take over and can safely drive after taking over the vehicle, and not being able to take over means that the driver cannot take over or the vehicle will have abnormal conditions such as collision or rollover after taking over. And for the case of not being able to take over, the safety driving system is started to take over the vehicle. In this way, the accuracy of the prediction result is improved by using the automatic takeover control method provided by the present disclosure, and the reliability and stability of the vehicle takeover driving safety are improved.

[0012] Optionally, the running state data includes vehicle motion state data, environment data, and driver state data; wherein the vehicle motion state data includes vehicle speed, acceleration, and body angle, angular velocity, angular acceleration, and tire pressure; the environment data includes weather, light intensity, road information, and relative distance to surrounding objects; and the driver state data includes one or more of the following: face direction, eye state, mouth opening and closing frequency and degree.

[0013] In this embodiment, the vehicle motion state is accurately determined to be within the safe boundary range by the longitudinal, lateral and vertical speed, acceleration, body roll, pitch and yaw angle, angular velocity and angular acceleration; the maximum braking force and braking distance of the vehicle are determined by the tire pressure, weather conditions such as rainfall in rainy weather, road curvature, slope information, water accumulation, and icing conditions; the vehicle handling stability is determined by the tire pressure and road potholes, bumps, water accumulation, and icing; the sensor distance measurement accuracy for surrounding objects is determined by the fog visibility and light intensity; the driver distraction level and fatigue driving level are determined by the driver face direction, eye state such as opening degree, blinking frequency, and mouth opening and closing frequency and degree. Optionally, the historical running state data sequence is obtained, including: obtaining the running state data of the vehicle at multiple continuous sampling time points within a preset time window before the current time; and sorting the running state data at the multiple continuous sampling time points according to time to form the historical running state data sequence.

[0014] In this embodiment, considering that the data at a single moment may have noise or contingency, which in turn affects the determination result. The application obtains the running state data of the vehicle at multiple continuous sampling moments within a preset time window before the current moment, and forms a historical running state data sequence. By introducing the historical running state data sequence, instantaneous noise and accidental interference can be effectively filtered out, making the evaluation result more stable and reliable, and not easily affected by instantaneous abnormal interference. In addition, it can capture the trend of state change, which provides the possibility for forward-looking safety decision, and greatly improves the robustness and reliability of the driver takeover ability evaluation.

[0015] Optionally, the step of constructing the driver takeover ability judgment model comprises: simulating a plurality of automatic driving scenarios by the driver under a plurality of test working conditions; obtaining the running state data at the current moment when the takeover request is triggered during each test process; obtaining the historical running state data sequence at the multiple continuous sampling moments before the current moment; obtaining the real takeover result in response to the takeover request; arranging the data obtained by multiple tests into a data pair comprising the running state data, the historical running state data sequence and the actual takeover result, to form an original data set; constructing an initial model for driver takeover ability judgment, the initial model comprising an input layer and an output layer, the input layer being used for inputting the running state data and the historical running state data sequence, and the output layer being used for outputting a predicted takeover result, the predicted takeover result comprising that the driver can safely take over or the driver cannot safely take over; training and testing the initial model using the original data set to obtain the driver takeover ability judgment model.

[0016] In this embodiment, a plurality of test scenarios under a plurality of test working conditions are constructed, and a plurality of drivers are used for simulation test. The running state data at the current moment when the takeover request is triggered during the test process and the historical running state data sequence at the multiple continuous sampling moments before the current moment are obtained; and the real takeover result corresponding to the takeover request is obtained, to obtain an original data set. The initial model is trained and tested using the original data set to obtain the driver takeover ability judgment model. By covering different driving environments, different takeover trigger reasons and different types of drivers, the diversity and representativeness of the data set are ensured. The original data set is used to train and verify the initial model, which improves the generalization ability of the model, and thus the model can accurately judge in various complex scenarios, reduces the false positive rate and the false negative rate of the takeover result prediction. Moreover, the overfitting problem of the model caused by single training data is avoided, and the robustness of the model in complex and variable actual applications is ensured.

[0017] Optionally, the real takeover result responsive to the takeover request is obtained, including: responsive to the takeover request, if the driver successfully takes over the vehicle and keeps the vehicle in a safe driving state, marking the real takeover result as capable of safe takeover; if the driver does not take over the vehicle or the vehicle is in a collision, lane deviation or instability situation after the driver takes over the vehicle, marking the real takeover result as incapable of safe takeover.

[0018] In this embodiment, by giving a clear label definition to the real takeover result in the model training process, the model can clearly understand which input data corresponds to a successful and safe takeover and which data corresponds to a failed and dangerous takeover during training. This enables the model to learn deeper and more essential feature associations, rather than just superficial behavior associations. By defining the decision boundary, the model can clearly divide the safe and dangerous decision boundary in the feature space, which is crucial for the model to make high-confidence judgments in actual applications.

[0019] Optionally, the initial model is trained and tested using the original data set to obtain the driver takeover ability judgment model, including: dividing the original data set into a training set, a validation set and a test set; using the training set to iteratively optimize the model parameters of the initial model; using the validation set to monitor model performance and control overfitting during training; using the test set to evaluate the performance of the trained model, and locking the model parameters to obtain the driver takeover ability judgment model under the condition that the predetermined performance conditions are met.

[0020] In this embodiment, the original data set is divided into a training set, a validation set and a test set; the model parameters of the initial model are iteratively optimized using the training set to improve the generalization ability of the model. The model performance is monitored using the validation set during training to assist in hyperparameter tuning and early stopping to prevent overfitting. When the model achieves satisfactory performance on both the training set and the validation set, the test set that has never participated in training and tuning is used to perform final evaluation on the model. If the performance on the test set meets the predetermined indicators, such as accuracy and recall, the model parameters at this time are locked to obtain the final driver takeover ability judgment model.

[0021] Optionally, the model parameters of the initial model are iteratively optimized using the training set, including: using the training set, using the back propagation algorithm and the optimization algorithm, and using the mean square error between the predicted takeover result of the model and the real takeover result as the loss function to train the initial model.

[0022] In this embodiment, the loss function sets a clear and safe-oriented optimization goal for the model; the back propagation algorithm solves the calculation problem of parameter optimization in high-dimensional space; the optimization algorithm realizes an efficient and automated optimization process; and the combination of the three improves the accuracy and confidence of the probability prediction output by the model.

[0023] Optionally, the formula of the initial model is as follows: y = f(s0, s1, s2…s n ); wherein, y is the takeover result in response to the takeover request; f is a mapping function or a neural network; s0 is the running state data at the current time; s1 is the historical running state data at the first sampling point before the takeover request is sent; s n is the historical running state data at the n-th sampling point before the takeover request is sent.

[0024] Optionally, triggering the safe driving system to take over the vehicle comprises: based on a model predictive control algorithm, combined with constraint conditions, solving the acceleration control instruction and the steering angle control instruction, the constraint conditions including: the safe limit value of the vehicle longitudinal acceleration and the lateral acceleration, the safe distance constraint from the surrounding obstacles, and the vehicle dynamics model constraint; and controlling the vehicle to run according to the acceleration control instruction and the steering angle control instruction.

[0025] In this embodiment, by using the model predictive control algorithm, the control instruction solved under the safe distance constraint can conform to the physical characteristics of the vehicle, avoiding secondary accidents such as vehicle sideslip and spin due to excessive steering or braking in an emergency, and ensuring the safety of the control process itself. Moreover, the acceleration and steering instructions generated by the optimization objective function of the model predictive control algorithm are continuous and smooth, rather than step or impact, and the smooth intervention mode reduces the tension and discomfort of the passengers. Furthermore, by adding constraint conditions, the adaptability to different driving scenarios can be improved.

[0026] Optionally, triggering the driving takeover request includes at least one of the following conditions: the vehicle is about to or has exited the design operating domain of its autonomous driving system; the perception system of the vehicle detects that its performance attenuation or information conflict exceeds a safety threshold; the predicted collision time based on the current state of the vehicle is below a safety threshold.

[0027] In some embodiments, an automatic driving takeover control device is provided, comprising: a data acquisition module configured to, when a driving takeover request is triggered during the autonomous driving of the vehicle, acquire the running state data at the current time and the historical running state data sequence within a preset time window; a takeover judgment module configured to input the running state data of the vehicle and the historical running state data sequence into a pre-trained driver takeover capability judgment model, the driver takeover capability judgment model being configured to map the input data to a prediction result for characterizing the driver takeover capability, the prediction result including being able to safely take over or being unable to safely take over; a safety control module configured to, if the prediction result is that the vehicle can be safely taken over, take over the vehicle by the driver, the vehicle can be safely taken over indicating that the driver can successfully take over the vehicle and keep the vehicle in a safe driving state, and configured to, if the prediction result is that the vehicle cannot be safely taken over, trigger the safety driving system to take over the vehicle, the vehicle cannot be safely taken over indicating that the driver cannot take over the vehicle or that, after the driver takes over the vehicle, the vehicle will be in an abnormal state.

[0028] In some embodiments, an automatic driving takeover control apparatus is provided, comprising a processor and a memory storing program instructions, the processor being configured to execute, when running the program instructions, the automatic driving takeover control method according to any one of the above embodiments.

[0029] In some embodiments, an intelligent driving system is provided, comprising: a safety driving system comprising the automatic driving takeover control apparatus according to any one of the above embodiments, and the safety driving system being configured to control the vehicle to run if the takeover judgment result is that the vehicle cannot be safely taken over; and an automatic driving system configured to control the vehicle to automatically drive and send a driving takeover request to the control apparatus.

[0030] In some embodiments, a vehicle is provided, comprising: a vehicle body; the automatic driving takeover control apparatus according to any one of the above embodiments, or the intelligent driving system according to any one of the above embodiments, mounted on the vehicle body.

[0031] In some embodiments, a readable storage medium is provided, storing program instructions for causing a computer to execute the automatic driving takeover control method according to any one of the above embodiments when running.

[0032] The automatic driving takeover control method and apparatus, the automatic driving system, the vehicle, and the readable storage medium provided by the embodiments of the present disclosure can achieve the following technical effects: (1) The automatic driving takeover control method provided by the present disclosure collects, at the moment when the automatic driving system sends a takeover request, data including running state data at the current moment and a historical running state data sequence within a preset time window before the current moment, and uses the running state data at the current moment and the historical running state data sequence within the preset time window before the current moment as a judgment basis for driving safety takeover capability. This multi-modal data type and continuous time-series running data improve the judgment of the driver's takeover capability, avoid the false triggering of the safety driving system when the driver can safely take over, avoid the missed triggering of the safety driving system when the driver cannot safely take over, and greatly improve the judgment accuracy for complex and dynamic takeover scenarios.

[0033] (2) The automatic driving takeover control method provided by the present disclosure adopts a driver takeover capability judgment model constructed based on a neural network or a mapping function. The model takes the running state data sequence in a preset time window before the time when the takeover request is sent as the model input, and can thus capture the evolution trend of the running data. The model output is an explicit safe takeover and non-safe takeover, which improves the executability of the prediction result. During the model training and testing process, the actual takeover success or failure result is used as a supervision signal, and the model directly learns the safe takeover and non-safe takeover mode boundary through supervised learning, thereby improving the accuracy of the takeover capability prediction.

[0034] (3) The automatic driving takeover control method provided by the present disclosure makes a hierarchical decision based on the prediction result output by the driver takeover capability judgment model. For the prediction result of safe takeover, the safe driving system does not intervene, the driver takes over, and the vehicle control is performed; for the prediction result of non-safe takeover, the safe driving system triggers the safe control vehicle strategy to directly perform the vehicle control. Moreover, during the vehicle control process of the safe driving system, the MPC (Model Predictive Control) algorithm is used to fully utilize the driving space, obtain the optimal vehicle safe operation control strategy through braking and steering combination, fully utilize the road drivable space compared with the braking collision avoidance, fully consider the safety of the nearby lane traffic participants compared with the steering collision avoidance, avoid collision with the objects in the original lane while reducing or avoiding the collision risk with other traffic participants in other lanes, thereby fully reducing the vehicle accident rate and improving the safety during vehicle operation.

[0035] (4) The automatic driving takeover control method provided by the present disclosure triggers the safe driving system to start the safe control vehicle strategy to directly perform the vehicle control for the prediction result of non-safe takeover. Compared with the active collision avoidance control method in the automatic driving system, the vehicle control by the safe driving system can trigger the obstacle avoidance control in advance, fully utilize the time before the potential accident occurs through precise and rapid safe control of the vehicle, and thus can reduce the accident rate.

[0036] The foregoing general description and the following description are merely exemplary and explanatory, and are not intended to limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0037] One or more embodiments are illustrated by way of example with reference to the accompanying drawings, which are schematic and are not intended to be limiting of the embodiments, in which like reference numerals designate similar items in the figures, and wherein: Figure 1is a schematic diagram of an automatic driving takeover control method provided by an embodiment of the present disclosure; Figure 2 is a driving scene diagram example provided by an embodiment of the present disclosure; Figure 3 is a schematic diagram of a method for constructing a driver takeover capability judgment model provided by an embodiment of the present disclosure; Figure 4 is an initialization model schematic diagram of an embodiment of the present disclosure; Figure 5 is a high-speed scene emergency collision avoidance working condition application schematic diagram of an embodiment of the present disclosure; Figure 6 is Figure 5 is a schematic diagram of a vehicle operating mode in the embodiment shown; Figure 7 is Figure 5 is a schematic diagram of the longitudinal speed and steering wheel angle during vehicle driving in the embodiment shown; Figure 8 is Figure 5 is a schematic diagram of the driving path of the vehicle in the embodiment shown; Figure 9 is a schematic diagram of an automatic driving takeover control device provided by an embodiment of the present disclosure; Figure 10 is a schematic diagram of an intelligent driving system provided by an embodiment of the present disclosure; Figure 11 is a schematic diagram of another automatic driving takeover control device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] In order to be able to understand the features and technical contents of the embodiments of the present disclosure more fully, the implementation of the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings, which are only used for reference and do not limit the embodiments of the present disclosure. In the following technical description, in order to facilitate explanation, a plurality of details are provided to provide a full understanding of the disclosed embodiments. However, one or more embodiments can still be implemented without these details. In other cases, well-known structures and devices can be simplified to show.

[0039] The terms "first", "second", and the like in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion.

[0040] The term "plurality" means two or more, unless otherwise specified.

[0041] In the embodiments of the present disclosure, the character " / " represents that the objects before and after the " / " are in an "or" relationship. For example, A / B means A or B.

[0042] The term "and / or" is a description of the association relationship of the objects, which means that there can be three relationships. For example, A and / or B means that there are three relationships of A or B, or A and B.

[0043] The term "corresponding" can refer to an association relationship or a binding relationship. A and B correspond to each other means that A and B are in an association relationship or a binding relationship.

[0044] In some embodiments, a control device for automatic driving takeover is provided, comprising a processor and a memory storing program instructions, the processor being configured to execute the automatic driving takeover control method as described in any of the embodiments below when running the program instructions.

[0045] In some embodiments, in combination with Figure 1 As shown, a control method for automatic driving takeover is provided, comprising: S101, in the process of automatic driving, when the automatic driving system triggers a driving takeover request, acquiring running state data at the current time and a historical running state data sequence within a preset time window.

[0046] Optionally, the running state data includes vehicle motion state data and environment perception data. The vehicle motion state data includes but is not limited to vehicle driving speed, acceleration, and vehicle body angle, angular velocity, angular acceleration, and tire pressure.

[0047] Exemplarily, the vehicle running state data includes but is not limited to vehicle longitudinal speed, vehicle lateral speed, vehicle vertical speed, vehicle longitudinal acceleration, vehicle lateral acceleration, vehicle vertical motion acceleration, vehicle body roll angle, vehicle body pitch angle, vehicle body yaw angle, vehicle yaw angular velocity, and tire pressure. The vehicle motion state is determined whether it is within the safe boundary range by the vehicle running state data.

[0048] For example, environmental perception data includes, but is not limited to, the relative state of the nearest obstacle to the vehicle within the vehicle's perception range, as well as weather, light intensity, and road information. Environmental perception data includes, but is not limited to: longitudinal relative distance to the vehicle, lateral relative distance to the vehicle, longitudinal relative speed to the vehicle, lateral relative speed to the vehicle, longitudinal relative acceleration to the vehicle, weather, light intensity, and road information. If there are no obstacles in a certain direction, preset values ​​are used. The vehicle's maximum braking force and braking distance are determined by tire pressure, weather conditions such as rainfall in rainy weather, road curvature, slope information, water accumulation, and icing conditions; the vehicle's handling stability is determined by tire pressure and road surface features such as potholes, bumps, water accumulation, and icing; and the accuracy of the sensor's distance measurement of surrounding objects is determined by visibility in foggy weather and light intensity.

[0049] For example, in combination Figure 2 The driving scenario diagram shown is as follows: Vehicle 0 is the autonomous vehicle under test, denoted as 200; the left boundary line of the lane where Vehicle 200 is located is denoted as 21; the right boundary line of the lane where Vehicle 200 is located is denoted as 22; the left boundary line of the lane to the left of Vehicle 200 is denoted as 20 (if there is no lane to the left of Vehicle 200, boundary line 20 does not exist); the right boundary line of the lane to the right of Vehicle 200 is denoted as 23 (if there is no lane to the right of Vehicle 200, boundary line 23 does not exist); the objects in front of Vehicle 200 to the left are denoted as 201, objects in front are denoted as 202, objects in front to the right are denoted as 203, objects in the right are denoted as 204, objects in the right rear are denoted as 205, objects in the rear are denoted as 206, objects in the left rear are denoted as 207, and objects in the left are denoted as 208.

[0050] Record the motion state v of the vehicle, i.e., vehicle 200. x00 v is the longitudinal speed of the vehicle moving 200. y00 The lateral movement speed of the vehicle is 200. The vertical speed of the vehicle is 200. Let a be the longitudinal acceleration of the vehicle moving 200. y00 The lateral acceleration of the vehicle is 200. The vertical acceleration of the vehicle at 200 degrees is... For a vehicle with a 200° roll angle, θ y00 For the vehicle's 200 pitch angle, For a 200° yaw angle of the bicycle, The 200° roll rate of the vehicle. For the vehicle's 200° pitch rate, For the yaw rate of the vehicle at 200 degrees, The acceleration due to a 200° roll angle of the vehicle. The acceleration of the vehicle at a pitch angle of 200 degrees. The acceleration is the yaw rate of the vehicle at a 200° angle. is the tire pressure of the left front tire, is the tire pressure of the right front tire, is the tire pressure of the left rear tire, is the tire pressure of the right rear tire. In addition, let: is the longitudinal relative distance between the ego vehicle 200 and the object 201, is the lateral relative distance between the ego vehicle 200 and the object 201, is the longitudinal relative velocity between the ego vehicle 200 and the object 201, is the lateral relative velocity between the ego vehicle 200 and the object 201, is the longitudinal relative acceleration between the ego vehicle 200 and the object 201, is the lateral relative acceleration between the ego vehicle 200 and the object 201; is the longitudinal relative distance between the ego vehicle 200 and the object 202, is the lateral relative distance between the ego vehicle 200 and the object 202, is the longitudinal relative velocity between the ego vehicle 200 and the object 202, is the lateral relative velocity between the ego vehicle 200 and the object 202, is the longitudinal relative acceleration between the ego vehicle 200 and the object 202, is the lateral relative acceleration between the object 200 and the object 202; is the longitudinal relative distance between the ego vehicle 200 and the object 203, is the lateral relative distance between the ego vehicle 200 and the object 203, is the longitudinal relative velocity between the ego vehicle 200 and the object 203, is the lateral relative velocity between the ego vehicle 200 and the object 203, is the longitudinal relative acceleration between the ego vehicle 200 and the object 203, is the lateral relative acceleration between the ego vehicle 200 and the object 203; is the longitudinal relative distance between the ego vehicle 200 and the object 204, is the lateral relative distance between the ego vehicle 200 and the object 204, is the longitudinal relative velocity between the ego vehicle 200 and the object 204, is the lateral relative velocity between the ego vehicle 200 and the object 204, is the longitudinal relative acceleration between the ego vehicle 200 and the object 204, is the lateral relative acceleration between the ego vehicle 200 and the object 204; is the longitudinal relative distance between the ego vehicle 200 and the object 205, is the lateral relative distance between the ego vehicle 200 and the object 205, is the longitudinal relative velocity between the ego vehicle 200 and the object 205, is a lateral relative velocity of the ego vehicle 200 and the object 205, is a longitudinal relative acceleration of the ego vehicle 200 and the object 205, is a lateral relative acceleration of the ego vehicle 200 and the object 205; is a longitudinal relative distance of the ego vehicle 200 and the object 206, is a lateral relative distance of the ego vehicle 200 and the object 206, is a longitudinal relative velocity of the ego vehicle 200 and the object 206, is a lateral relative velocity of the ego vehicle 200 and the object 206, is a longitudinal relative acceleration of the ego vehicle 200 and the object 206, is a lateral relative acceleration of the ego vehicle 200 and the object 206; is a longitudinal relative distance of the ego vehicle 200 and the object 207, is a lateral relative distance of the ego vehicle 200 and the object 207, is a longitudinal relative velocity of the ego vehicle 200 and the object 207, is a lateral relative velocity of the ego vehicle 200 and the object 207, is a longitudinal relative acceleration of the ego vehicle 200 and the object 207, is a lateral relative acceleration of the ego vehicle 200 and the object 207; is a longitudinal relative distance of the ego vehicle 200 and the object 208, is a lateral relative distance of the ego vehicle 200 and the object 208, is a longitudinal relative velocity of the ego vehicle 200 and the object 208, is a lateral relative velocity of the ego vehicle 200 and the object 208, is a longitudinal relative acceleration of the ego vehicle 200 and the object 208, is a lateral relative acceleration of the ego vehicle 200 and the object 208. is a picture of a vehicle driving environment acquired by a camera, 、 is road slope, road curvature information acquired by a vehicle-mounted map, is a picture of a driver's face acquired by a cockpit camera.

[0051] Thus, the vehicle motion state data s of the ego vehicle 200 at any time is:

[0052] Optionally, the running state data includes driver state data; the driver state data includes one or more of the following: face direction, eye state, mouth opening and closing frequency and opening and closing degree.

[0053] Optionally, the face direction is identified by a camera inside the cockpit, and the angle between the driver's face and the front direction is output. The eye state is calculated by an eye recognition algorithm, for example, PERCLOS (Percentage of Eyelid Closure over the Pupil over Time). PERCLOS refers to the percentage of time that the eyelid covers the pupil within a certain time period. The calculation formula is: PERCLOS = eyelid closure time / total observation time x 100%, which evaluates the degree of fatigue by calculating the percentage of the total observation time that the driver's eyes are closed. The mouth opening and closing frequency and degree are collected by a camera inside the vehicle cockpit. Face detection and facial feature point positioning are performed on each frame of image to obtain the key point coordinates of the mouth region, including the middle points of the upper and lower lips and the left and right corner points. Based on the mouth key point coordinates, the mouth aspect ratio (MAR) value is obtained, which is used as a real-time quantitative indicator of the degree of mouth opening. The larger the MAR value, the greater the degree of mouth opening. The MAR value is compared with the preset value to determine the degree of opening. The mouth opening and closing frequency is determined by setting a MAR threshold value to determine whether the mouth is open. When the MAR value rises from below the threshold value to above the threshold value and then falls below the threshold value again, it is counted as one complete mouth opening and closing action. The completion time of the action is recorded. In a sliding time window, for example, 60 seconds, the total number of completed opening and closing actions is counted, and the quotient of the total number of opening and closing actions and the time window is the mouth opening and closing frequency.

[0054] Optionally, the historical running state data sequence is obtained by: obtaining the running state data of the vehicle at a plurality of continuous sampling time points within a preset time window before the current time; and sorting the running state data at the plurality of continuous sampling time points according to time to form the historical running state data sequence.

[0055] In this embodiment, when the automatic driving system issues a takeover request, the running state data at the current time is collected in real time, i.e., the vehicle running parameters, environmental perception parameters and driver state at the current time are collected. In addition, the vehicle running parameters, environmental perception parameters and driver state at n sampling time points within a preset time window before the issuance of the takeover request are obtained. The preset time window can be customized according to the vehicle performance parameters. By introducing the historical running state data sequence, transient noise and accidental interference can be effectively filtered out, making the evaluation result more stable and reliable, and less susceptible to transient abnormal interference. In addition, it can capture the trend of state changes, providing the possibility for forward-looking safety decisions, and greatly improving the robustness and reliability of the evaluation of the driver's takeover ability.

[0056] Optionally, the triggering of the driving takeover request comprises at least one of the following conditions: the vehicle is about to or has exited a designed operating domain of the automatic driving system; the perception system of the vehicle detects that its performance degradation or information conflict exceeds a safety threshold; a predicted collision time based on the current state of the vehicle is below a safety threshold.

[0057] In this example, the designed operating domain refers to a specific condition range designed for the automatic driving system to work properly, including road types, such as limited to highways, geographical areas, such as cities, speed ranges, weather conditions, such as sunny days, daytime, etc. For example, the vehicle is automatically driving on a highway, and the navigation prompts that the highway section ends 2 kilometers ahead. The system issues a takeover request at 1.5 kilometers from the exit, reminding the driver to prepare to take over.

[0058] The perception system of the vehicle includes, but is not limited to, sensor devices such as cameras, radars, lidars, etc. Performance degradation refers to the performance degradation of sensors due to reasons such as dirt, bad weather (e.g. heavy rain, heavy fog), etc. Information conflict refers to the inconsistency of the perception results of different sensors on the same object, and the system cannot make a reliable judgment. By setting a self-checking program for the perception system. When the system detects that a key sensor is disabled (such as a camera being blocked), or the perception result calculated by the fusion algorithm has a confidence level below a safety threshold, or the radar feedbacks that there is an obstacle in front, while the camera feedbacks that there is no obstacle in front and cannot be arbitrated, the system cannot drive safely and must request human intervention. For example: heavy rain causes the front camera to have a blurred view, and the system detects that the camera confidence level has dropped sharply, and immediately issues a takeover request.

[0059] The predicted collision time based on the current state of the vehicle being below a safety threshold means that the system calculates the collision time based on the relative speed and distance between the vehicle and the obstacle in front; the collision time is below a preset safety threshold, which is greater than the triggering threshold of the emergency braking system, to reserve reaction time for the driver's operation. In this case, it means that even if the automatic driving system operates normally, the risk is already too high, and the driver needs to intervene or prepare to respond to the possible collision.

[0060] S102, input the vehicle operating state data and the historical operating state data sequence into a pre-trained driver takeover capability judgment model, the driver takeover capability judgment model is configured to map the prediction result for characterizing the driver takeover capability based on the input data.

[0061] The current running state data and the historical running state data sequence before the current time are synchronously input into the pre-trained driver takeover ability judgment model to predict the takeover ability of the driver. That is, by combining the running state data in the continuous time period and the pre-trained driver takeover ability judgment model, the accuracy and reliability of the prediction of the takeover ability of the driver can be greatly improved.

[0062] S103, if the prediction result is that the takeover can be safely performed, the vehicle is taken over by the driver.

[0063] S104, if the prediction result is that the takeover cannot be safely performed, the vehicle is taken over by the safety driving system.

[0064] The safe takeover refers to that the driver can successfully take over the vehicle and keep the vehicle in a safe driving state, and the unsafe takeover refers to that the driver cannot take over the vehicle or the vehicle will be in an abnormal state after the driver takes over the vehicle.

[0065] Optionally, the taking over of the vehicle by the safety driving system comprises: obtaining an acceleration control instruction and a steering angle control instruction based on a model predictive control algorithm and in combination with constraint conditions, the constraint conditions comprising: a safe limit value of a vehicle longitudinal acceleration and a vehicle lateral acceleration, a safe distance constraint from surrounding obstacles, and a vehicle dynamics model constraint; and controlling the vehicle to operate according to the acceleration control instruction and the steering angle control instruction.

[0066] In this embodiment, if the prediction result is that the takeover can be safely performed, the vehicle is taken over by the driver, and the safety driving system does not control the vehicle, that is, the control instruction of the safety driving system is , as shown in formula (1). If the prediction result is that the takeover cannot be safely performed, the vehicle is taken over by the safety driving system, and the safety driving system controls the vehicle, that is, the control instruction of the safety driving system is as shown in formula (2), and the safety driving system exits the control of the vehicle if the vehicle exits the risk and enters a safe state or enters an automatic driving state or a manual driving state.

[0067] Formula (1) In formula (1), u c is an acceleration control instruction of the vehicle at the next time, is a tire steering angle control instruction of the vehicle at the next time, is a safe takeover state of the vehicle by the driver, and the safety driving system does not control the vehicle.

[0068] Formula (2) In formula (2), u cu (1) is the acceleration control instruction of the vehicle at the next moment c= u (1) is obtained by formula (3), u (1) is the tire angle control instruction of the vehicle at the next moment u (1) is obtained by formula (3), The safe driving system controls the vehicle when the driver cannot safely take over the vehicle state.

[0069]

[0070] Formula (3) In formula (3), Δt is the sampling time interval, u(t) is the acceleration control instruction at the future tth sampling time, tn is the number of sampling at the future time point, u(0) is the current acceleration, u(1) is the acceleration control instruction at the next sampling time, φ(t) is the tire angle control instruction at the future tth sampling time, φ(0) is the current tire angle, φ(1) is the tire angle at the next sampling time, v x v is the longitudinal driving speed of the vehicle, v y is the lateral driving speed of the vehicle, θ(t) is the driving heading angle of the vehicle, θ(t) is the heading angle speed, m c m is the mass of the vehicle, I c I is the moment of inertia of the vehicle, k f k is the front axle tire cornering stiffness, k r k is the rear axle tire cornering stiffness, l f l is the distance between the front axle and the center of mass, l r l is the distance between the rear axle and the center of mass, μ is the ground adhesion coefficient, k u k is the acceleration control instruction weighting coefficient, k φ k is the tire angle control instruction weighting coefficient, a xminsafe a is the lower limit of the longitudinal acceleration value, a xmaxsafe a is the upper limit of the longitudinal acceleration value, a yminsafe a is the lower limit of the lateral acceleration value, a ymaxsafe a is the upper limit of the lateral acceleration value, c1 and c2 are constants greater than zero, objn is the number of surrounding obstacles, d j safe d is the safety distance between the ego vehicle and the jth obstacle.

[0071] Formula (3) integrates the constraint condition into the objective function J by Lagrange multiplier to form a new Lagrange function, simplifies the optimization process of the optimization objective in formula (3), and realizes the optimal numerical solution of the new Lagrange function by combining the pseudo-spectral method, to calculate the acceleration control instruction u c and the steering wheel angle control instruction φ c, to minimize the collision risk with obstacles while guaranteeing smooth control under the premise of satisfying vehicle dynamics constraints and road constraints.

[0072] The present disclosure adopts a model predictive control (MPC) framework to convert the control optimization problem into a quadratic programming (QP) problem: minimizing a quadratic objective function while satisfying a series of linear constraints.

[0073] In formula (3), the is a control amount penalty term, which aims to prevent the acceleration and steering angle from being too large, to ensure smoothness, and to avoid sudden acceleration, sudden braking, or sharp steering. is a control increment penalty term, which aims to control the smoothness of the control command at the adjacent time, to avoid shaking, and to improve comfort.

[0074] In formula (3), the formula in the curly braces is a constraint condition, where the first five formulas v x (t), v y (t), v x (t) are the state prediction equations derived from vehicle dynamics, which are used to describe the state of the vehicle at future time. y (t) is the predicted longitudinal velocity at the next time, v c (t) is the predicted lateral velocity at the next time, c (t) is the predicted heading angle velocity at the next time, and X(t) and Y(t) are the predicted positions of the vehicle in the system coordinate system at the next time. Through these formulas, the trajectory caused by the acceleration control command u j and the steering wheel angle control command φ safe can be predicted.

[0075] In formula (3), the four inequalities below the first five formulas are vehicle physical limit constraints, which are longitudinal acceleration limit, lateral acceleration limit, and combined acceleration limit, respectively. The longitudinal acceleration limit is to avoid excessive acceleration or deceleration, which exceeds the passenger comfort or vehicle performance limit. The lateral acceleration limit is to avoid excessive lateral force when cornering, which causes the vehicle to lose stability. The combined acceleration limit is to ensure that the total force between the vehicle tire and the ground does not exceed the maximum adhesion force (μ·g), to prevent the vehicle from skidding.

[0076] In formula (3), the next two inequalities and are constraints on the rate of change of the control command. The purpose is that the difference between the acceleration command at the next time and the current acceleration cannot be too large, to ensure smooth response of the actuator. The role of the next moment is that the difference between the steering angle command and the current steering angle cannot be too large. Through the two constraints, the vehicle rollover is prevented.

[0077] In formula (3), the role of the last formula is the safety distance constraint. The predicted future trajectory of the vehicle must maintain at least d j safe safe distance with the predicted future trajectory of the jthobstacle to achieve active collision avoidance.

[0078] The acceleration control command u c and the steering wheel angle control command φ c are sent to the wire control execution system through the vehicle network, and the wire control execution system includes a steer-by-wire system, a wire drive system, and a wire brake system, to realize active safety control of the vehicle until the vehicle exits the dangerous state, and returns the control right of the vehicle to the driver or restores the automatic driving. The vehicle exits the dangerous state includes successfully changing lanes to avoid obstacles, and stably decelerating to stop.

[0079] In some embodiments, as shown in Figure 3 , the step of constructing the driver takeover ability judgment model includes: S301, simulate a plurality of automatic driving scenes by the driver under a plurality of test working conditions; Optionally, various driving scenes, especially emergency and long tail scenes, are covered through in-loop simulation testing, driver-in-loop bench testing, or driver real vehicle site testing, real vehicle road testing, real vehicle road operation, etc. Among them, the long tail scene in automatic driving refers to an extreme traffic situation with low occurrence probability but high risk and difficult to predict, such as an abnormal-shaped vehicle, extreme weather, and sudden obstruction, etc.

[0080] S302, in each test process, obtain the running state data at the current moment under the condition that the takeover request is triggered; S303, obtain a historical running state data sequence at a plurality of continuous sampling moments before the current moment; Among them, the acquisition method and specific data type of the running state data at the current moment and the historical running state data sequence are the same as the content in the foregoing step S101, and will not be repeated here.

[0081] S304, obtain the real takeover result in response to the takeover request; Optionally, obtaining the real takeover result in response to the takeover request comprises: in response to the takeover request, if the driver successfully takes over the vehicle and keeps the vehicle in a safe driving state, marking the real takeover result as capable of safe takeover; if the driver does not take over the vehicle or after the driver takes over the vehicle, the vehicle is in a collision, lane deviation or instability situation, marking the real takeover result as incapable of safe takeover.

[0082] By giving a clear label definition to the real takeover result in the model training process, the model can clearly understand which input data corresponds to a successful and safe takeover and which data corresponds to a failed and dangerous takeover during training.

[0083] Specifically, a binary label is applied to the data of each takeover event. The takeover event corresponding to the driver successfully taking over and keeping the vehicle in a safe driving state is marked as +1, i.e. capable of safe takeover. The takeover event corresponding to any of the following conditions is marked as -1, i.e. incapable of safe takeover: the driver does not take over, a collision occurs after takeover, lane deviation occurs after takeover, and the vehicle is unstable after takeover. By binary labeling, the result can be based on objective and observable results rather than subjective inference, and only the success and safety of the takeover result are concerned, thereby improving the accuracy of the model prediction result and the safety controllability of the targeted result control.

[0084] Moreover, when training the neural network, the algorithm will continuously adjust the internal parameters to approach the real label for each input data, i.e. the vehicle, environment and driver state sequence at the takeover moment and before. In this way, the clear label provides a prepared target for model learning, which can shape a clear decision boundary in a complex high-dimensional space, thereby minimizing the ambiguous prediction area between yes and no, so that the correct control strategy can be triggered faster in actual application.

[0085] In this way, the model can learn deeper and more essential feature associations rather than just superficial behavior associations. By defining the decision boundary, the model can clearly divide the safe and dangerous decision boundary in the feature space, which is crucial for the subsequent model to make high-confidence judgments in actual application.

[0086] S305, the data obtained by multiple tests is arranged into a data pair including running state data, historical running state data sequence and actual takeover result, to form an original data set.

[0087] By repeatedly conducting extensive tests across multiple scenarios and with different drivers, recording operational status data, historical operational status data sequences, and actual takeover results during the tests, data pairs are formed for each test, constituting the original dataset. This approach covers diverse driving environments, such as highways, cities, traffic congestion, and inclement weather; different takeover triggers, such as system malfunctions, road construction, and emergency obstacle avoidance; and different types of drivers, such as age, driving experience, and reaction speed. This ensures the diversity and representativeness of the dataset, improves the model's generalization ability, and guarantees its robustness in complex and ever-changing real-world applications.

[0088] S306, Construct an initial model for judging the driver's ability to take over.

[0089] The initial model includes an input layer and an output layer. The input layer is used to input operating status data and historical operating status data sequences, and the output layer is used to output the predicted takeover result, which includes whether the driver can take over safely or not. Optionally, the formula for the initial model is as follows: ;in, The takeover result in response to the takeover request; f is a mapping function or neural network; s0 is the current running state data; s1 is the historical running state data at the first sampling point before the takeover request was issued; s n This refers to the historical operational status data at the nth sampling point before the takeover request was issued.

[0090] That is, combining Figure 2 The scenario example shown corresponds to the runtime status data s0 at the current moment when the takeover request was issued. ：

[0091] The vehicle motion state s at the nth sampling point before the time the takeover request was issued. n for:

[0092] Initialize the model as follows Figure 4 As shown, inpt i Let be the i-th input to the network, i∈[1,n0], where n0 is s0,s1,s2,...,s t The number of state information items in the composition, inpt i For [s0,s1,s2,...,s] t The i-th piece of information, Indicates information data inpt i Normalization processing, if inpt i For distance information, you can first process inpt.i is the inverse of the natural logarithm function, and f is the normalization processing. i,j is the function of the i-th layer j-th neuron, which can be selected as sigmoid or ReLu function, and f i,j (x) is the function output of the i-th layer j-th neuron, is the function output of the i-1-th layer j-th neuron, W i-1,j,k is the weight of the i-1-th layer j-th neuron output, b i,j is the offset in the i-th layer j-th neuron; f oupt is the output layer function, which can be selected as tanh function or other functions, y out is the output layer function output, w o,i is the weight of the k-th layer k-th neuron output, b o is the offset in the output layer function, y out =1 is the safe takeover, y out =-1 is the failure of the safe takeover.

[0093] The parameters in the initial model are obtained by using the following formula (4).

[0094] Formula (4) In formula (14), y m is the driver safe takeover result of the m-th test experiment, y m =1 indicates that the driver successfully takes over, y m =-1 indicates that the driver fails to take over; s 0,m is the motion state of the vehicle when the autonomous driving system prompts and requests the driver to take over in the m-th experiment, s 1,m is the motion state of the vehicle at the first sampling time before the autonomous driving system prompts and requests the driver to take over in the m-th experiment, s 2,m is the motion state of the vehicle at the second sampling time before the autonomous driving system prompts and requests the driver to take over in the m-th experiment, s n,m is the motion state of the vehicle at the n-th sampling time before the autonomous driving system prompts and requests the driver to take over in the m-th experiment; w 1,1,1 , w 1,1,2 ,..., w 1,1,n1 , b 1,1 ; w 1,2,1 , w 1,2,2 ,..., w 1,2,n2 , b 1,2 ;..., w 1,i,1 , w 1,i,2 ,..., w 1,i,ni ,..., w o,1 , w o,2 ,..., wo,nk ,b o These are the model parameters in the initial model.

[0095] S307 divides the original dataset into a training set, a validation set, and a test set; S308 uses the training set to iteratively optimize the model parameters of the initial model; Optionally, the model parameters of the initial model are iteratively optimized using the training set, including: using the training set, employing the backpropagation algorithm and the optimization algorithm, and using the mean squared error between the model's predicted takeover results and the actual takeover results as the loss function to train the initial model.

[0096] In this embodiment, forward propagation is performed. Specifically, a batch of sample data, such as 32 or 128 data pairs, is taken from the training set, and the state sequence of this batch of data is... , The data is input into the current state of the deep neural network model. Starting from the input layer, the data passes through hidden layers, where it is weighted, summed, and an activation function, such as ReLU, is applied. Finally, the data reaches the output layer, yielding the predicted values ​​y for all samples in the batch. out The output is a batch of predicted value vectors y. out batch Then, loss calculation is performed, and the predicted value y output by the model is used. out batch The true label Y corresponding to this batch of samples batch The comparison is performed. The mean-squared error (MSE) is used as the loss function to calculate the loss value. The MSE loss function amplifies the impact of larger errors, forcing the model to prioritize correcting samples with large prediction biases. The calculated loss value is a scalar representing the average prediction error of the current model on this batch of data. Then, backpropagation is performed. Based on the calculated loss value (Loss), the gradient of the loss function with respect to the weights W and biases b of each layer in the model is calculated using the chain rule. The gradient values ​​corresponding to all parameters are output. Using the calculated gradients, the Adam optimizer is used to update the model parameters. The Adam optimizer automatically adjusts the effective learning step size for each parameter, making the training process converge faster. The above steps are repeated until all samples in the training set have been learned by the model. Traversing the entire training set is called one epoch. The entire training process includes hundreds or even thousands of epochs. In each epoch, the training set is shuffled and divided into multiple batches for training, which helps improve the model's generalization ability and training stability. The specific training epoch can be limited according to the actual test data; no specific limit is set here.

[0097] S309 utilizes the validation set for model performance monitoring and overfitting control during training. During the training process, after one or several cycles, the training is paused, and the current model is evaluated on the validation set. Using the same MSE loss function as in training, the loss value of the model on the validation set is calculated. At the beginning of training, both the training set loss and the validation set loss continue to decrease. As training progresses, if the training set loss continues to decrease, but the validation set loss begins to stabilize or even rebound, it indicates that the model begins to overfit, that is, it excessively memorizes the noise and details of the training set, and loses the generalization ability. When overfitting occurs, training is immediately stopped. And roll back to the model parameters of the cycle with the lowest validation set loss, which is used as the best model.

[0098] S310, using the test set to evaluate the performance of the trained model, and locking the model parameters to obtain the driver takeover ability judgment model under the condition that the predetermined performance condition is met.

[0099] Before the training process is completely finished and the final parameters are locked, the best model parameters selected by the validation set are loaded. The test set data is input into the model for forward propagation. The final performance indicators of the model on the test set are calculated, including but not limited to: final MSE loss, accuracy, precision, recall, F1 score, etc. When the performance indicators of the model on the test set meet the predetermined performance conditions, such as the MSE loss being lower than the loss set threshold, or the accuracy being higher than the accuracy set threshold. The model structure and all corresponding weights and bias parameters at this time are locked to obtain the driver takeover ability judgment model.

[0100] Taking the high-speed scene emergency collision avoidance working condition as an example, as shown in Figures 5 to 8 , as shown in Figure 5 , the ego vehicle is driving on a two-lane highway in an automatic driving state, and at 5s, an accident vehicle is found 30 meters ahead, requesting the driver to take over, and the safe driving system judges that the driver cannot safely take over the vehicle and performs safety control, avoiding collision with the right vehicle on the premise of ensuring safety, and avoiding collision with the front accident vehicle by changing lanes. The vehicle operating mode is as shown in Figure 6 , status=1 is the automatic driving mode, status=0 is the manual driving mode, status=0.5 is the automatic driving system requesting the driver to take over the mode, status=1.5 is the driver starting the automatic driving mode, and status=2 is the safe driving system controlling the vehicle safe driving mode. During vehicle driving, the longitudinal speed and steering wheel angle are as shown in Figure 7 . The driving path of the vehicle is as shown in Figure 8 .

[0101] The present disclosure provides a control method for automatic driving takeover. According to a driver-in-the-loop simulation test, a driver-in-the-loop bench test, or a driver real vehicle field test, a real vehicle road test, a real vehicle road operation, and the like, the running state data of an autonomous vehicle when the autonomous driving system prompts and requests the driver to take over, the running state data of the autonomous vehicle at multiple time points within a preset window before the autonomous driving system prompts and requests the driver to take over, and the result of whether the driver successfully and safely takes over the vehicle are recorded. A driver takeover capability judgment model is designed and constructed. Then, according to all the data and model characteristics, the driver safe takeover judgment model parameters are determined through model parameter identification, and the model of whether the driver can complete safe takeover is established. During the operation of the autonomous driving system, whether the driver can safely take over the vehicle when the autonomous driving system sends a takeover request is determined by using the pre-trained driver takeover capability judgment model. If the driver can safely take over the vehicle, the safe driving system does not work. If the driver cannot safely take over the vehicle, the safe driving system controls the vehicle safely. In this way, the control method provided by the present disclosure can improve the accurate judgment of the driver's takeover capability, and according to the judgment result, the corresponding control operation is given, thereby reducing the accident rate of the autonomous driving system in emergency or long-tail scenarios and improving the running safety of the vehicle.

[0102] In some embodiments, in combination Figure 9 As shown in FIG. 9, a control device 90 for automatic driving takeover is provided, which includes: A data acquisition module 910 configured to, when the vehicle triggers a driving takeover request during automatic driving, acquire the running state data at the current time and the historical running state data sequence within a preset time window; A takeover judgment module 920 configured to input the vehicle running state data and the historical running state data sequence into a pre-trained driver takeover capability judgment model, the driver takeover capability judgment model being configured to map the input data to a prediction result for characterizing the driver's takeover capability, the prediction result including being able to safely take over or being unable to safely take over; A safety control module 930 configured to, when the prediction result is that the driver can safely take over the vehicle, the driver takes over the vehicle, and the driver can safely take over means that the driver can successfully take over the vehicle and keep the vehicle in a safe driving state; and configured to, when the prediction result is that the driver cannot safely take over, triggering the safe driving system to take over the vehicle, and the driver cannot safely take over means that the driver cannot take over the vehicle or the vehicle will be in an abnormal state after the driver takes over the vehicle.

[0103] In some embodiments, in combination Figure 10As shown, an intelligent driving system 100 is provided, comprising a safe driving system 110, the safe driving system comprising an automatic driving takeover control device as described in any of the above embodiments, and the safe driving system being configured to control the vehicle to run in the case that the takeover judgment result is that the takeover cannot be safely performed. An automatic driving system 120 is configured to control the vehicle to perform automatic driving, and to send a driving takeover request to the control device.

[0104] The intelligent driving system provided by the present disclosure comprises a safe driving system independent of the automatic driving system. The automatic driving system is a conventional automatic driving system in an intelligent driving vehicle, and can realize automatic driving of the vehicle. The automatic driving system can send a driving takeover request to hand over the driving right to the driver in the case that a condition satisfying the trigger takeover request occurs during automatic driving. In the related art, the takeover level is determined according to the current state of the driver and the current state of the vehicle, and a takeover prompt is given to remind the driver to take over. Or in the case that the driver does not respond, the automatic driving system still takes over.

[0105] In the present application, by setting a safe driving system independent of the automatic driving system, in the case that a takeover request is monitored, the running state data at the current time and the historical running state data sequence within a preset time window before the current time are obtained, and the running state data at the current time and the historical running state data sequence are input into a pre-trained driver takeover capability judgment model together, to predict whether the driver can take over the vehicle. When the prediction result output by the driver takeover capability judgment model is that the takeover can be safely performed, the control right of the vehicle is handed over to the driver. When the prediction result output by the driver takeover capability judgment model is that the takeover cannot be safely performed, the safe driving system is immediately activated, and the safe driving system takes over the vehicle. Compared with the emergency collision avoidance control system in the prior art, the safe driving system provided by the present disclosure makes a judgment on the takeover capability of the driver as soon as the automatic driving system sends a takeover request, and can take over the control of the vehicle in the case that the prediction result is that the takeover cannot be safely performed. In this way, in the case that the driver cannot take over, the safe driving system intervenes in the control of the vehicle in advance, fully utilizes the time before the accident collision, controls the vehicle, leaves more sufficient time for active collision avoidance control, and reduces the accident rate. That is, the present disclosure can start safe takeover control of the vehicle before the vehicle running parameters meet the trigger condition of the emergency collision avoidance control system. Moreover, the intervention of the safe driving system is not emergency braking, but MPC-based cooperative safety control, which fully utilizes the road space, realizes a smoother, safer and more intelligent obstacle avoidance strategy, avoids secondary hazards caused by single sharp braking or turning, and significantly improves the safety and reliability of the automatic driving vehicle in emergency working conditions, In combinationFigure 11 As shown, the embodiment of the present disclosure provides an automatic driving takeover control device 1100, which comprises a processor 1102 and a memory 1104. Optionally, the device 1100 can further comprise a communication interface 1106 and a bus 1108. The processor 1102, the communication interface 1106 and the memory 1104 can communicate with each other through the bus 1108. The communication interface 1106 can be used for information transmission. The processor 1102 can invoke the logic instructions in the memory 1104 to execute the automatic driving takeover control method of the above-mentioned embodiments.

[0106] In addition, the logic instructions in the memory 1104 described above can be implemented in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer readable storage medium.

[0107] The memory 1104 as a computer readable storage medium can be used to store software programs, computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor 100 executes the program instructions / modules stored in the memory 1104, thereby performing functional applications and data processing, i.e. implementing the automatic driving takeover control method in the above-mentioned embodiments.

[0108] The memory 1104 can include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required by at least one function; the data storage area can store data created according to the use of the terminal device, etc. In addition, the memory 1104 can include a high-speed random access memory, and can also include a non-volatile memory.

[0109] In some embodiments, a vehicle is provided, comprising: a vehicle body; an automatic driving takeover control device 90 (1100) or an intelligent driving system 100 as described in any of the above-mentioned embodiments, which is installed on the vehicle body.

[0110] The installation relationship expressed herein is not limited to being placed inside the vehicle body, but also includes installation connection with other components of the vehicle, including but not limited to physical connection, electrical connection or signal transmission connection, etc. Those skilled in the art can understand that the automatic driving takeover control device 90 (1100) can be adapted to a feasible vehicle body, and thus other feasible embodiments can be realized.

[0111] The embodiment of the present disclosure provides a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are set to execute the above-mentioned automatic driving takeover control method.

[0112] The technical solutions of the embodiments of the present disclosure can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes one or more instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method disclosed in the embodiments of the present disclosure. The aforementioned storage medium can be a non-transitory storage medium, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0113] The above description and drawings sufficiently illustrate the embodiments of the present disclosure to enable one skilled in the art to practice them. Other embodiments can include structural, logical, electrical, process, and other changes. The embodiments represent only a few of the possible variations. Individual components and functions are optional unless explicitly required, and the order of operations can be changed. Parts and features of some embodiments can be included or replaced by parts and features of other embodiments. Also, the words used in this application are only used to describe the embodiments and not to limit the claims. As used in the description of the embodiments and the claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of one or more associated listed items. In addition, when used in this application, the term "comprise" and its variants "comprises" and / or comprises" and the like mean the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, or device that includes the stated element. In this document, each embodiment focuses on the differences from other embodiments, and the same or similar parts between embodiments can be referred to each other. For the method, product, etc. disclosed in the embodiments, if it corresponds to the method part disclosed in the embodiments, the relevant part can be referred to the description of the method part.

[0114] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods for each specific application to realize the described functions, but such implementation should not be considered beyond the scope of the embodiments of the present disclosure. The skilled person can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0115] In the embodiments disclosed herein, the disclosed methods, products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the above-described device embodiments are only schematic, for example, the division of the units can only be a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms. The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to implement the embodiments. In addition, each functional unit in the embodiments of the present disclosure can be integrated in one processing unit, or each unit can be a physically independent unit, or two or more units can be integrated in one unit.

[0116] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other processing device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other processing device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

Claims

1. A control method for automatic driving takeover, characterized in that, include: When the vehicle is in autonomous driving mode and triggers the autonomous driving system to issue a driver takeover request, it acquires the current operating status data and the historical operating status data sequence within a preset time window. Vehicle operating status data and historical operating status data sequences are input into a pre-trained driver takeover capability judgment model. This model is configured to map the input data to predictive results characterizing the driver's takeover capability, including whether the driver can safely take over or cannot safely take over. The steps for constructing the driver takeover capability judgment model include: simulating various autonomous driving scenarios under multiple test conditions; acquiring the current operating status data when a takeover request is triggered during each test; acquiring historical operating status data sequences from multiple consecutive sampling times prior to the current time; acquiring the actual takeover result in response to the takeover request; organizing the data obtained from multiple tests into data pairs including operating status data, historical operating status data sequences, and actual takeover results to form the original dataset; constructing an initial model for driver takeover capability judgment, which includes an input layer and an output layer. The input layer is used to input the operating status data and historical operating status data sequences, and the output layer is used to output the predicted takeover result, including whether the driver can safely take over or cannot safely take over; and training and testing the initial model using the original dataset to obtain the driver takeover capability judgment model. If the prediction result is that the driver can take over safely, then the driver will take over the vehicle. Taking over safely means that the driver can successfully take over the vehicle and keep it in a safe driving state. If the prediction result indicates that the driver cannot safely take over, the safe driving system will be triggered to take over the vehicle. "Unsafe takeover" means that the driver cannot take over the vehicle or that the vehicle will exhibit abnormal behavior after the driver takes over.

2. The control method according to claim 1, characterized in that, Operational status data includes vehicle motion status data, environmental data, and driver status data; The vehicle motion data includes: vehicle speed, acceleration, vehicle body angle, angular velocity, angular acceleration, and tire pressure. Environmental data includes: weather, light intensity, road information, and relative distance to surrounding objects; Driver status data includes one or more of the following: facial orientation, eye status, mouth opening and closing frequency and degree.

3. The control method according to claim 1, characterized in that, Obtain historical running status data sequences, including: Acquire vehicle operating status data from multiple consecutive sampling moments within a preset time window prior to the current moment; The operational status data from multiple consecutive sampling times are sorted by time to form a historical operational status data sequence.

4. The control method according to any one of claims 1 to 3, characterized in that, Obtain the actual takeover result in response to the takeover request, including: In response to a takeover request, if the driver successfully takes over the vehicle and keeps it in a safe driving state, the actual takeover result is marked as a safe takeover. If the driver does not take over the vehicle, or if the vehicle collides, deviates from its lane, or becomes unstable after the driver takes over, the actual takeover result is marked as unsafe takeover.

5. The control method according to claim 4, characterized in that, The initial model was trained and tested using the original dataset to obtain a driver takeover capability assessment model, including: The original dataset is divided into a training set, a validation set, and a test set; Using the training set, iteratively optimize the model parameters of the initial model; During training, the validation set is used to monitor model performance and control overfitting. The performance of the trained model is evaluated using a test set. Under predetermined performance conditions, the model parameters are locked to obtain a driver takeover capability judgment model.

6. The control method according to claim 5, characterized in that, Using the training set, iteratively optimize the model parameters of the initial model, including: Using the training set, the backpropagation algorithm and optimization algorithm are employed, with the mean squared error between the model's predicted takeover results and the actual takeover results used as the loss function to train the initial model.

7. The control method according to any one of claims 1 to 3, characterized in that, The formula for the initial model is as follows: ; in, The takeover result in response to the takeover request; f is a mapping function or neural network; s0 is the current running state data; s1 is the historical running state data at the first sampling point before the takeover request was issued; s n This refers to the historical operational status data at the nth sampling point before the takeover request was issued.

8. The control method according to any one of claims 1 to 3, characterized in that, Triggering the safety driving system to take over the vehicle includes: Based on the model predictive control algorithm and combined with the constraints, the acceleration control command and steering angle control command are obtained. The constraints include: safety limits for longitudinal and lateral acceleration of the vehicle, safety distance constraints from surrounding obstacles, and vehicle dynamics model constraints. The vehicle is controlled according to acceleration control commands and steering angle control commands.

9. The control method according to any one of claims 1 to 3, characterized in that, A driver takeover request is triggered by at least one of the following conditions: The vehicle is about to leave or has already left the design operating domain of its autonomous driving system; The vehicle's perception system detected that its performance degradation or information conflict exceeded the safety threshold; The predicted collision time based on the vehicle's current state is below the safety threshold.

10. A control device for automatic driving takeover, characterized in that, include: The data acquisition module is configured to acquire the current operating status data and the historical operating status data sequence within a preset time window when the vehicle triggers a driver takeover request during autonomous driving. The takeover judgment module is configured to input vehicle operating status data and historical operating status data sequences into a pre-trained driver takeover capability judgment model. The driver takeover capability judgment model is configured to map a prediction result characterizing the driver's takeover capability based on the input data. The prediction result includes whether the driver can safely take over or cannot safely take over. The steps for constructing the driver takeover capability judgment model include: simulating various autonomous driving scenarios under multiple test conditions; acquiring the current operating status data when a takeover request is triggered during each test; acquiring historical operating status data sequences from multiple consecutive sampling times prior to the current time; acquiring the actual takeover result in response to the takeover request; organizing the data obtained from multiple tests into data pairs including operating status data, historical operating status data sequences, and actual takeover results to form the original dataset; constructing an initial model for driver takeover capability judgment, which includes an input layer and an output layer. The input layer is used to input operating status data and historical operating status data sequences, and the output layer is used to output the predicted takeover result, which includes whether the driver can safely take over or cannot safely take over; and training and testing the initial model using the original dataset to obtain the driver takeover capability judgment model. The safety control module is configured to allow the driver to take over the vehicle if the prediction indicates safe takeover capability. Safe takeover capability means the driver can successfully take over the vehicle and maintain it in a safe driving state. It is configured to trigger the safe driving system to take over the vehicle if the prediction result is that the driver cannot take over safely. "Unsafe takeover" means that the driver cannot take over the vehicle or that the vehicle will exhibit abnormal driving behavior after the driver takes over.

11. A control device for automatic driving takeover, comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to execute the control method for autonomous driving takeover as described in any one of claims 1 to 9 when running the program instructions.

12. An intelligent driving system, characterized in that, include: A safe driving system, comprising the control device for automatic driving takeover as described in claim 10 or 11, and configured to control vehicle operation when the takeover determination result indicates that safe takeover is not possible; An autonomous driving system is configured to control the vehicle to drive autonomously and to issue a driver takeover request to the control unit.

13. A vehicle, characterized in that, include: Vehicle body; The control device for automatic driving takeover as described in claim 10 or 11 is installed on the vehicle body; or The intelligent driving system as described in claim 12 is installed on the vehicle body.

14. A readable storage medium storing program instructions, characterized in that, When the program instructions are executed, they cause the computer to perform the control method for automatic driving takeover as described in any one of claims 1 to 9.

Images