Control method and device for automatic driving takeover, intelligent driving system, vehicle

Abstract

The application relates to the technical field of intelligent driving, for example, to a control method and device for automatic driving takeover, an automatic driving system, a vehicle and a readable storage medium. The control method comprises the following steps: in the process of automatic driving, when the automatic driving system sends a driving takeover request, obtaining running state data at the current time and a historical running state data sequence in a preset time window; inputting the vehicle running state data and the historical running state data sequence into a pre-trained driver takeover capability judgment model to output a prediction result; if the prediction result is that the vehicle can be safely taken over, then the driver takes over the vehicle; if the prediction result is that the vehicle cannot be safely taken over, then a safety driving system takes over the vehicle. The present application predicts the takeover capability by using the driver takeover capability judgment model, and triggers the safety driving system to start a safety control vehicle strategy based on the prediction result that the vehicle cannot be safely taken over, thereby reducing the accident rate.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving, for example, to a control method and device for automatic driving takeover, an automatic driving system, a vehicle, and a readable storage medium. BACKGROUND

[0002] In the case that the intelligent driving technology is faced with a complex or changeable traffic scene and there is a driving risk, the driving right is handed over to the driver for driving by issuing a takeover request of the driving right. In order to improve the safety of the driver taking over the vehicle, in the related technology, the image related to the driver and the environmental information of the vehicle are acquired, the takeover level is determined, and the corresponding takeover response required to be executed is determined according to the takeover level.

[0003] In the process of implementing the embodiments of the present disclosure, it is found that at least the following problems exist in the related technology:

[0004] The data in the related technology only depends on the current image of the driver and the environmental information of the vehicle, and the single-time data reduces the accuracy of determining the takeover level, which is prone to the risk of misjudgment or omission. Moreover, after the driver state features and the scene features are obtained and input into the intelligent takeover prompt model, the takeover level output by the intelligent takeover prompt model can be acquired. The manner of the intelligent takeover prompt model outputting the takeover level includes directly outputting the takeover level with the maximum matching probability, or outputting the matching probability corresponding to each takeover level. For example, the matching probability of the takeover level 1 is 0, the matching probability of the takeover level 2 is 0, the matching probability of the takeover level 3 is 45%, the matching probability of the takeover level 4 is 40%, and the matching probability of the takeover level 5 is 15%. Different takeover responses are performed for different takeover levels. The takeover response corresponding to the takeover level 1 is non-prompt, the takeover response corresponding to the takeover level 2 is visual prompt, the takeover response corresponding to the takeover level 3 is simultaneous visual and auditory prompt, and the takeover responses corresponding to the takeover level 4 and the takeover level 5 are active takeover.

[0005] It can be seen that in the related art, in the case that the vehicle issues a takeover request, the takeover level is divided into multiple levels by a solidified probability matching rule, resulting in low accuracy of the takeover level. Moreover, the multiple levels are divided into multiple levels with different matching probabilities, resulting in the need for the system to process multiple levels of takeover responses, and the takeover responses mainly include prompts and active takeover of the vehicle system. The prompts themselves have uncertainty, resulting in a low success probability of the takeover result and high safety risks. And the active takeover still adopts the automatic driving of the vehicle system, i.e., the risk of the automatic driving does not change before and after the takeover determination, resulting in low driving safety of the vehicle. As can be seen, the takeover control in the related art has poor accuracy in the determination of the takeover ability of the driver and the prediction of the takeover result, and there is a great safety hazard in the takeover control. SUMMARY

[0006] The following presents a simplified summary of some aspects of the disclosed embodiments in order to provide a basic understanding of such embodiments. The summary is not an extensive overview of the disclosure and is not intended to identify key / critical elements of the embodiments or to delineate the scope of the embodiments. Its sole purpose is to present some embodiments in a simplified form as a prelude to the more detailed description that is presented later.

[0007] The embodiments of the present disclosure provide a control method and device for automatic driving takeover, an automatic driving system, a vehicle, and a readable storage medium, to improve the accuracy of the determination of the takeover ability and thus improve the driving safety.

[0008] In some embodiments, a control method for automatic driving takeover is provided, including: in the case that a vehicle triggers the automatic driving system to issue a driving takeover request in the automatic driving process, acquiring running state data at the current time and a historical running state data sequence within a preset time window; inputting the vehicle running state data and the historical running state data sequence into a pre-trained driver takeover ability judgment model, the driver takeover ability judgment model being configured to map a prediction result for characterizing the takeover ability of the driver based on the input data, the prediction result including being able to safely take over or being unable to safely take over; in the case that the prediction result is being able to safely take over, taking over the vehicle by the driver, the being able to safely take over indicating that the driver can successfully take over the vehicle and keep the vehicle in a safe driving state; and in the case that the prediction result is being unable to safely take over, triggering a safe driving system to take over the vehicle, the being unable to safely take over indicating that the driver cannot take over the vehicle or the vehicle will be in an abnormal state after the driver takes over the vehicle.

[0009] The control method for automatic driving takeover provided by the present disclosure, at the current time when the takeover request is triggered, obtains the running state data at the current time and the historical running state data sequence within a preset time window. And the running state data at the current time and the historical running state data sequence are input into the driver takeover ability judgment model trained in advance. When the prediction result output by the driver takeover ability judgment model is that it can be safely taken over, the control right of the vehicle is given to the driver. When the prediction result output by the driver takeover ability judgment model is that it cannot be safely taken over, the safe driving system is immediately activated, and the vehicle is taken over by the safe driving system.

[0010] In this way, the control method provided by the present disclosure, at the current time when the takeover request is triggered, simultaneously obtains the running state data at the current time and the historical running state data sequence within a preset time window before the current time. That is, according to the current running state data and the historical running state data sequence before the current time, the running state data in a continuous time period is obtained, which can accurately reflect the change trend of the running state of the vehicle and the change trend of the state of the driver through the running state data in the continuous time period. Further, the current running state data and the historical running state data sequence before the current time are synchronously input into the driver takeover ability judgment model trained in advance to predict the takeover ability of the driver. That is, compared with the related art which uses single-time data to determine the takeover of the driver's state and the vehicle state, the present disclosure combines the running state data in a continuous time period and the driver takeover ability judgment model trained in advance, which can greatly improve the accuracy and reliability of the prediction of the driver's takeover ability, and reduce the risk of misjudgment or omission of the driver's state or the vehicle state by single data.

[0011] Further, in the scheme adopted by the present application, the current running state data and the historical running state data sequence before the current time are synchronously input into the pre-trained driver takeover ability judgment model, and the output prediction result includes two kinds, one is that the driver can safely take over, and the other is that the driver cannot safely take over. That is, in the scheme provided by the present application, the two-class prediction model with takeover success or failure as the output directly gives an explicit judgment result, and according to the two results, the corresponding processing logic is given, and then the control closed loop from triggering request, result prediction to operation is realized, and the safety of driving control is improved. Among them, for the prediction result of being able to safely take over, the driving right is given to the driver, that is, according to the input data, it is predicted that the driver can take over the vehicle, and after taking over the vehicle, the vehicle can be safely driven. And for the prediction result of being unable to safely take over, that is, according to the input data, it is predicted that the driver cannot take over the vehicle, or even if the driver takes over the vehicle, it is predicted that the driver takes over the vehicle according to the state of the driver or the running state of the vehicle. The probability of vehicle abnormality after taking over the vehicle is high, the safety driving system is immediately activated, and the vehicle is taken over by the safety driving system. In this way, in the case that the driver cannot take over, the safety driving system is intervened in the vehicle control in advance, the time before the accident collision is fully utilized, the vehicle is controlled, more sufficient time is left for active collision avoidance control, and the accident rate is reduced.

[0012] Compared with the way of predicting multiple takeover levels in the related art, and prompting or self-driving system takeover for different takeover levels, the present disclosure outputs the prediction result of the pre-trained driver takeover ability judgment model as a two-class prediction result of being able to take over or being unable to take over. Being able to take over means that the driver can take over and can safely drive after taking over the vehicle, and being unable to take over means that the driver cannot take over, or the vehicle will have abnormal situations such as collision and rollover after taking over. And for the case of being unable to take over, the present disclosure takes over the vehicle by starting the safety driving system. In this way, the automatic takeover control method provided by the present disclosure improves the accuracy of the prediction result, and can improve the reliability and stability of the vehicle takeover driving safety.

[0013] Optionally, the running state data includes vehicle motion state data, environment data and driver state data; wherein the vehicle motion state data includes: vehicle speed, acceleration, and angle, angular velocity, angular acceleration of the vehicle body, and tire pressure; the environment data includes: weather, illumination intensity, road information and relative distance to surrounding objects; the driver state data includes one or more of the following: face direction, eye state, mouth opening and closing frequency and opening and closing degree.

[0014] In this embodiment, the vehicle motion state is accurately determined whether it is within the safe boundary range through the longitudinal, lateral and vertical speed, acceleration, the angle, angular velocity and angular acceleration of the vehicle body roll, pitch and yaw in three directions; the maximum braking force and braking distance of the vehicle are determined through the tire pressure, weather conditions such as rainfall in rainy weather, road curvature, slope information, water accumulation and icing conditions; the vehicle handling stability is determined through the tire pressure and road potholes, bumps, water accumulation and icing; the sensor distance measurement accuracy for surrounding objects is determined through the fog visibility and light intensity. The driver's distraction degree and fatigue driving degree are determined through the driver's face direction, eye state such as opening degree, blinking frequency, mouth opening and closing frequency and opening and closing degree. Optionally, a historical running state data sequence is obtained, including: obtaining the running state data of the vehicle at a plurality of continuous sampling time points within a preset time window before the current time; the running state data at the plurality of continuous sampling time points is sorted according to time to form a historical running state data sequence.

[0015] In this embodiment, considering that the data at a single time point may have noise or contingency, which may affect the determination result. The application obtains the running state data of the vehicle at a plurality of continuous sampling time points within a preset time window before the current time, and forms a historical running state data sequence. By introducing the historical running state data sequence, the instantaneous noise and accidental interference can be effectively filtered out, so that the evaluation result is more stable and reliable, and is not easily affected by instantaneous abnormal interference. In addition, the state change trend can be captured, which provides the possibility for forward-looking safety decision, and greatly improves the robustness and reliability of the driver's takeover ability evaluation.

[0016] Optionally, the step of constructing the driver takeover ability judgment model comprises: simulating a plurality of automatic driving scenes by the driver under a plurality of test working conditions; obtaining the running state data at the current time when the takeover request is triggered during each test; obtaining a historical running state data sequence at a plurality of continuous sampling time points before the current time; obtaining the real takeover result in response to the takeover request; arranging the data obtained by the plurality of tests into a data pair including the running state data, the historical running state data sequence and the actual takeover result to form an original data set; constructing an initial model for driver takeover ability judgment, the initial model including an input layer and an output layer, the input layer being used for inputting the running state data and the historical running state data sequence, and the output layer being used for outputting a predicted takeover result, the predicted takeover result including that the driver can safely take over or the driver cannot safely take over; training and testing the initial model by using the original data set to obtain the driver takeover ability judgment model.

[0017] In this embodiment, test scenarios under various test conditions are constructed, and multiple drivers are used for simulation testing. During the test, the running state data at the current time when the takeover request is triggered and the historical running state data sequence at multiple consecutive sampling times before the current time are obtained; and the original data set is obtained in response to the real takeover result corresponding to the takeover request. The initial model is trained and tested using the original data set to obtain the driver takeover ability judgment model. By covering different driving environments, different takeover trigger reasons and different types of drivers, the diversity and representativeness of the data set are ensured. The initial model is trained and verified using the original data set, which improves the generalization ability of the model, and thus the model can make accurate judgments in various complex scenarios, reducing the false positive rate and false negative rate of the takeover result prediction. Moreover, the overfitting problem of the model caused by single training data is avoided, ensuring the robustness of the model in complex and variable actual applications.

[0018] Optionally, the real takeover result in response to the takeover request is obtained, including: in response to the takeover request, if the driver successfully takes over the vehicle and keeps the vehicle in a safe driving state, marking the real takeover result as capable of safe takeover; if the driver does not take over the vehicle or the vehicle collides, deviates from the lane or loses stability after the driver takes over the vehicle, marking the real takeover result as incapable of safe takeover.

[0019] In this embodiment, by giving a clear label definition to the real takeover result during the model training process, the model can clearly understand which input data corresponds to successful and safe takeover and which data corresponds to failed and dangerous takeover during training. This enables the model to learn deeper and more essential feature associations, rather than just superficial behavior associations. By defining the decision boundary, the model can clearly divide the safe and dangerous decision boundary in the feature space, which is crucial for the model to make high-confidence judgments in actual applications.

[0020] Optionally, the initial model is trained and tested using the original data set to obtain the driver takeover ability judgment model, including: dividing the original data set into a training set, a validation set and a test set; using the training set to iteratively optimize the model parameters of the initial model; using the validation set to monitor the performance of the model and control overfitting during the training process; using the test set to evaluate the performance of the trained model, and locking the model parameters to obtain the driver takeover ability judgment model under the condition that the predetermined performance condition is met.

[0021] In this embodiment, by dividing the original data set into a training set, a validation set and a test set; the model parameters of the initial model are iteratively optimized using the training set to improve the generalization ability of the model. The validation set is used to monitor the model performance during training to assist in hyperparameter tuning and early stopping to prevent overfitting. When the model achieves satisfactory performance on both the training set and the validation set, the test set that has never participated in training and tuning is used to evaluate the model. If the performance on the test set meets the predetermined indicators, such as accuracy, recall rate, etc., the model parameters at this time are locked, and the final driver takeover ability judgment model is obtained.

[0022] Optionally, the model parameters of the initial model are iteratively optimized using the training set, including: using the training set, using the back propagation algorithm and the optimization algorithm, taking the mean square error between the predicted takeover result of the model and the real takeover result as the loss function, training the initial model.

[0023] In this embodiment, the loss function sets a clear and safe oriented optimization goal for the model; the back propagation algorithm solves the calculation problem of parameter optimization in high-dimensional space; the optimization algorithm realizes an efficient and automated optimization process; and the combination of the three improves the accuracy and confidence of the probability prediction of the model output.

[0024] Optionally, the formula of the initial model is as follows: y = f(s0, s1, s2…s n ); wherein y is the takeover result in response to the takeover request; f is a mapping function or a neural network; s0 is the running state data at the current time; s1 is the historical running state data at the first sampling point before the takeover request is sent; and sn is the historical running state data at the nth sampling point before the takeover request is sent. n

[0025] Optionally, triggering the safe driving system to take over the vehicle comprises: based on the model predictive control algorithm, combining the constraint conditions, solving the acceleration control instruction and the steering angle control instruction, the constraint conditions including: the safety limit value of the vehicle longitudinal acceleration and the lateral acceleration, the safety distance constraint with the surrounding obstacles, and the vehicle dynamics model constraint; and controlling the vehicle to run according to the acceleration control instruction and the steering angle control instruction.

[0026] ​In this embodiment, by adopting the model predictive control algorithm, the control command solved under the safety distance constraint can conform to the physical characteristics of the vehicle, avoiding secondary accidents such as vehicle sideslip and spin due to excessive steering or braking in emergency situations, and ensuring the safety of the control process itself. Moreover, the acceleration and steering commands generated by the optimization objective function of the model predictive control algorithm are continuous and smooth, rather than step or impact, and the smooth intervention mode reduces the tension and discomfort of the passengers. Furthermore, by adding constraint conditions, the adaptability of different driving scenarios can be improved.

[0027] Optionally, triggering the driving takeover request comprises at least one of the following conditions: the vehicle is about to exit or has exited the designed operating domain of its autonomous driving system; the perception system of the vehicle detects that its performance degradation or information conflict exceeds a safety threshold; the predicted collision time based on the current state of the vehicle is below a safety threshold.

[0028] In some embodiments, an automatic driving takeover control device is provided, comprising:

[0029] A data acquisition module configured to, in a case where the vehicle triggers a driving takeover request during autonomous driving, acquire running state data at the current time and a historical running state data sequence within a preset time window;

[0030] A takeover judgment module configured to input the vehicle running state data and the historical running state data sequence into a pre-trained driver takeover capability judgment model, the driver takeover capability judgment model being configured to map a prediction result for characterizing the driver's takeover capability based on the input data, the prediction result including being able to safely take over or being unable to safely take over;

[0031] A safety control module configured to, in a case where the prediction result is being able to safely take over, take over the vehicle by the driver, the being able to safely take over meaning that the driver can successfully take over the vehicle and keep the vehicle in a safe driving state; and configured to, in a case where the prediction result is being unable to safely take over, trigger the safety driving system to take over the vehicle, the being unable to safely take over meaning that the driver cannot take over the vehicle or the vehicle will be in an abnormal state after the driver takes over the vehicle.

[0032] In some embodiments, an automatic driving takeover control device is provided, comprising a processor and a memory storing program instructions, the processor being configured to execute an automatic driving takeover control method as described in any of the above embodiments when running the program instructions.

[0033] In some embodiments, an intelligent driving system is provided, comprising: a safe driving system, the safe driving system comprising the automatic driving takeover control device according to any one of the preceding embodiments, and the safe driving system being configured to control the vehicle to run in a case where the takeover judgment result is that the takeover cannot be safely performed; and an automatic driving system configured to control the vehicle to perform automatic driving and send a driving takeover request to the control device.

[0034] In some embodiments, a vehicle is provided, comprising: a vehicle body; the automatic driving takeover control device according to any one of the preceding embodiments, or the intelligent driving system according to any one of the preceding embodiments, mounted on the vehicle body.

[0035] In some embodiments, a readable storage medium is provided, storing program instructions which, when executed, cause a computer to perform the automatic driving takeover control method according to any one of the preceding embodiments.

[0036] The automatic driving takeover control method and device, the automatic driving system, the vehicle, and the readable storage medium provided by the embodiments of the present disclosure can achieve the following technical effects:

[0037] (1) The automatic driving takeover control method provided by the present disclosure collects data at the moment when the automatic driving system sends a takeover request, and the data includes running state data at the current moment and a historical running state data sequence within a preset time window before the current moment. The running state data at the current moment and the historical running state data sequence within the preset time window before the current moment are used as the basis for judging the driving safety takeover ability. This multi-modal data type and continuous time-series running data improve the judgment of the driver's takeover ability, avoid the false triggering of the safe driving system when the driver can safely take over, avoid the missed triggering of the safe driving system when the driver cannot safely take over, and greatly improve the judgment accuracy of complex and dynamic takeover scenarios.

[0038] (2) The automatic driving takeover control method provided by the present disclosure uses a driver takeover ability judgment model constructed based on a neural network or a mapping function. The model takes the moment when the takeover request is sent and the running state data sequence within a preset time window before the moment as the model input, and thus can capture the evolution trend of the running data. The model output is an explicit safe takeover and an explicit unsafe takeover, which improves the executability of the prediction result. In the model training and testing process, the actual takeover success or takeover failure result is used as a supervision signal, and the model directly learns the safe takeover and unsafe takeover mode boundary through supervised learning, thereby improving the accuracy of the takeover ability prediction.

[0039] (3) The automatic driving takeover control method provided by the present disclosure makes a hierarchical decision based on the prediction result output by the driver takeover capability judgment model. If the prediction result is that the driver can safely take over, the safe driving system does not intervene, the driver takes over, and the vehicle control is performed. If the prediction result is that the driver cannot safely take over, the safe driving system triggers the safe control vehicle strategy of the safe driving system to directly perform vehicle control. During the vehicle control performed by the safe driving system, the MPC (Model Predictive Control) algorithm is used to fully utilize the driving space, obtain the optimal control strategy for safe operation of the vehicle by combining braking and steering, fully utilize the drivable space compared with the collision avoidance by braking, fully consider the safety of nearby lane traffic participants compared with the collision avoidance by steering, avoid or reduce the risk of collision with other traffic participants when switching to other lanes, and thus reduce the vehicle accident rate and improve the safety during vehicle operation.

[0040] (4) The automatic driving takeover control method provided by the present disclosure triggers the safe driving system to start the safe control vehicle strategy if the prediction result is that the driver cannot safely take over, and directly performs vehicle control. Compared with the active collision avoidance control method in the automatic driving system, the vehicle control performed by the safe driving system can trigger the entering of the obstacle avoidance control in advance, fully utilize the time before the potential accident occurs by precise and rapid safe control of the vehicle, and thus reduce the accident rate.

[0041] The foregoing general description and the following description are only exemplary and explanatory, and are not intended to limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0042] One or more embodiments are exemplarily illustrated by corresponding drawings, which are not intended to limit the embodiments, elements with the same reference numerals in the drawings are shown as similar elements, the drawings do not constitute a proportional limit, and wherein:

[0043] Figure 1 is a schematic diagram of an automatic driving takeover control method provided by an embodiment of the present disclosure;

[0044] Figure 2 is an example diagram of a driving scene diagram provided by an embodiment of the present disclosure;

[0045] Figure 3 is a schematic diagram of a method for constructing a driver takeover capability judgment model provided by an embodiment of the present disclosure;

[0046] Figure 4 is an initialization model schematic diagram of an embodiment of the present disclosure;

[0047] Figure 5is a high-speed scene emergency collision avoidance working condition application schematic diagram of an embodiment of the disclosure;

[0048] Figure 6 is Figure 5 is a schematic diagram of a vehicle running mode in the embodiment shown in the figure;

[0049] Figure 7 is Figure 5 is a schematic diagram of the longitudinal speed and the steering wheel angle during the vehicle driving process in the embodiment shown in the figure;

[0050] Figure 8 is Figure 5 is a schematic diagram of the driving path of the vehicle in the embodiment shown in the figure;

[0051] Figure 9 is a schematic diagram of an automatic driving takeover control device provided by an embodiment of the disclosure;

[0052] Figure 10 is a schematic diagram of an intelligent driving system provided by an embodiment of the disclosure;

[0053] Figure 11 is a schematic diagram of another automatic driving takeover control device provided by an embodiment of the disclosure. DETAILED DESCRIPTION

[0054] In order to be able to understand the features and technical contents of the embodiments of the disclosure more fully, the implementation of the embodiments of the disclosure will be described in detail below with reference to the accompanying drawings, which are only used for reference and do not limit the embodiments of the disclosure. In the following technical description, in order to facilitate explanation, through multiple details, a sufficient understanding of the disclosed embodiments is provided. However, one or more embodiments can still be implemented without these details. In other cases, in order to simplify the drawings, well-known structures and devices can be simplified.

[0055] The terms "first", "second", and the like in the specification and claims of the embodiments of the disclosure and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the disclosure described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion.

[0056] Unless otherwise specified, the term "a plurality of" means two or more.

[0057] In the embodiments of the disclosure, the character " / " represents an "or" relationship between the objects before and after it. For example, A / B represents: A or B.

[0058] The term "and / or" is a descriptive term that refers to an association relationship, which means that there can be three relationships. For example, A and / or B means that there are three relationships of A or B, or A and B.

[0059] The term "corresponding" can refer to an association relationship or a binding relationship. A corresponds to B means that there is an association relationship or a binding relationship between A and B.

[0060] In some embodiments, a control device for automatic driving takeover is provided, comprising a processor and a memory storing program instructions, the processor being configured to execute the automatic driving takeover control method as described in any of the embodiments below when running the program instructions.

[0061] In some embodiments, in combination Figure 1 As shown, a control method for automatic driving takeover is provided, comprising:

[0062] S101, in the process of automatic driving, when the automatic driving system triggers a driving takeover request, acquiring running state data at the current time and a historical running state data sequence within a preset time window.

[0063] Optionally, the running state data includes vehicle motion state data and environment perception data. The vehicle motion state data includes but is not limited to vehicle speed, acceleration, and vehicle body angle, angular velocity, angular acceleration, and tire pressure.

[0064] For example, the vehicle running state data includes but is not limited to vehicle longitudinal speed, vehicle lateral speed, vehicle vertical speed, vehicle longitudinal acceleration, vehicle lateral acceleration, vehicle vertical motion acceleration, vehicle body roll angle, vehicle body pitch angle, vehicle body yaw angle, vehicle yaw angular velocity, and tire pressure. The vehicle motion state is determined by the vehicle running state data whether it is within the safe boundary range.

[0065] For example, the environment perception data includes but is not limited to the relative state of the closest obstacle within the vehicle's perception range, and weather, light intensity, and road information. The environment perception data includes but is not limited to longitudinal relative distance from the vehicle, lateral relative distance from the vehicle, longitudinal relative speed from the vehicle, lateral relative speed from the vehicle, longitudinal relative acceleration from the vehicle, weather, light intensity, and road information. If there is no obstacle in a certain direction, a preset value is filled. The maximum braking force and braking distance of the vehicle are determined by the tire pressure, weather conditions such as rainfall in rainy weather, road curvature, slope information, water accumulation, and icing conditions; the vehicle handling stability is determined by the tire pressure and road potholes, bumps, water accumulation, and icing; the sensor distance measurement accuracy for surrounding objects is determined by the visibility in foggy weather and light intensity.

[0066] An exemplary driving scenario is shown in FIG. 2, where a vehicle 0 is a subject autonomous vehicle (AV) 200, and a vehicle 1 is a non-AV vehicle. The AV 200 is in a lane with a left boundary 21 and a right boundary 22. The AV 200 is also in a lane to the left of the AV 200 with a left boundary 20, and a lane to the right of the AV 200 with a right boundary 23. The AV 200 has a front object 1, a front object 2, a front object 3, a right object 4, a right object 5, a right object 6, a left object 7, and a left object 8. Figure 2 An exemplary driving scenario is shown in FIG. 2, where a vehicle 0 is a subject autonomous vehicle (AV) 200, and a vehicle 1 is a non-AV vehicle. The AV 200 is in a lane with a left boundary 21 and a right boundary 22. The AV 200 is also in a lane to the left of the AV 200 with a left boundary 20, and a lane to the right of the AV 200 with a right boundary 23. The AV 200 has a front object 1, a front object 2, a front object 3, a right object 4, a right object 5, a right object 6, a left object 7, and a left object 8.

[0067] Let v x00 be the longitudinal movement speed of the AV 200, v y00 be the lateral movement speed of the AV 200, be the vertical movement speed of the AV 200, be the longitudinal movement acceleration of the AV 200, a y00 be the lateral movement acceleration of the AV 200, be the vertical movement acceleration of the AV 200, be the roll angle of the AV 200, θ y00 be the pitch angle of the AV 200, be the yaw angle of the AV 200, be the roll angular velocity of the AV 200, be the pitch angular velocity of the AV 200, be the yaw angular velocity of the AV 200, be the roll angular acceleration of the AV 200, be the pitch angular acceleration of the AV 200, be the yaw angular acceleration of the AV 200. be the left front tire pressure of the AV 200, be the right front tire pressure of the AV 200, be the left rear tire pressure of the AV 200, be the right rear tire pressure of the AV 200. In addition, let be the longitudinal relative distance between the AV 200 and the object 1, be the lateral relative distance between the AV 200 and the object 1, be the longitudinal relative speed between the AV 200 and the object 1, be the lateral relative speed between the AV 200 and the object 1, be the longitudinal relative acceleration between the AV 200 and the object 1, the lateral relative acceleration of ego vehicle 200 and object 201; the longitudinal relative distance of ego vehicle 200 and object 202, the lateral relative distance of ego vehicle 200 and object 202, the longitudinal relative velocity of ego vehicle 200 and object 202, the lateral relative velocity of ego vehicle 200 and object 202, the longitudinal relative acceleration of ego vehicle 200 and object 202, the lateral relative acceleration of ego vehicle 200 and object 202; the longitudinal relative distance of ego vehicle 200 and object 203, the lateral relative distance of ego vehicle 200 and object 203, the longitudinal relative velocity of ego vehicle 200 and object 203, the lateral relative velocity of ego vehicle 200 and object 203, the longitudinal relative acceleration of ego vehicle 200 and object 203, the lateral relative acceleration of ego vehicle 200 and object 203; the longitudinal relative distance of ego vehicle 200 and object 204, the lateral relative distance of ego vehicle 200 and object 204, the longitudinal relative velocity of ego vehicle 200 and object 204, the lateral relative velocity of ego vehicle 200 and object 204, the longitudinal relative acceleration of ego vehicle 200 and object 204, the lateral relative acceleration of ego vehicle 200 and object 204; the longitudinal relative distance of ego vehicle 200 and object 205, the lateral relative distance of ego vehicle 200 and object 205, the longitudinal relative velocity of ego vehicle 200 and object 205, the lateral relative velocity of ego vehicle 200 and object 205, the longitudinal relative acceleration of ego vehicle 200 and object 205, the lateral relative acceleration of ego vehicle 200 and object 205; the longitudinal relative distance of ego vehicle 200 and object 206, the lateral relative distance of ego vehicle 200 and object 206, the longitudinal relative velocity of ego vehicle 200 and object 206, the lateral relative velocity of ego vehicle 200 and object 206, the longitudinal relative acceleration of ego vehicle 200 and object 206, The lateral relative acceleration between the vehicle 200 and the object 206; The longitudinal relative distance between the vehicle 200 and the object 207. The lateral relative distance between vehicle 200 and object 207. Let be the longitudinal relative velocity between the vehicle 200 and the object 207. Let be the lateral relative velocity between vehicle 200 and object 207. Let be the longitudinal relative acceleration between the vehicle 200 and the object 207. The lateral relative acceleration between the vehicle 200 and the object 207; The longitudinal relative distance between the vehicle 200 and the object 208. The lateral relative distance between vehicle 200 and object 208. Let be the longitudinal relative velocity between the vehicle 200 and the object 208. Let be the lateral relative velocity between the vehicle 200 and the object 208. Let be the longitudinal relative acceleration between the vehicle 200 and the object 208. The lateral relative acceleration is between the vehicle 200 and the object 208. Images of the vehicle's driving environment acquired via camera. , To obtain road slope and curvature information through in-vehicle maps, To obtain facial images of the driver using cockpit cameras.

[0068] Thus, the vehicle motion state data s recorded at any given moment for vehicle 200 is:

[0069]

[0070] Optionally, the operational status data includes driver status data; the driver status data includes one or more of the following: facial orientation, eye status, mouth opening and closing frequency and degree.

[0071] Optionally, the face direction is identified by a camera inside the cockpit, and the angle between the driver's face and the front direction is output. The eye state is calculated by an eye recognition algorithm, for example, PERCLOS (Percentage of Eyelid Closure over the Pupil over Time). PERCLOS refers to the percentage of time that the eyelid covers the pupil within a certain time period. The calculation formula is: PERCLOS = eyelid closure time / total observation time x 100%, which evaluates the degree of fatigue by calculating the percentage of the total observation time that the driver's eyes are closed. The mouth opening and closing frequency and degree are collected by a camera inside the vehicle cockpit. Face detection and facial feature point positioning are performed on each frame of image to obtain the key point coordinates of the mouth region, including the middle points of the upper and lower lips and the left and right corner points. Based on the mouth key point coordinates, the mouth aspect ratio (MAR) value is obtained, which is used as a real-time quantitative indicator of the degree of mouth opening. The larger the MAR value, the greater the degree of mouth opening. The MAR value is compared with the preset value to determine the degree of opening. The mouth opening and closing frequency is determined by setting a MAR threshold value to determine whether the mouth is open. When the MAR value rises from below the threshold value to above the threshold value and then falls below the threshold value again, it is counted as one complete mouth opening and closing action. The completion time of the action is recorded. In a sliding time window, for example, 60 seconds, the total number of completed opening and closing actions is counted, and the quotient of the total number of opening and closing actions and the time window is the mouth opening and closing frequency.

[0072] Optionally, the historical running state data sequence is obtained by: obtaining the running state data of the vehicle at a plurality of continuous sampling time points within a preset time window before the current time; and sorting the running state data at the plurality of continuous sampling time points according to time to form the historical running state data sequence.

[0073] In this embodiment, when the automatic driving system issues a takeover request, the running state data at the current time is collected in real time, i.e., the vehicle running parameters, environmental perception parameters and driver state at the current time are collected. In addition, the vehicle running parameters, environmental perception parameters and driver state at n sampling time points within a preset time window before the issuance of the takeover request are obtained. The preset time window can be customized according to the vehicle performance parameters. By introducing the historical running state data sequence, transient noise and accidental interference can be effectively filtered out, making the evaluation result more stable and reliable, and less susceptible to transient abnormal interference. In addition, it can capture the trend of state changes, providing the possibility for forward-looking safety decisions, and greatly improving the robustness and reliability of the evaluation of the driver's takeover ability.

[0074] Optionally, triggering the driving takeover request comprises at least one of the following conditions: the vehicle is about to or has exited a designed operating domain of the automatic driving system; the perception system of the vehicle detects that its performance degradation or information conflict exceeds a safety threshold; a predicted collision time based on the current state of the vehicle is below a safety threshold.

[0075] In this example, the designed operating domain refers to a specific condition range in which the automatic driving system is designed to work normally, including road types such as limited to highways, geographical areas such as cities, speed ranges, weather conditions such as sunny days, daytime, etc. For example, the vehicle is automatically driving on a highway, and the navigation prompts that the highway section ends 2 kilometers ahead. The system issues a takeover request at 1.5 kilometers from the exit, reminding the driver to prepare to take over.

[0076] The perception system of the vehicle includes, but is not limited to, sensor devices such as cameras, radars, lidars, etc. Performance degradation refers to the performance degradation of sensors due to reasons such as dirt, bad weather (e.g. heavy rain, heavy fog), etc. Information conflict refers to the inconsistency of the perception results of different sensors on the same object, and the system cannot make a reliable judgment. By setting a self-checking program for the perception system. When the system detects that a key sensor is disabled (such as a camera being blocked), or the perception result calculated by the fusion algorithm has a confidence level below a safety threshold, or the radar feedbacks that there is an obstacle in front, while the camera feedbacks that there is no obstacle in front and cannot be arbitrated, the system cannot drive safely and must request human intervention. For example: heavy rain causes the front camera to have a blurred view, and the system detects that the camera confidence level has dropped sharply, and immediately issues a takeover request.

[0077] The predicted collision time based on the current state of the vehicle being below a safety threshold means that the system calculates the collision time based on the relative speed and distance between the vehicle and the obstacle in front; the collision time is below a preset safety threshold, which is greater than the trigger threshold of the emergency braking system, to reserve reaction time for the driver's operation. In this case, it means that even if the automatic driving system operates normally, the risk is already too high, and the driver needs to intervene or prepare to respond to the possible collision.

[0078] S102, input the vehicle running state data and the historical running state data sequence into a pre-trained driver takeover capability judgment model, the driver takeover capability judgment model is configured to map the prediction result for characterizing the driver takeover capability based on the input data.

[0079] The current running state data and the historical running state data sequence before the current time are synchronously input into the pre-trained driver takeover ability judgment model to predict the takeover ability of the driver. That is, by combining the running state data in the continuous time period and the pre-trained driver takeover ability judgment model, the accuracy and reliability of the prediction of the takeover ability of the driver can be greatly improved.

[0080] S103, if the prediction result is that the takeover can be safely performed, the vehicle is taken over by the driver.

[0081] S104, if the prediction result is that the takeover cannot be safely performed, the vehicle is taken over by the safety driving system.

[0082] The safe takeover refers to that the driver can successfully take over the vehicle and keep the vehicle in a safe driving state, and the unsafe takeover refers to that the driver cannot take over the vehicle or the vehicle will be in an abnormal state after the driver takes over the vehicle.

[0083] Optionally, the taking over of the vehicle by the safety driving system comprises: obtaining an acceleration control instruction and a steering angle control instruction based on a model predictive control algorithm and in combination with constraint conditions, the constraint conditions comprising: a safe limit value of a vehicle longitudinal acceleration and a vehicle lateral acceleration, a safe distance constraint from surrounding obstacles, and a vehicle dynamics model constraint; and controlling the vehicle to operate according to the acceleration control instruction and the steering angle control instruction.

[0084] In this embodiment, if the prediction result is that the takeover can be safely performed, the vehicle is taken over by the driver, and the safety driving system does not control the vehicle, that is, the control instruction of the safety driving system is , as shown in formula (1). If the prediction result is that the takeover cannot be safely performed, the vehicle is taken over by the safety driving system, and the safety driving system controls the vehicle, that is, the control instruction of the safety driving system is as shown in formula (2), and the safety driving system exits the control of the vehicle if the vehicle exits the risk and enters a safe state or enters an automatic driving state or a manual driving state.

[0085] Formula (1)

[0086] In formula (1), u c is an acceleration control instruction of the vehicle at the next time, is a tire steering angle control instruction of the vehicle at the next time, is a state in which the driver can safely take over the vehicle, and the safety driving system does not control the vehicle.

[0087] Formula (2)

[0088] In formula (2), u c is the acceleration control instruction u c= of the vehicle at the next time point, u (1) is obtained through formula (3), u is the tire angle control instruction of the vehicle at the next time point, and u (1) is obtained through formula (3).

[0089]

[0090] Formula (3)

[0091] In formula (3), Δt is a sampling time interval, u(t) is an acceleration control instruction at a future tth sampling time, tn is a sampling number at a future time point, u(0) is a current acceleration, u(1) is an acceleration control instruction at a next sampling time, φ(t) is a tire angle control instruction at a future tth sampling time, φ(0) is a current tire angle, φ(1) is a tire angle at a next sampling time, v x is a vehicle longitudinal driving speed, v y is a vehicle lateral driving speed, θ(t) is a vehicle driving heading angle, θ(t) is a heading angle speed, m c is a vehicle mass, I c is a vehicle moment of inertia, k f is a front axle tire cornering stiffness, k r is a rear axle tire cornering stiffness, l f is a front axle to mass center distance, l r is a rear axle to mass center distance, μ is a ground adhesion coefficient, k u is an acceleration control instruction weighting coefficient, k φ is a tire angle control instruction weighting coefficient, a xminsafe is a longitudinal acceleration lower limit, a xmaxsafe is a longitudinal acceleration upper limit, a yminsafe is a lateral acceleration lower limit, a ymaxsafe is a lateral acceleration upper limit, c1 and c2 are constants greater than zero, and objn is a number of surrounding obstacles. j safe is a safety distance between the ego vehicle and the jth obstacle.

[0092] Formula (3) integrates the constraint condition into the objective function J through the Lagrange operator to form a new Lagrange function, simplifies the optimization process of the optimization objective in formula (3), and combines the pseudo-spectral method to realize the optimal numerical solution of the new Lagrange function, and calculates the acceleration control instruction u c and the steering wheel angle control instruction φ c, to minimize the collision risk with obstacles while guaranteeing smooth control under the premise of satisfying vehicle dynamics constraints and road constraints.

[0093] The present disclosure adopts a model predictive control (MPC) framework, which converts the control optimization problem into a quadratic programming (QP) problem: minimizing a quadratic objective function while satisfying a series of linear constraints.

[0094] In formula (3), the is a control amount penalty term, which aims to prevent the acceleration and steering angle from being too large, to ensure smoothness, and to avoid sudden acceleration, sudden braking, or sharp steering. is a control increment penalty term, which aims to control the smoothness of the control command at the adjacent time, to avoid shaking, and to improve comfort.

[0095] In formula (3), the formula in the curly braces is a constraint condition, where the first five formulas v x (t), v y (t), v x (t) are the state prediction equations derived from vehicle dynamics, which describe the state of the vehicle at future time. y (t) is the predicted longitudinal velocity at the next time, v c (t) is the predicted lateral velocity at the next time, c (t) is the predicted heading angle velocity at the next time, and X(t) and Y(t) are the predicted positions of the vehicle in the system coordinate system at the next time. Through these formulas, the trajectory caused by the acceleration control command u j and the steering wheel angle control command φ safe can be predicted.

[0096] In formula (3), the four inequalities below the first five formulas are vehicle physical limit constraints, which are longitudinal acceleration limit, lateral acceleration limit, and combined acceleration limit, respectively. The longitudinal acceleration limit is to avoid excessive acceleration or deceleration, which exceeds the passenger comfort or vehicle performance limit. The lateral acceleration limit is to avoid excessive lateral force when cornering, which causes the vehicle to lose stability. The combined acceleration limit is to ensure that the total force between the vehicle tire and the ground does not exceed the maximum adhesion force (μ·g), to prevent the vehicle from skidding.

[0097] In formula (3), the next two inequalities and are constraints on the rate of change of the control command. The purpose is that the difference between the acceleration command at the next time and the current acceleration cannot be too large, to ensure smooth response of the actuator. The role of the difference between the steering angle command of the next moment and the current steering angle cannot be too large. Through the two constraint limits, the vehicle rollover is prevented.

[0098] In formula (3), the role of the last formula is the safety distance constraint. The predicted future trajectory of the vehicle must maintain at least d j safe safe distance with the predicted future trajectory of the jthobstacle to achieve active collision avoidance.

[0099] The acceleration control command u c and the steering wheel angle control command φ c are sent to the drive-by-wire execution system through the vehicle network, and the drive-by-wire execution system includes a steer-by-wire system, a drive-by-wire system, and a brake-by-wire system, to realize active safety control of the vehicle until the vehicle exits the dangerous state, and then returns the control right of the vehicle to the driver or restores the automatic driving. The vehicle exits the dangerous state includes successfully changing lanes to avoid obstacles and stably decelerating to stop.

[0100] In some embodiments, as shown in Figure 3 , the step of constructing the driver takeover ability judgment model includes:

[0101] S301, simulating a plurality of automatic driving scenarios by the driver under a plurality of test working conditions;

[0102] Optionally, various driving scenarios, especially emergency and long tail scenarios, are covered through in-loop simulation testing, driver-in-loop bench testing, or driver real vehicle site testing, real vehicle road testing, real vehicle road operation, etc. The long tail scenario in automatic driving refers to an extreme traffic situation with low occurrence probability but high risk and difficulty to predict, such as an abnormal-shaped vehicle, extreme weather, and sudden obstruction, etc.

[0103] S302, obtaining the running state data at the current moment when the takeover request is triggered;

[0104] S303, obtaining a historical running state data sequence at a plurality of continuous sampling moments before the current moment;

[0105] The obtaining method and specific data type of the running state data at the current moment and the historical running state data sequence are the same as the content in the foregoing step S101, and will not be repeated here.

[0106] S304, obtaining the real takeover result in response to the takeover request;

[0107] Optionally, the real takeover result responsive to the takeover request is obtained, including: responsive to the takeover request, if the driver successfully takes over the vehicle and keeps the vehicle in a safe driving state, marking the real takeover result as capable of safe takeover; if the driver does not take over the vehicle or after the driver takes over the vehicle, the vehicle is in a collision, lane deviation or instability situation, marking the real takeover result as incapable of safe takeover.

[0108] By giving a clear label definition to the real takeover result in the model training process, the model can clearly understand which input data corresponds to a successful and safe takeover and which data corresponds to a failed and dangerous takeover during training.

[0109] Specifically, a binary label is marked for the data of each takeover event. The takeover event corresponding to the driver successfully taking over and the vehicle keeping in a safe driving state is marked as +1, i.e. capable of safe takeover. The takeover event corresponding to any of the following conditions is marked as -1, i.e. incapable of safe takeover: the driver does not take over, a collision occurs after takeover, lane deviation occurs after takeover, and the vehicle is unstable after takeover. By binary labeling, the result can be based on objective and observable results rather than subjective inference, and only the success and safety of the takeover result are concerned, thereby improving the accuracy of the model prediction result and the safety controllability of the targeted result control.

[0110] Moreover, when training the neural network, the algorithm will continuously adjust the internal parameters to approach the real label for each input data, i.e. the vehicle, environment and driver state sequence at the takeover moment and before. In this way, the clear label provides a prepared target for model learning, which can shape a clear decision boundary in a complex high-dimensional space, thereby minimizing the ambiguous prediction area between yes and no, so that the correct control strategy can be triggered faster in actual application.

[0111] In this way, the model can learn more deep and essential feature correlations rather than just superficial behavior correlations. By defining the decision boundary, the model can clearly divide the safe and dangerous decision boundary in the feature space, which is crucial for the subsequent model to make a high-confidence judgment in actual application.

[0112] S305, the data obtained by multiple tests is arranged into a data pair including running state data, historical running state data sequence and actual takeover result, to form an original data set.

[0113] By repeatedly conducting extensive tests across multiple scenarios and with different drivers, recording operational status data, historical operational status data sequences, and actual takeover results during the tests, data pairs are formed for each test, constituting the original dataset. This approach covers diverse driving environments, such as highways, cities, traffic congestion, and inclement weather; different takeover triggers, such as system malfunctions, road construction, and emergency obstacle avoidance; and different types of drivers, such as age, driving experience, and reaction speed. This ensures the diversity and representativeness of the dataset, improves the model's generalization ability, and guarantees its robustness in complex and ever-changing real-world applications.

[0114] S306, Construct an initial model for judging the driver's ability to take over.

[0115] The initial model includes an input layer and an output layer. The input layer is used to input operating status data and historical operating status data sequences, and the output layer is used to output the predicted takeover result, which includes whether the driver can take over safely or not.

[0116] Optionally, the formula for the initial model is as follows: ;in, The takeover result in response to the takeover request; f is a mapping function or neural network; s0 is the current running state data; s1 is the historical running state data at the first sampling point before the takeover request was issued; s n This refers to the historical operational status data at the nth sampling point before the takeover request was issued.

[0117] That is, combining Figure 2 The scenario example shown corresponds to the runtime status data s0 at the current moment when the takeover request was issued. ：

[0118]

[0119] The vehicle motion state s at the nth sampling point before the time the takeover request was issued. n for:

[0120]

[0121] Initialize the model as follows Figure 4 As shown, inpt i Let be the i-th input to the network, i∈[1,n0], where n0 is s0,s1,s2,...,s t The number of state information items in the composition, inpt i For [s0,s1,s2,...,s] t The i-th piece of information, Indicates information data inpt inormalized, if inpt i is distance information, inpt i is inverse and then normalized; f i,j is the function of the i-th layer j-th neuron, which can be selected as sigmoid or ReLu function, f i,j (x) is the function output of the i-th layer j-th neuron, is the function output of the i-1-th layer j-th neuron, W i-1,j,k is the weight of the i-1-th layer j-th neuron output, b i,j is the offset in the i-th layer j-th neuron; f oupt is the output layer function, which can be selected as tanh function or other functions, y out is the output layer function output, w o,i is the weight of the k-th layer k-th neuron output, b o is the offset in the output layer function, y out =1 is safe takeover, y out =-1 is safe takeover failure.

[0122] The following formula (4) is used to obtain the parameters in the initial model.

[0123] Formula (4)

[0124] In formula (14), y m is the driver safe takeover result of the m-th test experiment, y m =1 indicates that the driver successfully takes over, y m =-1 indicates that the driver fails to take over; s 0,m is the motion state of the vehicle when the autonomous driving system prompts and requests the driver to take over in the m-th experiment, s 1,m is the motion state of the vehicle at the first sampling time before the autonomous driving system prompts and requests the driver to take over in the m-th experiment, s 2,m is the motion state of the vehicle at the second sampling time before the autonomous driving system prompts and requests the driver to take over in the m-th experiment, s n,m is the motion state of the vehicle at the n-th sampling time before the autonomous driving system prompts and requests the driver to take over in the m-th experiment; w 1,1,1 , w 1,1,2 ,..., w 1,1,n1 , b 1,1 ; w 1,2,1 , w 1,2,2 ,..., w 1,2,n2 , b 1,2 ;..., w 1,i,1 , w 1,i,2 ,..., w1,i,ni ..., w o,1 , w o,2 ..., w o,nk , b o are model parameters in the initial model.

[0125] S307, dividing the original data set into a training set, a validation set and a test set;

[0126] S308, using the training set, iteratively optimizing the model parameters of the initial model;

[0127] Optionally, using the training set, iteratively optimizing the model parameters of the initial model, comprising: using the training set, using a back propagation algorithm and an optimization algorithm, taking the mean square error between the predicted takeover result of the model and the real takeover result as a loss function, training the initial model.

[0128] In this embodiment, forward propagation is performed, specifically, a batch of sample data is taken out from the training set, for example, 32 or 128 data pairs, the state sequence of the batch of data , is input into the deep neural network model of the current state. The data starts from the input layer, passes through the hidden layer layer by layer, each layer performs weighted summation and applies an activation function such as a ReLU function, and finally reaches the output layer to obtain the predicted value y out of all samples in the batch. out batch Then loss calculation is performed, and the predicted value y out batch output by the model is compared with the real label Y batchThe comparison is performed. The mean-square error (MSE) is used as the loss function for calculation, and the loss value is obtained. The MSE loss function amplifies the influence of large errors, forcing the model to preferentially correct samples with large prediction deviations. The calculated loss value is a scalar representing the average prediction error of the current model on this batch of data. Then, backpropagation is performed, and the gradients of the loss function with respect to the weights W and biases b of each layer in the model are calculated according to the calculated loss value Loss by the chain rule. The output obtains the gradient values corresponding to all parameters. Using the calculated gradients, the Adam optimizer is used to update the model parameters. The Adam optimizer automatically adjusts the effective learning step for each parameter, making the training process converge faster. Repeat the above steps until all samples in the training set have been learned by the model. Traversing the entire training set is called a cycle. The entire training process includes hundreds or even thousands of cycles. In each cycle, the training set is shuffled and divided into multiple batches for training, which helps to improve the model's generalization ability and training stability. Specifically, the training cycle can be specifically limited according to the actual test data, which is not limited here.

[0129] S309, in the training process, the validation set is used to monitor the performance of the model and control overfitting;

[0130] During training, after one or several cycles, the training is paused, and the current model is evaluated on the validation set. The same MSE loss function as in training is used to calculate the loss value of the model on the validation set. In the early stages of training, both the training set loss and the validation set loss will continue to decrease. As training progresses, if the training set loss continues to decrease, but the validation set loss begins to stabilize or even rebound, it indicates that the model has begun to overfit, that is, it has over-learned the noise and details of the training set, and has lost its generalization ability. When overfitting occurs, training is immediately stopped. And roll back to the model parameters of the cycle with the lowest validation set loss, which is used as the best model.

[0131] S310, the performance of the trained model is evaluated using the test set, and the model parameters are locked to obtain the driver takeover ability judgment model under the condition that the predetermined performance condition is met.

[0132] Before the training process is completely finished and the final parameters are locked, the optimal model parameters selected from the validation set are loaded. Test set data is input into the model for forward propagation. The final performance metrics of the model on the test set are calculated, including but not limited to: final MSE loss, accuracy, precision, recall, F1 score, etc. When the model's performance metrics on the test set meet predetermined performance conditions, such as MSE loss below a set loss threshold or accuracy above a set precision threshold, the model structure and all corresponding weights and bias parameters are locked, resulting in the driver takeover capability assessment model.

[0133] Taking emergency collision avoidance scenarios at high speeds as an example, such as Figures 5 to 8 As shown: Figure 5 As shown, the vehicle is driving in automatic mode on a two-lane highway. At 5 seconds, it detects an accident vehicle 30 meters ahead and requests driver intervention. The safety driving system determines that the driver cannot safely take over and initiates safety control. While ensuring a collision with the vehicle on the right, it changes lanes to avoid the accident vehicle ahead. The vehicle's operating mode is as follows. Figure 6 As shown, status=1 indicates automatic driving mode, status=0 indicates manual driving mode, status=0.5 indicates the automatic driving system requests driver takeover, status=1.5 indicates the driver has initiated automatic driving mode, and status=2 indicates the safety driving system is controlling the vehicle in safety driving mode. During vehicle movement, the longitudinal speed and steering wheel angle are as follows... Figure 7 As shown. The vehicle's travel path is as follows. Figure 8 As shown.

[0134] The present disclosure provides a control method for automatic driving takeover. According to a driver-in-the-loop simulation test, a driver-in-the-loop bench test, or a driver real vehicle field test, a real vehicle road test, a real vehicle road operation, and the like, the running state data of an autonomous vehicle when the autonomous driving system prompts and requests the driver to take over, the running state data of the autonomous vehicle at multiple time points within a preset window before the autonomous driving system prompts and requests the driver to take over, and the result of whether the driver successfully and safely takes over the vehicle are recorded. A driver takeover capability judgment model is designed and constructed. Then, according to all the data and model characteristics, the driver safe takeover judgment model parameters are determined through model parameter identification, and the model of whether the driver can complete safe takeover is established. During the operation of the autonomous driving system, whether the driver can safely take over the vehicle when the autonomous driving system sends a takeover request is determined by using the pre-trained driver takeover capability judgment model. If the driver can safely take over the vehicle, the safe driving system does not work. If the driver cannot safely take over the vehicle, the safe driving system controls the vehicle safely. In this way, the control method provided by the present disclosure can improve the accurate judgment of the driver's takeover capability, and according to the judgment result, the corresponding control operation is given, thereby reducing the accident rate of the autonomous driving system in emergency or long tail scenarios and improving the running safety of the vehicle.

[0135] In some embodiments, in combination Figure 9 As shown in FIG. 9, a control device 90 for automatic driving takeover is provided, which includes:

[0136] The data acquisition module 910 is configured to acquire the running state data at the current time and the historical running state data sequence within a preset time window when the vehicle triggers a driving takeover request during automatic driving.

[0137] The takeover judgment module 920 is configured to input the vehicle running state data and the historical running state data sequence into a pre-trained driver takeover capability judgment model. The driver takeover capability judgment model is configured to map the prediction result for characterizing the driver's takeover capability based on the input data. The prediction result includes safe takeover or unsafe takeover.

[0138] The safety control module 930 is configured to take over the vehicle by the driver when the prediction result is safe takeover, which means that the driver can successfully take over the vehicle and keep the vehicle in a safe driving state. In addition, the safety control module 930 is configured to trigger the safety driving system to take over the vehicle when the prediction result is unsafe takeover, which means that the driver cannot take over the vehicle or the vehicle will be in an abnormal state after the driver takes over the vehicle.

[0139] In some embodiments, in combinationFigure 10 As shown, an intelligent driving system 100 is provided, comprising a safe driving system 110, the safe driving system comprising an automatic driving takeover control device as described in any of the above embodiments, and the safe driving system being configured to control the vehicle to run in the case that the takeover judgment result is that the takeover cannot be safely performed. An automatic driving system 120 is configured to control the vehicle to perform automatic driving, and to send a driving takeover request to the control device.

[0140] The intelligent driving system provided by the present disclosure comprises a safe driving system independent of the automatic driving system. The automatic driving system is a conventional automatic driving system in an intelligent driving vehicle, and can realize automatic driving of the vehicle. The automatic driving system can send a driving takeover request to hand over the driving right to the driver in the case that a condition satisfying the trigger takeover request occurs during automatic driving. In the related art, the takeover level is determined according to the current state of the driver and the current state of the vehicle, and a takeover prompt is given to remind the driver to take over. Or in the case that the driver does not respond, the automatic driving system still takes over.

[0141] In the present application, by setting a safe driving system independent of the automatic driving system, in the case that a takeover request is monitored, the running state data at the current time and the historical running state data sequence within a preset time window before the current time are obtained, and the running state data at the current time and the historical running state data sequence are input into a pre-trained driver takeover capability judgment model together, to predict whether the driver can take over the vehicle. When the prediction result output by the driver takeover capability judgment model is that the takeover can be safely performed, the control right of the vehicle is handed over to the driver. When the prediction result output by the driver takeover capability judgment model is that the takeover cannot be safely performed, the safe driving system is immediately activated, and the safe driving system takes over the vehicle. Compared with the emergency collision avoidance control system in the prior art, the safe driving system provided by the present disclosure makes a judgment on the takeover capability of the driver as soon as the automatic driving system sends a takeover request, and can take over the control of the vehicle in the case that the prediction result is that the takeover cannot be safely performed. In this way, in the case that the driver cannot take over, the safe driving system intervenes in the control of the vehicle in advance, fully utilizes the time before the accident collision, controls the vehicle, leaves more sufficient time for active collision avoidance control, and reduces the accident rate. That is, the present disclosure can start safe takeover control of the vehicle before the vehicle running parameters meet the trigger condition of the emergency collision avoidance control system. Moreover, the intervention of the safe driving system is not emergency braking, but MPC-based cooperative safety control, which fully utilizes the road space, realizes a smoother, safer and more intelligent obstacle avoidance strategy, avoids secondary hazards caused by single sharp braking or turning, and significantly improves the safety and reliability of the automatic driving vehicle in emergency working conditions,

[0142] In combination Figure 11 As shown in FIG. 11, the control device 1100 for automatic driving takeover provided by the embodiments of the present disclosure includes a processor 1102 and a memory 1104. Optionally, the device 1100 can further include a communication interface 1106 and a bus 1108. The processor 1102, the communication interface 1106 and the memory 1104 can communicate with each other through the bus 1108. The communication interface 1106 can be used for information transmission. The processor 1102 can invoke the logic instructions in the memory 1104 to execute the control method for automatic driving takeover of the above-mentioned embodiments.

[0143] In addition, when the logic instructions in the memory 1104 are implemented in the form of a software functional unit and sold or used as an independent product, they can be stored in a computer-readable storage medium.

[0144] The memory 1104 as a computer-readable storage medium can be used to store software programs, computer executable programs, such as program instructions / modules corresponding to the method in the embodiments of the present disclosure. The processor 100 executes the function application and data processing by running the program instructions / modules stored in the memory 1104, that is, implements the control method for automatic driving takeover in the above-mentioned embodiments.

[0145] The memory 1104 can include a program storage area and a data storage area. The program storage area can store an operating system and application programs required by at least one function; the data storage area can store data created during use of the terminal device, etc. In addition, the memory 1104 can include a high-speed random access memory, and can also include a non-volatile memory.

[0146] In some embodiments, a vehicle is provided, including: a vehicle body; and the control device 90 (1100) for automatic driving takeover or the intelligent driving system 100 as described in any of the above-mentioned embodiments, which is installed on the vehicle body.

[0147] The installation relationship described herein is not limited to being placed inside the vehicle body, but also includes installation connection with other components of the vehicle, including but not limited to physical connection, electrical connection or signal transmission connection, etc. Those skilled in the art can understand that the control device 90 (1100) for automatic driving takeover can be adapted to a feasible vehicle body, and thus other feasible embodiments can be realized.

[0148] The embodiment of the present disclosure provides a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are configured to execute the automatic driving takeover control method.

[0149] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes one or more instructions to make a computer device (which can be a personal computer, a server, or a network device) execute all or part of the steps of the method disclosed in the embodiment of the present disclosure. The foregoing storage medium can be a non-transitory storage medium, for example, a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0150] The above description and drawings sufficiently illustrate the embodiments of the present disclosure to enable one skilled in the art to practice them. Other embodiments can include structural, logical, electrical, process, and other changes. The embodiments represent only a few of the possible variations. Individual components and functions are optional unless explicitly required, and the order of operations can be changed. Parts and features of some embodiments can be included in or replace parts and features of other embodiments. Also, the words used in this application are only used to describe the embodiments and not to limit the claims. As used in the description of the embodiments and the claims, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise. Similarly, as used in this application, the term "and / or" refers to any and all possible combinations of one or more associated listed items. In addition, when used in this application, the term "comprise" and its variants "comprises" and / or comprises" and the like mean the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups of these. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, or device that includes the stated element. In this document, each embodiment focuses on the differences from other embodiments, and the same or similar parts between various embodiments can be referred to each other. For the method, product, etc. disclosed in the embodiments, if it corresponds to the method part disclosed in the embodiments, the relevant part can be referred to the description of the method part.

[0151] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods for each specific application to realize the described functions, but such implementation should not be considered beyond the scope of the embodiments of the present disclosure. The skilled person can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0152] In the embodiments disclosed herein, the disclosed methods, products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the above-described device embodiments are only schematic, for example, the division of the units can only be a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms. The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to implement the embodiments. In addition, each functional unit in the embodiments of the present disclosure can be integrated in one processing unit, or each unit can be a physically independent unit, or two or more units can be integrated in one unit.

[0153] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other processing device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other processing device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

Claims

1. A control method for automatic takeover, characterized by, The method comprises the following steps: In the process of automatic driving of the vehicle, when a driving takeover request is triggered, running state data at the current time and a historical running state data sequence within a preset time window are obtained; wherein, the historical running state data sequence is obtained by obtaining running state data of the vehicle at a plurality of continuous sampling times within the preset time window before the current time; and the running state data at the plurality of continuous sampling times is sorted according to time to form the historical running state data sequence; The vehicle running state data and the historical running state data sequence are input into a pre-trained driver takeover capability judgment model, which is configured to map a prediction result for representing the driver takeover capability based on the input data, the prediction result including safe takeover or unsafe takeover; wherein, the driver takeover capability judgment model is constructed by: simulating a plurality of automatic driving scenarios by the driver under a plurality of test conditions; obtaining running state data at the current time when a takeover request is triggered during each test; obtaining a historical running state data sequence at a plurality of continuous sampling times before the current time; obtaining a real takeover result in response to the takeover request; arranging the data obtained through the plurality of tests into a data pair including the running state data, the historical running state data sequence and the actual takeover result to form an original data set; constructing an initial model for driver takeover capability judgment, the initial model including an input layer and an output layer, the input layer being used for inputting the running state data and the historical running state data sequence, and the output layer being used for outputting a predicted takeover result, the predicted takeover result including safe takeover by the driver or unsafe takeover by the driver; training and testing the initial model using the original data set to obtain the driver takeover capability judgment model; When the prediction result is safe takeover, the driver takes over the vehicle, and safe takeover means that the driver can successfully take over the vehicle and keep the vehicle in a safe driving state; When the prediction result is unsafe takeover, a safe driving system takes over the vehicle, and unsafe takeover means that the driver cannot take over the vehicle or the vehicle will be in an abnormal state after the driver takes over the vehicle.

2. The control method according to claim 1, wherein: The running state data includes vehicle motion state data, environmental data and driver state data; The vehicle motion state data includes vehicle speed, acceleration, and the angle, angular velocity, angular acceleration of the vehicle body, and tire pressure; The environmental data includes weather, light intensity, road information and relative distance to surrounding objects; The driver state data includes one or more of the following: face direction, eye state, mouth opening frequency and opening degree.

3. The control method according to claim 1 or 2, characterized by, The real takeover result in response to the takeover request is obtained by: In response to the takeover request, if the driver successfully takes over the vehicle and keeps the vehicle in a safe driving state, the real takeover result is marked as safe takeover. If the driver does not take over the vehicle or the driver takes over the vehicle after the vehicle collision, lane deviation or instability, the real takeover result is marked as unable to safely take over.

4. The control method according to claim 3, characterized by The initial model is trained and tested using the original data set to obtain a driver takeover ability judgment model, including: Divide the original data set into a training set, a validation set and a test set; Use the training set to iteratively optimize the model parameters of the initial model; During the training process, the validation set is used for model performance monitoring and overfitting control; Use the test set to evaluate the performance of the trained model, and lock the model parameters to obtain the driver takeover ability judgment model under the condition that the predetermined performance condition is met.

5. The control method according to claim 4, characterized by Use the training set to iteratively optimize the model parameters of the initial model, including: Using the training set, using the back propagation algorithm and the optimization algorithm, taking the mean square error between the model's predicted takeover result and the real takeover result as the loss function, and training the initial model.

6. The control method of claim 1 or 2, wherein The formula of the initial model is as follows: ; wherein, a takeover result in response to the takeover request; f is a mapping function or a neural network; s0is the running state data at the current time; s1is the historical running state data at the first sampling point before the takeover request is issued; s2is the historical running state data at the second sampling point before the takeover request is issued; s n is the historical running state data at the nth sampling point before the takeover request is issued.

7. The control method according to claim 1 or 2, characterized by, Triggering the safety driving system to take over the vehicle, including: Based on the model predictive control algorithm, combined with the constraint condition, the acceleration control instruction and the steering angle control instruction are obtained, and the constraint condition includes: the safety limit value of the vehicle longitudinal acceleration and the lateral acceleration, the safety distance constraint with the surrounding obstacles, and the vehicle dynamics model constraint; According to the acceleration control instruction and the steering angle control instruction, control the vehicle to run.

8. The control method according to claim 1 or 2, characterized by, Triggering the driving takeover request includes at least one of the following conditions: The vehicle is about to leave or has left the design operating domain of its autonomous driving system; The vehicle's perception system detects that its performance degradation or information conflict exceeds the safety threshold; The collision time predicted based on the current state of the vehicle is less than the safety threshold.

9. A control device for automatic takeover, characterized in that Including: The data acquisition module is configured to obtain the running state data at the current time and the historical running state data sequence within a preset time window when the driving takeover request is triggered during the autonomous driving of the vehicle; wherein obtaining the historical running state data sequence includes: obtaining the running state data of the vehicle at multiple continuous sampling time points within a preset time window before the current time; the running state data at multiple continuous sampling time points is sorted according to time to form a historical running state data sequence; The takeover judgment module is configured to input the vehicle running state data and the historical running state data sequence into a pre-trained driver takeover ability judgment model, the driver takeover ability judgment model is configured to map out a prediction result for characterizing the driver takeover ability based on the input data, the prediction result includes being able to safely take over or being unable to safely take over; wherein the steps of constructing the driver takeover ability judgment model include: simulating a plurality of automatic driving scenes by the driver under a plurality of test working conditions; in each test process, obtaining the running state data at the current time under the condition that the takeover request is triggered; and obtaining the historical running state data sequence at a plurality of continuous sampling times before the current time; obtaining the real takeover result in response to the takeover request; arranging the data obtained by the plurality of tests into a data pair including the running state data, the historical running state data sequence and the actual takeover result, to form an original data set; constructing an initial model for driver takeover ability judgment, the initial model includes an input layer and an output layer, the input layer is used to input the running state data and the historical running state data sequence, and the output layer is used to output the predicted takeover result, the predicted takeover result includes that the driver can safely take over or the driver cannot safely take over; training and testing the initial model by using the original data set to obtain the driver takeover ability judgment model; The safety control module is configured to take over the vehicle by the driver if the prediction result is that the driver can safely take over, and the driver can safely take over means that the driver can successfully take over the vehicle and keep the vehicle in a safe driving state; and The safety control module is configured to trigger the safety driving system to take over the vehicle if the prediction result is that the driver cannot safely take over, and the driver cannot safely take over means that the driver cannot take over the vehicle or the vehicle will be in an abnormal state after the driver takes over the vehicle.

10. A control device for automatic takeover, comprising a processor and a memory having stored program instructions, characterized in that, The processor is configured to execute the automatic driving takeover control method according to any one of claims 1 to 8 when running the program instructions.

11. An intelligent driving system, characterized by, The safety driving system includes the automatic driving takeover control device according to claim 9 or 10, and the safety driving system is configured to control the vehicle to run if the takeover judgment result is that the driver cannot safely take over; The automatic driving system is configured to control the vehicle to automatically drive, and send a driving takeover request to the control device. The vehicle body; 12. A vehicle characterized by comprising: The automatic driving takeover control device according to claim 9 or 10 is installed on the vehicle body; Or The intelligent driving system according to claim 11 is installed on the vehicle body. The program instructions are used to make the computer execute the automatic driving takeover control method according to any one of claims 1 to 8 when running. ​ 13. A readable storage medium, storing program instructions, characterized in that, ​

Images