Emergency takeover triggering method and system based on environmental perception and driver state recognition

Abstract

The application discloses an emergency takeover triggering method and system based on environment perception and driver state recognition, relates to the technical field of emergency takeover, and comprises the following steps: acquiring environment perception data and driver state data, and respectively performing risk level evaluation; determining whether to initiate a takeover request according to the risk level; if the takeover request is initiated, performing instruction consistency detection based on game modeling, abstracting the driver and the automatic driving system as two intelligent agents, and respectively reasoning the operation intention of the two intelligent agents; if the detection result is inconsistent, entering a conflict strategy processing flow, evaluating the advantages and disadvantages of the strategies of the two parties according to the driver state confidence and the environment risk level, and outputting a game reasoning result; and the problems of lack of consistency judgment, high calculation complexity and difficulty in real-time response in the prior art are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of emergency takeover, and more particularly, to an emergency takeover triggering method and system based on environmental perception and driver state recognition. BACKGROUND

[0002] The emergency takeover mechanism in current autonomous driving systems mainly relies on the combination of environmental perception and driver state recognition technology to realize the switching of control right. When the system detects that it is about to enter a complex scene that cannot be safely coped with, such as obstacles approaching, severe weather, lane information loss, or system internal failure, a takeover request (TOR) will be triggered, and the driver will be reminded through voice prompts, image flashing, steering wheel vibration, etc.

[0003] In existing systems, the takeover process generally consists of three steps: 1) identifying risks; 2) issuing a takeover prompt; 3) monitoring whether the driver has performed a physical takeover operation, such as holding the steering wheel, stepping on the accelerator or brake, etc. Most systems only consider the driver to have taken over after a physical action occurs, and the takeover determination is mainly based on input behavior rather than a deep understanding of the driver's state or intent. The entire process is based on the assumption that the driver will correctly and timely cooperate with the takeover after receiving the prompt, and the takeover mechanism is more of a one-way triggering system with the driver responding. In the above disclosed technical solution, at least the following technical problems exist.

[0004] In current assisted driving systems, conflicts between drivers and systems often occur in emergency situations, i.e., the system has issued a risk warning based on environmental perception and prompted a takeover, but the driver chooses to defy the takeover prompt and forcefully operate due to nervousness, misjudgment, or lack of trust in the system, ultimately leading to serious accidents. This type of human-machine conflict exposes the lack of recognition and response mechanism for driver defiance behavior in existing systems, and it is difficult to achieve human-machine collaborative decision-making in high-pressure scenarios, increasing the risk of takeover failure and the severity of accidents.

[0005] In existing emergency takeover triggering methods, the takeover process is usually based on the comprehensive evaluation results of environmental risks and driver states by the system, and a fixed threshold is used to trigger the takeover instruction. However, in actual emergency scenarios, the driver may make operation behaviors that defy the system's takeover instruction due to stress response, cognitive bias, or emotional interference, or even take actions opposite to the system's instruction (such as the system suggesting to slow down to avoid danger, but the driver instead speeds up and turns), leading to human-machine instruction conflicts. If this conflict is not identified and handled in time, it can easily lead to operational errors and traffic accidents. Most current systems do not dynamically identify driver defiance behavior, and lack of evaluation of whether the driver's operation is consistent with the system's intent Figure OneThe takeover strategy lacks a hierarchical response and adjustment capability to cope with conflicts, resulting in insufficient reliability and safety of the system in emergency scenarios.

[0006] To solve the above problems, the application provides a solution. SUMMARY

[0007] In order to overcome the above-mentioned defects of the prior art, the embodiments of the application provide an emergency takeover triggering method and system based on environment perception and driver state recognition, which solves the problems of lack of consistency judgment, high computational complexity and difficulty in real-time response in the prior art by using an instruction consistency detection method based on agent game reasoning, combining a finite state machine to simplify the high-dimensional state space.

[0008] To achieve the above object, the application provides the following technical scheme:

[0009] The emergency takeover triggering method and system based on environment perception and driver state recognition comprises the following steps: acquiring environment perception data and driver state data, and respectively performing risk level evaluation; determining whether to initiate a takeover request according to the risk level; if the takeover request is sent, performing instruction consistency detection based on game modeling, abstracting the driver and the automatic driving system as two agents, and respectively reasoning the operation intention thereof; if the detection result is inconsistent, entering a conflict strategy processing flow, evaluating the advantages and disadvantages of the strategies of the two parties according to the driver state confidence and the environment risk level, and outputting a game reasoning result; the game reasoning simplifies the game model judgment through a finite state machine; and triggering the takeover according to the game reasoning result.

[0010] In a preferred embodiment, the acquisition of the environment perception data and the driver state data and the respective risk level evaluation are specifically: calculating an environment risk score based on the environment perception data by constructing a weighted scoring model; calculating a driver state risk score based on the driver state data; the weighted scoring model quantitatively processes various risk parameters according to a plurality of preset risk factors, and determines the risk level according to the comparison between the risk score and a set threshold.

[0011] In a preferred embodiment, the driver and the automatic driving system are abstracted as two agents, and the operation intention of each is inferred, specifically: the two agents include a driver agent and an automatic driving system agent; the agent is composed of an agent state space, a set of agent strategies, and a decision mechanism; the environment perception data and the driver state data are extracted and screened to obtain screened features; based on the screened features, the driver agent state space and the automatic driving system agent state space are constructed; both the driver agent state space and the automatic driving system agent state space are defined by a multi-dimensional vector space, and the vector elements are the screened driver state data feature value set and the environment perception data feature value set; for each agent state space, a set of selectable strategies is defined; the driver agent strategy set includes the operation intentions of actively taking over, delaying response, refusing to take over, and irrational intervention; the automatic driving system agent strategy set includes the control actions of continuous control, request for takeover, forced control, and emergency braking; the agent strategy set and the agent state space are mapped based on a state-strategy matching mapping model to obtain a decision mechanism; the consistency of the agent instructions is detected.

[0012] In a preferred embodiment, the screened features are as follows: a historical driving database containing multiple scenarios, multiple drivers, and multiple road conditions is constructed, and the database includes historical environment perception data and driver state data, and corresponding actual takeover event labels; based on real-time environment perception data and driver state data, a preliminary feature set is constructed; a preset database is used to apply a supervised learning algorithm to train the preliminary feature set, evaluate the contribution of each feature to the takeover event judgment, and output a feature importance score; based on the feature importance score and the actual calculation resource constraint, high-score features are screened out to form the first screened features; the first screened features are subjected to correlation analysis, and redundant features are removed to obtain the screened features.

[0013] In a preferred embodiment, the agent strategy set and the agent state space are mapped based on a state-strategy matching mapping model to obtain a decision mechanism, specifically: for each strategy, an ideal agent state space template is defined, representing the most reasonable applicable agent state space combination for the agent strategy; the distance between the current state space and each agent state space template is compared, and the nearest agent strategy is selected as the mapped agent strategy; the mapping relationship is used as the decision mechanism.

[0014] In a preferred embodiment, if the detection results are inconsistent, a conflict strategy processing flow is entered, the strategies of the two parties in the conflict are evaluated according to the driver state confidence and the environmental risk level, and a game reasoning result is output. Specifically, a driver state confidence model and an environmental perception risk level model are constructed, and a confidence score of the driver strategy and a safety priority score of the automatic driving system strategy are output respectively. The driver state confidence model uses a machine learning model to train historical driving data, the model learns the feature distribution of the driver in normal and abnormal states, real-time input of the current driver state feature is output, and the confidence score is output. The confidence score is a confidence score between 0 and 1, which represents the confidence of the driver state data and the rationality of the current intention of the driver to take over. The environmental perception risk level model maps multi-dimensional environmental data to a comprehensive risk score through a risk score function, and converts the comprehensive risk score to a value in the range of 0-1 through a normalization function as a safety priority score. The driver strategy and the automatic driving system strategy are evaluated by using a weight weighting method and compared with a threshold to evaluate the advantages and disadvantages.

[0015] In a preferred embodiment, the state space is composed of the screened multi-dimensional feature vector; a preset number of feature dimensions are screened based on feature importance analysis; the multi-dimensional feature vector is reduced to the preset number of feature dimensions based on principal component analysis to obtain a continuous low-dimensional state vector space; a second threshold is set according to the statistical distribution of the features, and the continuous low-dimensional state vector space is divided into several discontinuous intervals; each interval is taken as a finite state set, and several finite state sets are used to construct an agent state space for game reasoning.

[0016] The system of the emergency takeover triggering method based on environmental perception and driver state recognition includes a risk perception module, a takeover issuing module, a consistency detection and man-machine conflict processing module, a game simplification module, and a takeover module. The risk perception module is used to acquire environmental perception data and driver state data, and to perform risk level evaluation respectively. The takeover issuing module is used to determine whether to initiate a takeover request according to the risk level. The consistency detection and man-machine conflict processing module is used to perform instruction consistency detection based on game modeling if the takeover request is issued, abstract the driver and the automatic driving system as two agents, and reason their operation intentions respectively. If the detection results are inconsistent, a conflict strategy processing flow is entered, the strategies of the two parties in the conflict are evaluated according to the driver state confidence and the environmental risk level, and a game reasoning result is output. The game simplification module is used to simplify the game model for judgment by the game reasoning through a finite state machine. The takeover module is used to trigger the takeover according to the game reasoning result.

[0017] The application is based on the technical effects and advantages of an emergency takeover triggering method and system based on environment perception and driver state recognition.

[0018] 1. The application realizes dynamic judgment of the consistency of the operation intentions of the driver and the automatic driving system by abstracting the driver and the automatic driving system as two agents and performing game reasoning, effectively fills the gap of lack of instruction consistency detection in the prior art, improves the accuracy and safety of takeover decision, reduces the risk caused by the conflict between the driver and the system operation, and enhances the intelligence and reliability of emergency takeover.

[0019] 2. The application significantly reduces the computational complexity and real-time response delay of the algorithm by using a finite state machine to reasonably divide and reduce the high-dimensional complex state space, and combining a probability approximation method to simplify game reasoning calculation, realizing efficient application of the game model in real-time driving environment, ensuring that the emergency takeover system has high intelligent judgment ability and meets real-time and system performance requirements. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 The flowchart of the emergency takeover triggering method based on environment perception and driver state recognition of the application.

[0021] Figure 2 The structural diagram of the emergency takeover triggering system based on environment perception and driver state recognition of the application. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0023] Embodiment 1, Figure 1 The emergency takeover triggering method based on environment perception and driver state recognition of the application is given, including the following steps:

[0024] S1, obtain environment perception data and driver state data, and perform risk level evaluation respectively.

[0025] In this embodiment, environment perception data and driver state data are obtained, and risk level evaluation is performed respectively, specifically as follows:

[0026] An environment risk score is calculated based on the environment perception data by constructing a weighted scoring model;

[0027] A driver state risk score is calculated based on the driver state data;

[0028] The weighting score model quantitatively processes various risk parameters according to preset multi-dimensional risk factors, and determines a risk level according to comparison of a risk score with a set threshold.

[0029] The environment perception data includes vehicle driving speed, front obstacle type and relative position, lane line state, road construction sign, traffic flow density, light and weather conditions, and system drivable area range.

[0030] The driver state data includes line of sight direction, head posture, hand position, operation frequency, physiological characteristics (such as heart rate fluctuation and breathing rhythm), and historical operation response record.

[0031] It should be noted that the environment perception data includes but is not limited to: vehicle current driving speed, relative obstacle position, obstacle type (such as static obstacle, dynamic vehicle or construction equipment), drivable area boundary, lane line clarity, road sign state, traffic flow density, weather condition, and light condition, and the data dimension and collection frequency can be flexibly adjusted according to sensor type, calculation capacity and task demand.

[0032] It should be noted that the driver state data includes but is not limited to: line of sight direction, head posture angle, hand position and stability, steering wheel interaction behavior (such as holding degree and time), operation rhythm, posture change frequency, and physiological parameters (such as heart rate and skin electric response) that can reflect attention and control ability of the driver, and the specific feature dimension can be dynamically cut and expanded according to target scene and monitoring means.

[0033] It should be noted that the preset multi-dimensional risk factors quantitatively process various risk parameters, and determine whether the takeover trigger condition is reached according to the comparison result of the risk score and the set threshold. In the prior art, there are mature applications, for example, in the existing advanced driver assistance system (ADAS) and part of the automatic driving system, obstacle distance, relative speed, lane line recognition quality, road structure complexity, driver gaze time and hand state are generally used as basic risk factors, and risk quantification is performed through weighting score or rule engine model, and takeover trigger judgment is performed according to the set threshold.

[0034] It should be noted that any method for multi-dimensional analysis and judgment of takeover trigger can be applied to the present application, the method is not limited to the weighting score model, and also includes fuzzy logic reasoning method, Bayesian network model, support vector machine model, lightweight neural network model and other algorithm frameworks that can perform joint risk modeling and level judgment on multi-source perception data and driver state data. The specific method to be used can be selected according to the calculation resource of the deployment platform, real-time requirement and system adaptability.

[0035] S2, determining whether to initiate a takeover request according to the risk level.

[0036] In this embodiment, whether to initiate a takeover request is determined according to the risk level, specifically:

[0037] When the environmental risk score exceeds the environmental risk setting threshold, or the driver state risk score exceeds the driver state risk setting threshold, the system triggers a takeover request; the environmental risk setting threshold and the driver state risk setting threshold can be preset according to different driving modes, road levels or traffic states, and dynamic fine-tuning is supported according to historical data to realize early response to potential high-risk scenarios.

[0038] In this embodiment, as described above, by constructing a weighted scoring model, combining multi-dimensional environmental perception data and driver state data, accurate quantification and evaluation of environmental risk and driver state risk are realized, and based on the set risk threshold, it is determined whether to initiate a takeover request. This method not only effectively integrates multi-source perception information, but also supports flexible adjustment of risk threshold to adapt to different driving scenarios and real-time state changes. Based on this takeover triggering mechanism, subsequent further introduction of instruction consistency detection based on game modeling abstracts the driver and the automatic driving system as agents, and through simplified game reasoning of finite state machine, consistency judgment of human-machine operation intention is realized, thereby improving the safety and reliability of emergency takeover.

[0039] S3, if a takeover request is issued, instruction consistency detection is performed based on game modeling, the driver and the automatic driving system are abstracted as two agents, and their operation intentions are reasoned respectively; if the detection result is inconsistent, a conflict strategy processing flow is entered, the driver state confidence and the environmental risk level are evaluated to assess the pros and cons of the strategies of the two parties in conflict, and a game reasoning result is output.

[0040] In this embodiment, the driver and the automatic driving system are abstracted as two agents, specifically:

[0041] The two agents include a driver agent and an automatic driving system agent;

[0042] The agent is composed of an agent state space, an agent policy set, and a decision mechanism;

[0043] The environmental perception data and the driver state data are feature extracted and screened to obtain screened features;

[0044] Based on the screened features, a driver agent state space and an automatic driving system agent state space are constructed respectively;

[0045] The driver agent state space and the autonomous driving system agent state space are both defined by a multi-dimensional vector space, and the vector elements are a set of filtered driver state data feature values and a set of environmental perception data feature values, respectively.

[0046] For each agent state space, a set of optional strategies is defined.

[0047] The driver agent strategy set includes the operation intentions of active takeover, delayed response, refusal to take over, and irrational intervention.

[0048] The autonomous driving system agent strategy set includes the control actions of continuous control, request for takeover, forced control, and emergency braking.

[0049] The agent strategy set and the agent state space are mapped based on the SSMM (State-Strategy Matching Mapping Model) to obtain a decision mechanism.

[0050] It should be noted that the autonomous driving system agent state space reflects the current environmental awareness and control strategy execution state of the system.

[0051] It should be noted that the driver agent state space reflects the driver's current attention concentration, operation intention strength, and emergency response ability.

[0052] Further, the environmental perception data and the driver state data are feature extracted and filtered to obtain filtered features, specifically:

[0053] A historical driving database containing multiple scenarios, multiple drivers, and multiple road conditions is constructed, which includes historical environmental perception data and driver state data, and corresponding actual takeover event labels.

[0054] Based on real-time environmental perception data and driver state data, a preliminary feature set is constructed.

[0055] Using a preset database, a supervised learning algorithm is applied to train the preliminary feature set, evaluate the contribution of each feature to the takeover event discrimination, and output a feature importance score.

[0056] According to the feature importance score and the actual computing resource constraint, high-score features are filtered out to form the first filtered features.

[0057] The first filtered features are analyzed for correlation, and redundant features are removed to obtain filtered features.

[0058] It should be noted that the data in the historical driving database is standardized to ensure data consistency and comparability.

[0059] It should be noted that the feature importance score can be calculated based on model sensitivity analysis or feature contribution evaluation mechanism; specifically, the feature importance score can be obtained by applying supervised models such as gradient boosting decision tree (GBDT), random forest (RF) or shallow neural network, recording the frequency, information gain or weight change of each feature in the splitting decision during the training process, to quantify the contribution of each feature in judging the occurrence of the takeover event; At the same time, the model-agnostic explanation framework based on perturbation method (such as SHAP, LIME) can also be used to evaluate the impact of feature changes on the prediction output, so as to obtain an explanatory index for the actual scene.

[0060] It should be noted that the data in the historical driving database is standardized to ensure consistency and comparability of the data. In order to ensure that data collected from different sources, different time periods and different devices can be used for training and analysis in the same model, the historical driving database is standardized during construction. The original data includes normalization of numerical features (such as maximum and minimum scaling, Z-score standardization), consistent encoding of classification features (such as one-hot encoding, label mapping), and timestamp alignment processing, to ensure uniformity of input data dimensions and consistency of scales, thereby improving the training stability and result comparability of the supervised learning algorithm.

[0061] It should be noted that the supervised learning algorithm can be flexibly selected according to the real-time performance and computing resources of the target system, and a lightweight and highly interpretable model is preferred. In this embodiment, the supervised learning algorithm can be implemented using classical models such as random forest (RandomForest), support vector machine (SVM), logistic regression (Logistic Regression); for embedded platforms or low-power computing platforms, gradient boosting algorithms (such as LightGBM, XGBoost) with low latency and high robustness are preferred; in terminal platforms where lightweight neural networks can be deployed, shallow convolutional neural networks or multi-layer perceptron networks can also be introduced. Model selection can be dynamically optimized in combination with feature types, data distribution, real-time requirements and platform computing power. Correlation analysis to eliminate redundant features can be based on correlation coefficient matrix or information redundancy metric.

[0062] It should be noted that, in order to avoid model overfitting or waste of computing resources caused by redundant features, when processing the first screening features, the correlation between the features can be analyzed based on Pearson correlation coefficient, Spearman rank correlation or mutual information (Mutual Information), and the information dimensions with high linear correlation or high repetitive expression are identified and processed by merging, replacing or deleting, to finally obtain a set of refined features with information independence and strong discrimination ability.

[0063] It should be noted that the feature screening process not only improves the response efficiency of the agent, but also ensures that the game reasoning mechanism has a real-time basis that can be landed in complex traffic scenarios; in the system design of the present application, feature screening not only exists as a data preprocessing step, but is also directly bound to subsequent agent modeling. By retaining only input variables that significantly contribute to game reasoning results, the model input dimension can be significantly reduced, improving the speed of strategy reasoning and consistency detection, thereby meeting the millisecond-level takeover response requirements in high-speed driving conditions.

[0064] Further, the agent strategy set and the agent state space are mapped based on a state-strategy matching mapping model to obtain a decision mechanism, specifically:

[0065] Define an ideal agent state space template for each strategy, representing the most reasonable application of the agent state space combination for the agent strategy;

[0066] Compare the distance between the current state space and each agent state space template, and select the nearest agent strategy as the mapped agent strategy;

[0067] The mapping relationship is used as a decision mechanism.

[0068] It should be noted that the "agent state space template" refers to a set of typical state feature vectors corresponding to each strategy, which is summarized from a large amount of historical data and expert experience. Each template reflects the typical performance interval of the strategy in the multi-dimensional state space through clustering analysis or expert-defined methods, and is used to guide the matching judgment of real-time state and strategy. The design of the template not only ensures the coverage of the strategy application scenario, but also helps to reduce the misjudgment probability of strategy selection.

[0069] It should be noted that "multi-dimensional distance measurement" is used to quantify the similarity between the current state vector and each strategy template. Common distance measurements include Euclidean distance, Mahalanobis distance, and cosine similarity, etc. According to the distribution characteristics and covariance structure of the state features, select the appropriate measurement method to ensure that the distance calculation can truly reflect the importance of different features to strategy matching, and support more accurate mapping results.

[0070] It should be noted that the "decision mechanism" refers to the output of the optimal response strategy of the agent through the mapping of the state space to the strategy set.

[0071] This mechanism ensures that in a complex and dynamic driving environment, the agent can quickly select a reasonable strategy according to the real-time state, taking into account the actual operating ability and intention of the driver, as well as the safety control needs of the autonomous driving system, achieving human-machine collaborative emergency takeover decision-making.

[0072] In the present embodiment, the consistency of the instructions is detected based on the agent, specifically:

[0073] obtaining the current optimal strategy from the decision mechanism of the driver agent and the autonomous driving system agent respectively;

[0074] mapping the actions or intentions in the strategy set of the driver and the autonomous driving system to a unified operation category space, such as classifying actions such as “active takeover”, “delayed response”, “refuse takeover”, “irrational intervention”, and “continuous control”, “request takeover”, “forced control”, “emergency braking” into category labels such as “takeover intention”, “autonomous driving intention” or “conflict intention”, facilitating subsequent comparison.

[0075] predefining a set of strategy consistency judgment rules to define which strategy combination belongs to a consistent (cooperative) state and which belongs to an inconsistent (conflict) state. For example:

[0076] the driver strategy is “active takeover” and the autonomous driving strategy is “request takeover” or “forced control”, which is considered consistent;

[0077] the driver strategy is “refuse takeover” and the autonomous driving strategy is “forced control” or “emergency braking”, which is considered a conflict.

[0078] According to the above rules, it is judged whether the instructions are consistent. If they are consistent, the instruction consistency detection passes, and the takeover process continues;

[0079] If they are not consistent, enter the conflict strategy processing flow.

[0080] In this embodiment, if the detection result is inconsistent, the conflict strategy processing flow is entered, the pros and cons of the strategies of the two parties are evaluated according to the driver state confidence and the environmental risk level, and the game reasoning result is output, specifically:

[0081] a driver state confidence model and an environmental perception risk level model are constructed to output the confidence score of the driver strategy and the safety priority score of the autonomous driving system strategy respectively;

[0082] The driver state confidence model uses a machine learning model to train historical driving data, the model learns the feature distribution of the driver in normal and abnormal states, real-time input of the current driver state features, and output of the confidence score;

[0083] The confidence score is a confidence score between 0 and 1, indicating the confidence of the driver state data and the rationality of the current takeover intention of the driver;

[0084] The environmental perception risk level model maps multi-dimensional environmental data to a comprehensive risk score through a risk score function, and converts the comprehensive risk score to a value in the range of 0~1 through a normalization function as a safety priority score;

[0085] In combination with the confidence score of the driver strategy and the safety priority score of the autonomous driving system strategy, the driver strategy and the autonomous driving system strategy are evaluated using a weight weighting method and compared with a threshold to evaluate the pros and cons;

[0086] The following is an example of optional pros and cons evaluation:

[0087]

[0088] In the formula, is a decision value; and is a weight, representing different emphasis on the importance of the driver state and the environmental risk, and adding up to 1; is the confidence score of the driver strategy, the higher indicating that the driver state is more reliable, and the more sufficient reason for giving priority to the driver's intention; is the safety priority score of the autonomous driving system strategy, the higher indicating that the environment is more dangerous, and the more sufficient reason for giving priority to the autonomous driving system control strategy;

[0089] When the decision value is greater than zero, the driver strategy is preferred; when the decision value is less than or equal to zero, the autonomous driving system strategy is preferred.

[0090] It should be noted that the driver state confidence model is based on machine learning technology, which is trained on historical driving data containing various driving behaviors and states. The model can accurately capture the feature distribution of the driver in normal driving, fatigue, distraction, and irrational behavior, and realize effective evaluation and confidence output of the real-time driver state. There is mature technology, which will not be repeated here.

[0091] It should be noted that the confidence score is a probabilistic measure of the rationality of the driver's current state and takeover intention, with a numerical range limited to 0 to 1. The higher the value, the more stable the driver's state and the more reliable the takeover intention. On the contrary, it indicates that there is potential risk or interference. The confidence output of the model helps to assist in takeover decision-making.

[0092] It should be noted that the environmental perception risk level model uses a multi-dimensional risk scoring function, which considers vehicle speed, obstacle distance, road complexity, traffic flow density, and weather and lighting, etc. Environmental factors are considered. After weighted calculation, a comprehensive risk score is obtained, which is then converted to the interval of 0 to 1 using a normalization method as the safety priority score of the autonomous driving system strategy, reflecting the degree of environmental danger. There is mature technology, which will not be repeated here.

[0093] It should be noted that the weight weighting method flexibly adjusts the relative influence of the driver state confidence score and the environmental perception safety priority score in the comprehensive decision by setting a weight parameter, the sum of the weights is fixed at 1, and can be dynamically adjusted according to different scenes, driving modes and risk preferences, to realize more reasonable strategy advantage and disadvantage judgment.

[0094] It should be noted that the calculation form of the decision value is a weighted difference value, which reflects the trade-off between the reliability of the driver state and the safety demand of the environmental risk, a positive value indicates that the current driver has higher control ability and reasonable takeover intention, and the driver strategy is preferred; a negative value indicates that the environmental risk is higher, and the automatic driving system strategy is safer and more reasonable, and the system control is preferred.

[0095] It should be noted that the threshold value can be adjusted according to actual application requirements and safety standards to ensure the robustness and sensitivity of the takeover decision, and combined with historical data and real-time feedback to continuously optimize, improve the overall safety and response accuracy of the system, and the threshold value in the example is zero.

[0096] S4, the game reasoning simplifies the game model judgment through a finite state machine.

[0097] In this embodiment, the game reasoning simplifies the game model judgment through a finite state machine, specifically:

[0098] The state space is composed of the screened multi-dimensional feature vector;

[0099] A preset number of feature dimensions are screened out based on feature importance analysis;

[0100] The multi-dimensional feature vector is reduced to a preset number of feature dimensions based on principal component analysis, to obtain a continuous low-dimensional state vector space;

[0101] A second threshold value is set according to the statistical distribution of the features, and the continuous low-dimensional state vector space is divided into several discontinuous intervals;

[0102] Each interval is taken as a finite state set, and several finite state sets are constructed to form an agent state space for game reasoning.

[0103] It should be noted that the "feature importance analysis" uses a feature weight evaluation method based on supervised learning, such as random forest, gradient boosting tree and other algorithms, to judge the contribution of each feature to the takeover event classification through model training, so as to screen out the feature dimensions with the greatest impact, and ensure that the screening process is objective and data-driven.

[0104] It should be noted that the "principal component analysis method" is an unsupervised dimensionality reduction technique that maps a multi-dimensional feature space to a low-dimensional space through linear transformation, retains most of the variance information in the data, reduces feature redundancy, reduces state space dimension, and improves subsequent calculation efficiency and generalization ability.

[0105] It should be noted that the "second threshold value is set according to the statistical distribution of the feature" refers to dividing the continuous numerical interval according to the mean, standard deviation, quantile and other statistical indicators of each dimension of the reduced feature, such as dividing into "low", "medium", "high" three level intervals, to realize the discretization of the state space.

[0106] It should be noted that the "finite state set" is formed by the combination of the intervals divided by each feature dimension, and the number is the product of the number of intervals in each dimension, which constitutes a finite and enumerable agent state space, which is convenient for state transition and game reasoning combined with finite state machine, and takes into account the description ability and calculation efficiency.

[0107] The game reasoning simplifies the game model judgment through the finite state machine, significantly reduces the calculation complexity of the traditional complete game model caused by the huge state space and strategy combination, and improves the response speed and adaptability of the system to the real-time dynamic environment; at the same time, the finite state machine makes the complex instruction consistency detection process more clear and controllable through discretization and structured state representation, effectively guarantees the accuracy and stability of the emergency takeover trigger, and solves the key technical bottlenecks of real-time deficiency and limited computing resources.

[0108] S5, triggering takeover according to the game reasoning result.

[0109] Embodiment 2, Figure 2 The application provides an emergency takeover triggering system based on environment perception and driver state recognition, which comprises a risk perception module, a takeover issuing module, a consistency detection and man-machine conflict processing module, a game simplification module and a takeover module.

[0110] The risk perception module is used for acquiring environment perception data and driver state data, and performing risk level evaluation respectively.

[0111] The takeover issuing module is used for judging whether to initiate a takeover request according to the risk level.

[0112] The consistency detection and man-machine conflict processing module is used for, if the takeover request is issued, performing instruction consistency detection based on game modeling, abstracting the driver and the automatic driving system as two agents, and reasoning the operation intention of each agent respectively; if the detection result is inconsistent, entering a conflict strategy processing flow, evaluating the advantages and disadvantages of the strategies of the two parties according to the driver state confidence and the environment risk level, and outputting a game reasoning result.

[0113] The game simplification module is configured to simplify the game model by a finite state machine for the game reasoning to determine;

[0114] The takeover module is configured to trigger takeover according to the game reasoning result.

[0115] The above formulas are all dimensionless values calculated, and the formulas are obtained by collecting a large amount of data to simulate a formula of the most recent real situation, and the preset parameters in the formula are set by a person skilled in the art according to actual conditions.

[0116] The above embodiments can be realized wholly or partially by software, hardware, firmware or any combination thereof. When realized by software, the above embodiments can be realized wholly or partially in the form of a computer program product.

[0117] Those skilled in the art can realize that the modules and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0118] In addition, each functional module in each embodiment of the present application can be integrated in one processing module, or each module can exist physically independently, or two or more modules can be integrated in one module.

[0119] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0120] Finally, the above is only the preferred embodiments of the present application, and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for triggering an emergency takeover based on environmental perception and driver state recognition, characterized in that, The method comprises the following steps: obtaining environment perception data and driver state data, and respectively performing risk level evaluation; determining whether to initiate a takeover request according to the risk level; if the takeover request is sent, performing instruction consistency detection based on game modeling, abstracting the driver and the automatic driving system as two agents, and respectively reasoning the operation intention of the two agents; if the detection result is inconsistent, entering a conflict strategy processing flow, evaluating the strategies of the two parties according to the driver state confidence and the environment risk level, and outputting a game reasoning result, specifically: constructing a driver state confidence model and an environment perception risk level model, and respectively outputting a credibility score of the driver strategy and a safety priority score of the automatic driving system strategy; the driver state confidence model uses a machine learning model to train historical driving data, the model learns the feature distribution of the driver in normal and abnormal states, real-time inputs the current driver state features, and outputs the credibility score; the credibility score is a confidence score between 0 and 1, indicating the credibility of the driver state data and the rationality of the current takeover intention of the driver; the environment perception risk level model maps multi-dimensional environment data into a comprehensive risk score through a risk score function, and converts the comprehensive risk score into a value in the range of 0 to 1 through a normalization function as the safety priority score; combining the credibility score of the driver strategy and the safety priority score of the automatic driving system strategy, the driver strategy and the automatic driving system strategy are evaluated by using a weight weighting method and compared with a threshold to evaluate the advantages and disadvantages; the game reasoning is simplified by using a finite state machine to determine the game model, specifically: a preset number of feature dimensions are selected based on feature importance analysis; the multi-dimensional feature vector is reduced to a preset number of feature dimensions based on principal component analysis to obtain a continuous low-dimensional state vector space; a second threshold is set according to the statistical distribution of the features, and the continuous low-dimensional state vector space is divided into several discontinuous intervals; each interval is regarded as a finite state set, and several finite state sets are used to construct an agent state space for game reasoning; the takeover is triggered according to the game reasoning result.

2. The method of claim 1, wherein the method further comprises: The environment perception data and the driver state data are obtained, and the risk level is evaluated, specifically: an environment risk score is calculated based on the environment perception data by constructing a weighted scoring model; a driver state risk score is calculated based on the driver state data; the weighted scoring model quantitatively processes various risk parameters according to preset multi-dimensional risk factors, and determines the risk level according to the comparison between the risk score and the set threshold. 3.The method of claim 2, wherein, The driver and the automatic driving system are abstracted as two agents, and the operation intention of the two agents is reasoned, specifically: the two agents include a driver agent and an automatic driving system agent; the agent is composed of an agent state space, an agent strategy set and a decision mechanism; the environment perception data and the driver state data are extracted and selected to obtain filtered features; based on the filtered features, the driver agent state space and the automatic driving system agent state space are respectively constructed; the takeover is triggered according to the game reasoning result. The driver agent state space and the autonomous driving system agent state space are both defined by a multi-dimensional vector space, and the vector elements are a set of filtered driver state data feature values and a set of environmental perception data feature values respectively; For each agent state space, a set of optional strategies is defined; The driver agent strategy set includes the operation intentions of active takeover, delayed response, refusal to take over, and irrational intervention; The autonomous driving system agent strategy set includes the control actions of continuous control, request for takeover, forced control, and emergency braking; The agent strategy set and the agent state space are mapped based on a state-strategy matching mapping model to obtain a decision mechanism; The consistency of the instructions of the agent is detected.

4. The method of claim 3, wherein the method further comprises: The filtering features are specifically: A historical driving database containing multiple scenarios, multiple drivers, and multiple road conditions is constructed, and the database includes historical environmental perception data and driver state data, and corresponding actual takeover event labels; Based on real-time environmental perception data and driver state data, a preliminary feature set is constructed; Using a preset database, a supervised learning algorithm is applied to train the preliminary feature set, evaluate the contribution of each feature to the takeover event judgment, and output a feature importance score; According to the feature importance score and the actual computing resource constraint, high-score features are selected to form the first filtered features; The first filtered features are analyzed for correlation, and redundant features are removed to obtain the filtered features.

5. The method of claim 4, wherein the method further comprises: The agent strategy set and the agent state space are mapped based on a state-strategy matching mapping model to obtain a decision mechanism, specifically: The agent strategy set and the agent state space are mapped based on a state-strategy matching mapping model to obtain a decision mechanism, specifically: An ideal agent state space template is defined for each strategy, representing the most reasonable applicable agent state space combination for the agent strategy; The distance between the current state space and each agent state space template is compared, and the nearest agent strategy is selected as the mapped agent strategy; The mapping relationship is used as the decision mechanism.

6. A system using the emergency takeover trigger method based on environmental perception and driver state recognition according to any one of claims 1-5, characterized in that, The system includes a risk perception module, a takeover issuance module, a consistency detection and human-machine conflict processing module, a game simplification module, and a takeover module; The risk perception module is used to obtain environmental perception data and driver state data, and perform risk level evaluation respectively; The takeover issuance module is used to determine whether to initiate a takeover request according to the risk level; The consistency detection and human-machine conflict processing module is used to perform instruction consistency detection based on game modeling if a takeover request is issued, and abstract the driver and the autonomous driving system as two agents to reason their operation intentions; If the detection result is inconsistent, the conflict strategy processing flow is entered, the strategy of the conflict parties is evaluated according to the driver state confidence and the environmental risk level, and the game reasoning result is output; The game simplification module is used to simplify the game model by a finite state machine to determine the game reasoning; The takeover module is used to trigger the takeover according to the game reasoning result.

Images