Automatic driving takeover method and system based on cognition-execution double-layer decision

Abstract

The invention belongs to the field of road vehicle control systems, and relates to an automatic driving takeover method and system based on cognition-execution double-layer decision. The system comprises a multi-mode large model, a takeover mode judgment module, an early warning intensity calculation module, a sensing reminding module, a preparation state evaluation module, a self-adaptive weight distribution module, a control signal output module and a guide interface generation module. Wherein the multi-modal large model is used for reasoning according to input data and outputting structured takeover decision information; the early warning intensity calculation module is used for calculating takeover early warning intensity changing along with time; the preparation state evaluation module is used for evaluating the driver's preparation for taking over; the control signal output module is used for realizing smooth transfer of the man-machine control right under the safety constraint through a control fusion mechanism according to the self-adaptive control weight of the driver; the system not only improves the accuracy and interpretability of the takeover decision, but also effectively improves the takeover performance and reduces the takeover risk.

Description

Technical Field

[0001] The present invention belongs to the field of road vehicle control systems and relates to the takeover of autonomous driving vehicles. Specifically, it relates to an autonomous driving takeover method and system based on cognitive-execution dual-layer decision-making, which is particularly suitable for the human-machine co-driving control switching scenario of L2-L3 autonomous driving vehicles. Background Art

[0002] In recent years, with the rapid development of autonomous driving technology, its practical application has become very common. However, due to the limitations of existing technical solutions, L2-L3 autonomous driving systems still need to transfer vehicle control to the driver in certain operating scenarios that are difficult for autonomous driving systems to handle, thus raising the issue of driving control handover. Existing takeover decision-making methods are mainly based on preset rules or threshold judgments. They suffer from poor scenario adaptability, uneven control transfer, lack of explainability in the decision-making process, and lack of effective takeover guidance. They are unable to meet the requirements of safe takeover in complex driving environments.

[0003] The rapid development of large multimodal language models (MLLMs), with their powerful language understanding and cognitive reasoning capabilities, has opened up new technological avenues for autonomous driving systems. For example, Chinese patent CN 118810764 A discloses a multi-level driving risk management method, system, device, and medium. On the vehicle side, this method performs an initial risk assessment on the simultaneously captured forward-view video stream, vehicle body information data, and DMS video stream to obtain real-time and cumulative risks. A timely alert is issued for real-time risks, and the corresponding cumulative risk data is uploaded to the cloud. A second-level risk assessment is then performed using multiple trained MLLM models to obtain driver status and algorithm confidence information. Alert intervention is initiated for data with high algorithm confidence and high risk, improving risk warning efficiency. However, large multimodal language models have not yet been used in autonomous driving takeover systems. Therefore, how to fully utilize the cognitive reasoning capabilities of large multimodal models and combine them with adaptive execution control strategies to design an accurate, smooth, interpretable, and guided takeover decision-making method has become a key issue urgently needed in the development of autonomous driving technology. Summary of the Invention

[0004] In view of the above technical problems and defects, the purpose of the present invention is to provide an automatic driving takeover method based on cognitive-execution dual-layer decision-making, which collects information such as environmental perception information around the vehicle, the status of the vehicle, the driver's status and the health status of the intelligent driving system; at the cognitive layer, multimodal deep features are extracted and feature fusion processing is performed through a cross-attention mechanism, and risk assessment, takeover decision parameters, safe operation boundaries and natural language prompts are output based on multimodal large model reasoning, and whether to trigger takeover and the takeover mode are determined according to the necessity and urgency of takeover; at the execution layer, based on the takeover mode determined by the cognitive layer, the driver is guided to prepare for takeover through multi-level warning signals, and the control weight is dynamically adjusted according to the driver's readiness status, so as to achieve smooth transfer of human-machine control rights under safety constraints. The cognitive-execution dual-layer architecture not only improves the accuracy and interpretability of takeover decisions, but also effectively improves the takeover performance of L2-L3 automatic driving and reduces takeover risks.

[0005] To achieve the above object, the present invention adopts the following technical solutions: An autonomous driving takeover method based on a cognitive-execution dual-layer decision-making process includes the following steps: Step S10. Collecting environmental perception information, vehicle motion status information, driver status information, and intelligent driving system health monitoring information; Step S20: extracting features from the multi-source information obtained in step S10, and then performing multimodal feature fusion; Step S30. Based on the fusion features of step S20, reasoning is performed through a multimodal large model to output structured takeover decision information including risk assessment results, takeover decision parameters, safe operation boundaries, and natural language prompts; Step S40. Determine whether to trigger takeover and the takeover mode based on the takeover decision parameters output in step S30. If it is determined that the driver needs to take over, execute step S50. Step S50: The execution layer guides the driver to prepare for takeover through multi-level warning signals based on the takeover mode determined by the cognitive layer; Step S60. After the warning guidance signal of step S50 is activated, the execution layer gradually realizes the smooth transfer of human-machine control rights through risk-adaptive control weight allocation, control fusion under safety constraints, and real-time visual guidance based on real-time feedback of the driver's readiness status.

[0006] As a preferred embodiment of the present invention, the environmental perception information includes image data collected by visual sensors, point cloud data collected by lidar, and target detection data collected by millimeter-wave radar; the vehicle motion status information includes vehicle speed, acceleration, and heading angle; the driver status information includes eye movement trajectory, head posture, and steering wheel grip strength; the intelligent driving system health monitoring information includes sensor working status, positioning accuracy, and computing power load rate.

[0007] As a preferred embodiment of the present invention, step S20 includes the following steps: Step S201. The acquired environmental perception information, vehicle motion state information, driver state information, and intelligent driving system health monitoring information are respectively input into the environmental perception fusion encoder, vehicle dynamics encoder, driver state encoder, and intelligent driving system monitoring information encoder for feature extraction; Step S202: After concatenating the four types of feature vectors, the extracted multimodal features are fused using a cross-attention mechanism.

[0008] As a preferred embodiment of the present invention, the risk assessment result R in step S30 includes the risk severity and the risk source boundary distribution, the takeover decision parameter D includes the necessity, urgency and available takeover time of takeover; the safe operation boundary B includes the speed safety boundary, the acceleration safety boundary and the position safety boundary, and the natural language prompt is generated by the natural language generation module based on the risk assessment result R, the takeover decision parameter D and the safe operation boundary B, and is used to convert the decision information into text prompts that the driver can understand.

[0009] As a preferred embodiment of the present invention, in step S40, according to the necessity of taking over outputted in step S30, and urgency , determine whether to trigger takeover and select the takeover mode Mode: ; Where, The threshold for necessity of takeover; is the low urgency threshold; is the high urgency threshold.

[0010] As a preferred embodiment of the present invention, step S50 includes the following steps: Step S501. According to the severity of the risk in step S30 , Available takeover time And the takeover mode of step S40 , calculate the takeover warning intensity over time : ; Where, The trigger moment for takeover; is the urgency adjustment factor; exp is an exponential function; The basic warning intensity is determined according to the takeover mode; Step S502 . Select and activate one or more driver perception reminder modes according to the takeover warning intensity calculated in step S501 , wherein the driver perception reminder modes include a visual reminder mode, an auditory reminder mode, and a tactile reminder mode.

[0011] As a preferred embodiment of the present invention, step S60 includes the following steps: Step S601. Calculate the driver's readiness H(t) based on the driver's status information to assess the driver's readiness to take over; Step S602. Calculate the driver's adaptive control weight based on the driver's readiness H(t) obtained in step S601 ; Step S603: Driver adaptive control weight obtained in step S602 , through the control fusion mechanism, a smooth transfer of human-machine control rights under safety constraints is achieved: Fusion control signal The safe operation boundary B(t) output in step S30 should be met, and the final control signal after safety constraint processing for: ; Where, The projection operator to the safety boundary ensures that the control signal is always within the safe operation boundary; Step S604. During the control fusion process of executing step S603, the takeover decision information outputted in step S30 is converted into an intuitive guidance interface to assist the driver in completing the takeover operation.

[0012] As a further preferred embodiment of the present invention, the expression of driver readiness H(t) is: ; Where, for alertness; In the holding state; For visual attention; 、 、 They are alertness weight, grip state weight and sight attention weight respectively.

[0013] As a further preferred embodiment of the present invention, the driver's adaptive control weight The expression is: ; Where, is the transition rate parameter; t is the current moment; is the weight transfer center moment, and its value is ,in The takeover trigger moment.

[0014] The present invention also provides an autonomous driving takeover system based on a cognitive-execution dual-layer decision-making process. The autonomous driving takeover system includes a data acquisition module, a data processing module, a multimodal large model, a takeover mode determination module, a warning intensity calculation module, a perception reminder module, a readiness status assessment module, an adaptive weight allocation module, a control signal output module, and a guidance interface generation module. The data acquisition module is connected to an on-board sensor system and is used to collect environmental perception information, vehicle motion status information, driver status information, and intelligent driving system health monitoring information. The data processing module is used to extract features from the collected data and perform multimodal feature fusion processing using a cross-attention mechanism; The multimodal large model is used to perform reasoning based on the fused features and output structured takeover decision information including risk assessment, takeover decision parameters, safe operation boundaries, and natural language prompts; The takeover mode judgment module is used to determine whether to trigger takeover and the takeover mode based on the takeover decision parameters output by the multimodal large model; The warning intensity calculation module is used to calculate the takeover warning intensity that changes over time based on the risk assessment, takeover decision parameters, and the determined takeover mode output by the multimodal large model; The perception reminder module is used to selectively activate visual, auditory, and tactile reminder modes according to the takeover warning intensity to remind the driver to prepare for takeover; The readiness assessment module is configured to assess the driver's readiness to take over after the reminder mode is activated; The adaptive weight allocation module is used to calculate the driver's adaptive control weight according to the driver's readiness; The control signal output module realizes smooth transfer of human-machine control rights under safety constraints through a control fusion mechanism based on the driver's adaptive control weight and outputs the final control signal; The guidance interface generation module is used to generate a guidance interface based on takeover decision parameters, safe operation boundaries, and natural language prompts, and present the drivable area, risk source location, and operation suggestions in real time.

[0015] The advantages and beneficial effects of the present invention are: (1) The present invention proposes an autonomous driving takeover method with a cognitive-execution dual-layer decision-making architecture. The method divides the takeover process into high-level decision-making at the cognitive layer and low-level control at the execution layer. The cognitive layer outputs risk assessment, takeover decision parameters, safe operation boundaries and natural language prompts based on multimodal large model reasoning, and determines whether to trigger takeover and the takeover mode according to the necessity and urgency of takeover; the execution layer guides the driver to prepare for takeover through multi-level warning signals based on the takeover mode determined by the cognitive layer, and dynamically adjusts the control weight according to the driver's readiness status, so as to achieve a smooth transfer of human-machine control rights under safety constraints. The cognitive-execution dual-layer architecture not only improves the accuracy and explainability of takeover decisions, but also effectively improves the takeover performance of L2-L3 autonomous driving, reduces takeover risks, breaks through the limitations of traditional single decision-making models, and provides a new hierarchical decision-making framework for autonomous driving takeover in complex scenarios.

[0016] (2) The present invention proposes a takeover cognitive reasoning method based on a multimodal large model. The output structure is specially designed according to the takeover control requirements. The trained model can provide clear takeover trigger conditions, time constraints, safety boundaries and other specific parameters, so that the driver can understand "why takeover" (through R risk assessment and T semantic information) and "how to take over" (through D decision parameters and B safety boundaries). The completeness and executability of the takeover decision output are guaranteed, and the technical problems of the existing takeover decision process lacking interpretability, the difficulty for the driver to understand the system intention, and the serious impact on the human-machine trust relationship are solved.

[0017] (3) The MLLM in the present invention realizes accurate identification, risk assessment and takeover necessity judgment of complex scenarios through deep semantic understanding, overcoming the shortcomings of traditional rule-based methods in that they have poor adaptability in unpredictable scenarios and are difficult to cope with complex unpredictable scenarios, and significantly improves the accuracy and generalization ability of takeover decisions.

[0018] (4) The present invention organically combines the scene understanding capability of MLLM with the real-time control requirements, and realizes the coordinated optimization of high-level cognition and low-level control through a two-layer architecture, achieving the smoothness (through weight gradient) and security (through boundary constraints) of the takeover process, thereby improving the practicality and reliability of the system.

[0019] (5) The present invention proposes an adaptive control weight allocation scheme, which adjusts the human-machine control rights allocation ratio in real time based on the driver's readiness, solves the vehicle instability problem caused by the sudden change of control rights in the traditional takeover method, and realizes a smooth and safe transfer of control rights.

[0020] (6) The present invention integrates a natural language generation module and a real-time visual guidance interface, converting the decision information of the cognitive layer into semantic prompts and intuitive visual guidance that the driver can understand, significantly improving the interpretability of the takeover decision and the efficiency of human-computer interaction, and reducing the driver's cognitive load and operational risks.

[0021] (7) The present invention proposes a multi-mode takeover strategy adaptive selection mechanism, which selects progressive, collaborative, and emergency takeover modes according to the necessity and urgency of the takeover, achieving the optimal balance between takeover efficiency and security under different risk levels.

[0022] (8) The present invention introduces security operation boundary constraints to ensure that the entire takeover process is always within the security domain, effectively reducing the takeover risk. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is an overall flow chart of an autonomous driving takeover method based on cognition-execution dual-layer decision-making provided by an embodiment of the present invention; Figure 2 1 is a structural diagram of a large multimodal model (MLLM) of the cognitive layer in an embodiment of the present invention; Figure 3 This is a takeover control flow chart of the execution layer in an embodiment of the present invention; Figure 4 This is a structural block diagram of an autonomous driving takeover system based on cognition-execution dual-layer decision-making provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the technical solutions and advantages of the present invention, the present application is described in detail below with reference to the accompanying drawings, but this is not intended to limit the scope of protection of the present invention.

[0025] Example 1:

[0026] Figure 1 This is an overall flow chart of an autonomous driving takeover method based on cognition-execution dual-layer decision-making provided by the present invention, such as Figure 1 As shown, the automatic driving takeover method includes the following steps: Step S10. Collecting environmental perception information, vehicle motion status information, driver status information, and intelligent driving system (hereinafter referred to as intelligent driving system) health monitoring information through the vehicle sensor system; Specifically, in this embodiment, the environmental perception information includes but is not limited to image data collected by visual sensors, point cloud data collected by lidar, and target detection data collected by millimeter-wave radar; the vehicle motion state information includes but is not limited to dynamic parameters such as vehicle speed, acceleration, and heading angle; the driver state information includes but is not limited to driver behavior state parameters such as eye movement trajectory, head posture, and steering wheel grip strength; the intelligent driving system health monitoring information includes but is not limited to the working status of each sensor, positioning accuracy, and computing power load rate.

[0027] Step S20: The cognitive layer extracts features from the multi-source information obtained in step S10 through the corresponding feature encoder and performs multimodal feature fusion processing using a cross-attention mechanism; like Figure 2 As shown, in this embodiment, step S20 includes the following steps: Step S201. The various types of data information obtained are input into the corresponding feature encoder for feature extraction; For environmental perception information, an environmental perception fusion encoder is used to output a unified scene feature vector :

[0028] Where, It is an environment perception fusion encoder; For visual data; Point cloud data; Target detection data.

[0029] Specifically, in this embodiment, a convolutional neural network is used to extract visual features, a point cloud feature extraction network is used to process point cloud data, and a multi-layer perceptron is used to process target detection data. The extracted features are then spliced ​​and input into the fully connected layer, and finally a unified scene feature vector is output. , the expression is:

[0030] Where, stands for Convolutional Neural Network, which is used to extract visual features; Represents the point cloud feature extraction network, which is used to extract point cloud features; Represents a multi-layer perceptron, which is used to process target detection data; It is the feature splicing operation; is the weight; is bias; It is the GELU (Gaussian Error Linear Unit) activation function.

[0031] For vehicle motion state information, the vehicle dynamics encoder is used to output the motion mode feature vector :

[0032] Where, for vehicle dynamics encoders; Vehicle motion status information.

[0033] Specifically, in this embodiment, the motion state vector at each moment is first obtained, and then the local temporal features are extracted through one-dimensional convolution, and then the temporal dependency is captured through the long short-term memory network, and the motion pattern feature vector is output. , the expression is:

[0034] Where, is the motion state vector at time t; Represents one-dimensional convolution, used to extract local temporal features; LSTM is a long short-term memory network used to capture temporal dependencies; n is the length of the time window.

[0035] For the driver state information, the driver state encoder is used to output the driver state feature vector :

[0036] Where, It is the driver status encoder; Driver status information.

[0037] Specifically, in this embodiment, the features of eye movement, head posture and grip strength data are extracted respectively, and then the multi-source features are adaptively fused through the attention mechanism, and finally the driver state feature vector is output. , the expression is:

[0038] Where, 、 、 They are eye movement, head posture, and grip strength data; 、 、 is the corresponding feature extraction function; Attention It is an attention mechanism to achieve adaptive fusion of multi-source features.

[0039] For the health monitoring information of the intelligent driving system, the intelligent driving system monitoring information encoder is used to output the intelligent driving system monitoring information feature vector. :

[0040] Where, Monitor information encoder for intelligent driving system; Monitor information for the intelligent driving system.

[0041] Specifically, in this embodiment, the health monitoring information of the intelligent driving system is normalized first, and then the feature vector of the intelligent driving system monitoring information is output after two fully connected layers. , the expression is:

[0042] Where, is the normalized system state vector; 、 is the weight matrix; 、 is the bias vector; is the rectified linear unit activation function.

[0043] It should be noted that the feature extraction process for environmental perception information, vehicle motion status information, driver status information, and intelligent driving system health monitoring information in this embodiment is only used for illustration and is not intended to limit the present invention. Those skilled in the art can refer to the existing technology to extract the above-mentioned feature vectors.

[0044] Step S202: After concatenating the four types of feature vectors, the extracted multimodal features are fused using a cross-attention mechanism:

[0045] Where, is the fused feature vector; It is the cross attention fusion module.

[0046] Step S30. Based on the fusion features of step S20, reasoning is performed through a multimodal large model to output structured takeover decision information including risk assessment, takeover decision parameters, safe operation boundaries, and natural language prompts; Continue as Figure 2 As shown, in this embodiment, step S30 includes the following steps: The fusion feature vector obtained in step S20 Input into the pre-trained multimodal large model (MLLM) and output structured takeover decision information :

[0047] Where, The results of the risk assessment; Parameters for takeover decision; To provide safe operating boundaries; For natural language prompts.

[0048] The risk assessment result R includes the risk level (risk severity) and the risk source boundary distribution:

[0049] Where, is the severity of the risk; is the boundary position of each risk source.

[0050] The takeover decision parameter D includes the key parameters of takeover control:

[0051] Where, To take over the necessity; For the degree of urgency; The available takeover time.

[0052] Safe operation boundary B provides a clear safety boundary for the subsequent takeover process:

[0053] Where, It is the speed safety boundary; is the acceleration safety boundary; The location safety boundary.

[0054] Natural language prompts It is generated by the natural language generation module based on the risk assessment result R, takeover decision parameter D, and safe operation boundary B:

[0055] Where, The natural language generation module converts decision information into text prompts that the driver can understand.

[0056] In this embodiment, the collected data is used to train the multimodal large model (MLLM). In terms of training data, an expert system is used in combination with real driving scenarios to construct a training set, and each driving scenario contains a structured annotation {R, D, B, T} quadruple. Specifically, the training data set covers a variety of working conditions such as normal driving, takeover scenarios, and emergency scenarios to ensure that the model can accurately output the above-mentioned structured information. In terms of the model architecture of this embodiment, the MLLM body follows the existing structure, with the focus on the special design of input encoding and output structure. Four dedicated encoders are set at the input end: the environment perception fusion encoder EncoderScene, the vehicle dynamics encoder EncoderMotion, the driver state encoder EncoderDriver, and the intelligent driving system monitoring information encoder EncoderSystem. The feature vectors output by these encoders are fused through the cross-attention mechanism CrossAttention to form a unified fusion feature F; the output end sets a structured output head based on the MLLM, so that the model can directly output the structured decision information TD={R, D, B, T}. The specific model training process can refer to existing methods; after training, the model trained based on the input data can directly output risk assessment results, takeover decision parameters, safe operating boundaries and natural language prompts, so that the driver can understand both "why take over" and "how to take over".

[0057] Step S40. Determine whether to trigger takeover and the takeover mode based on the necessity and urgency of takeover output in step S30; when step S40 determines that the driver needs to take over, execute step S50; like Figure 3 As shown, in this embodiment, in step S40: according to the necessity of taking over output in step S30 and urgency , determine whether to trigger takeover and select the takeover mode Mode:

[0058] Where, The threshold for necessity of takeover; is the low urgency threshold; is the high urgency threshold.

[0059] Step S50: The execution layer guides the driver to prepare for takeover through multi-level warning signals based on the takeover mode determined by the cognitive layer; Continue as Figure 3 As shown, in this embodiment, step S50 includes the following steps: Step S501. According to the severity of the risk in step S30 , Available takeover time And the takeover mode of step S40 , calculate the takeover warning intensity over time :

[0060] Where, The trigger moment for takeover; is the urgency adjustment factor, ;exp is an exponential function; The basic warning intensity is determined according to the takeover mode. The value is as follows:

[0061] Where, It is the low basic warning intensity, with a value of 0.3; The medium basic warning intensity is 0.6; It is the high basic warning intensity, with a value of 1.0.

[0062] Step S502: Activate the corresponding driver perception reminder mode according to the takeover warning intensity calculated in step S501:

[0063] Where, It is a visual reminder mode; It is auditory reminder mode; It is a tactile reminder mode; Activate thresholds for visual reminders; Activate thresholds for auditory reminders; Activate the threshold for the haptic alert.

[0064] Step S60. After the warning guidance signal of step S50 is activated, the execution layer gradually realizes the smooth transfer of human-machine control rights through risk-adaptive control weight allocation, control fusion under safety constraints, and real-time visual guidance based on real-time feedback of the driver's readiness status.

[0065] Continue as Figure 3 As shown, in this embodiment, step S60 includes the following steps: Step S601. Calculate the driver's readiness based on the driver's status information obtained in step S10 and evaluate the driver's readiness to take over; define the driver's readiness function :

[0066] Where, for alertness; In the holding state; For visual attention; 、 、 They are alertness weight, grip state weight and sight attention weight respectively.

[0067] Specifically, alertness Calculated through eye movement trajectory and head posture data:

[0068] Where, is the percentage of eyelid closure time at time t, calculated from eye movement trajectory data, with a value range of [0,1]; The angle at which the head deviates from the front is obtained through head posture data; It is the maximum threshold of head deviation, usually 45°.

[0069] Grip state Calculated by steering wheel grip force data:

[0070] Where, is the steering wheel grip force sensor measurement value at time t; This is the standard grip strength during normal driving, generally 15N.

[0071] Eye attention Calculated from eye tracking data:

[0072] Where, The time that the line of sight is focused on the road area within the time window; For the evaluation time window length, 3 seconds is generally taken.

[0073] Step S602. Calculate the driver's adaptive control weight based on the driver's readiness H(t) obtained in step S601 :

[0074] Where, is the transition rate parameter, and its value range is [1, 5]; t is the current time; is the weight transfer center moment, and its value is ,in The takeover trigger moment.

[0075] Step S603: Driver adaptive control weight obtained in step S602 , through the control fusion mechanism, a smooth transfer of human-machine control rights under safety constraints is achieved:

[0076] Where, is the fused control signal; Control input for the driver; It is the control input of the autonomous driving system.

[0077] In addition, the fused control signal The final control signal after safety constraint processing should meet the safety boundary B(t) defined in step S30. for:

[0078] Where, It is a projection operator to the safety boundary, ensuring that the control signal is always within the safe operation boundary.

[0079] Step S604. During the control fusion process of step S603, the takeover decision information outputted in step S30 is converted into an intuitive guidance interface to assist the driver in completing the takeover operation:

[0080] Where, For the interface rendering function; To generate the guide interface, guide interface The drivable area, risk source location, and operation suggestions are presented in real time through the head-up display or central control screen, allowing the driver to intuitively understand the current takeover status and safety boundaries, ensuring the safety of the takeover process.

[0081] Example 2: Figure 4 This is a structural diagram of an autonomous driving takeover system based on cognitive-execution dual-layer decision-making provided by the present invention, such as Figure 4 As shown, the autonomous driving takeover system includes a data acquisition module, a data processing module, a multimodal large model, a takeover mode judgment module, a warning intensity calculation module, a perception reminder module, a readiness status assessment module, an adaptive weight allocation module, a control signal output module, and a guidance interface generation module; The data acquisition module is connected to the vehicle sensor system to collect environmental perception information, vehicle motion status information, driver status information and intelligent driving system health monitoring information; The data processing module is used to extract features from the collected data and perform multimodal feature fusion processing using a cross-attention mechanism; The multimodal large model is used to perform reasoning based on the fused features and output structured takeover decision information including risk assessment, takeover decision parameters, safe operation boundaries, and natural language prompts; The takeover mode judgment module is used to determine whether to trigger takeover and the takeover mode based on the takeover decision parameters output by the multimodal large model; The warning intensity calculation module is used to calculate the takeover warning intensity that changes over time based on the risk assessment, takeover decision parameters, and the determined takeover mode output by the multimodal large model; The perception reminder module is used to selectively activate visual, auditory, and tactile reminder modes according to the takeover warning intensity to remind the driver to prepare for takeover; The readiness assessment module is configured to assess the driver's readiness to take over after the reminder mode is activated; The adaptive weight allocation module is used to calculate the driver's adaptive control weight according to the driver's readiness; The control signal output module realizes smooth transfer of human-machine control rights under safety constraints through a control fusion mechanism based on the driver's adaptive control weight and outputs the final control signal; The guidance interface generation module is used to generate a guidance interface based on takeover decision parameters, safe operation boundaries, and natural language prompts, presenting the drivable area, risk source location, and operation suggestions in real time, so that the driver can intuitively understand the current takeover status and safe boundaries, ensuring the safety of the takeover process.

[0082] The present invention also provides an electronic device comprising: one or more processors and a memory; wherein the memory is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned method of autonomous driving takeover based on cognition-execution dual-layer decision-making.

[0083] The present invention also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for autonomous driving takeover based on a cognitive-execution dual-layer decision-making.

[0084] Those skilled in the art will appreciate that all or part of the functions of the various methods / modules in the above embodiments may be implemented via hardware or via computer programs. When all or part of the functions in the above embodiments are implemented via computer programs, the program may be stored in a computer-readable storage medium, which may include a read-only memory, random access memory, a magnetic disk, an optical disk, a hard disk, etc., and the program is executed by a computer to implement the above functions. For example, the program may be stored in a memory of a device, and when the program in the memory is executed by a processor, all or part of the above functions may be implemented.

[0085] In addition, when all or part of the functions in the above-mentioned embodiments are implemented by means of a computer program, the program can also be stored in a storage medium such as a server, another computer, a disk, an optical disk, a flash drive or a mobile hard disk, and saved to the memory of a local device by downloading or copying, or the system of the local device is updated. When the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented.

[0086] The above description of the present invention using specific examples is intended only to facilitate understanding of the present invention and is not intended to limit the present invention. A person skilled in the art of the present invention may make several simple deductions, modifications, or substitutions based on the principles of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. An autonomous driving takeover method based on cognitive-execution dual-layer decision-making, characterized in that: The method comprises the following steps: Step S10. Collecting environmental perception information, vehicle motion status information, driver status information, and intelligent driving system health monitoring information; Step S20: extracting features from the multi-source information obtained in step S10, and then performing multimodal feature fusion; Step S30. Based on the fusion features of step S20, reasoning is performed through a multimodal large model to output structured takeover decision information including risk assessment results, takeover decision parameters, safe operation boundaries, and natural language prompts; Step S40. Determine whether to trigger takeover and the takeover mode based on the takeover decision parameters output in step S30. If it is determined that the driver needs to take over, execute step S50. Step S50: The execution layer guides the driver to prepare for takeover through multi-level warning signals based on the takeover mode determined by the cognitive layer; Step S60. After the warning guidance signal of step S50 is activated, the execution layer gradually realizes the smooth transfer of human-machine control rights through risk-adaptive control weight allocation, control fusion under safety constraints, and real-time visual guidance based on real-time feedback of the driver's readiness status.

2. The autonomous driving takeover method based on cognition-execution dual-layer decision-making according to claim 1, characterized in that: The environmental perception information includes image data collected by visual sensors, point cloud data collected by lidar, and target detection data collected by millimeter-wave radar; the vehicle motion status information includes vehicle speed, acceleration, and heading angle; the driver status information includes eye movement trajectory, head posture, and steering wheel grip strength; the intelligent driving system health monitoring information includes sensor working status, positioning accuracy, and computing power load rate.

3. The autonomous driving takeover method based on cognition-execution dual-layer decision-making according to claim 1, characterized in that: Step S20 includes the following steps: Step S201. The acquired environmental perception information, vehicle motion state information, driver state information, and intelligent driving system health monitoring information are respectively input into the environmental perception fusion encoder, vehicle dynamics encoder, driver state encoder, and intelligent driving system monitoring information encoder for feature extraction; Step S202: After concatenating the four types of feature vectors, the extracted multimodal features are fused using a cross-attention mechanism.

4. The autonomous driving takeover method based on cognition-execution dual-layer decision-making according to claim 1, characterized in that: In step S30, the risk assessment result R includes the risk severity and risk source boundary distribution, and the takeover decision parameter D includes the takeover necessity, urgency, and available takeover time; The safe operating boundary B includes the speed safety boundary, acceleration safety boundary, and position safety boundary. The natural language prompt is generated by the natural language generation module based on the risk assessment result R, the takeover decision parameter D, and the safe operating boundary B. It is used to convert the decision information into text prompts that the driver can understand.

5. The autonomous driving takeover method based on cognition-execution dual-layer decision-making according to claim 4, characterized in that: In step S40, according to the necessity of taking over outputted in step S30 and urgency , determine whether to trigger takeover and select the takeover mode Mode: ； Where, The threshold for necessity of takeover; is the low urgency threshold; is the high urgency threshold.

6. The autonomous driving takeover method based on cognition-execution dual-layer decision-making according to claim 4, characterized in that: Step S50 includes the following steps: Step S501. According to the severity of the risk in step S30 , Available takeover time And the takeover mode of step S40 , calculate the takeover warning intensity over time : ； Where, The trigger moment for takeover; is the urgency adjustment factor; exp is an exponential function; The basic warning intensity is determined according to the takeover mode; Step S502 . Select and activate one or more driver perception reminder modes according to the takeover warning intensity calculated in step S501 , wherein the driver perception reminder modes include a visual reminder mode, an auditory reminder mode, and a tactile reminder mode.

7. The method for autonomous driving takeover based on cognition-execution dual-layer decision-making according to claim 4, characterized in that: Step S60 includes the following steps: Step S601. Calculate the driver's readiness H(t) based on the driver's status information to assess the driver's readiness to take over; Step S602. Calculate the driver's adaptive control weight based on the driver's readiness H(t) obtained in step S601 ; Step S603: Driver adaptive control weight obtained in step S602 , through the control fusion mechanism, a smooth transfer of human-machine control rights under safety constraints is achieved: Fusion control signal The final control signal after safety constraint processing should meet the safety operation boundary output in step S30. for: ； Where, is the projection operator to the safety boundary, ensuring that the control signal is always within the safety operation boundary, and B(t) is the safety operation boundary at time t; Step S604. During the control fusion process of executing step S603, the takeover decision information outputted in step S30 is converted into an intuitive guidance interface to assist the driver in completing the takeover operation.

8. The autonomous driving takeover method based on cognition-execution dual-layer decision-making according to claim 7, characterized in that: The expression of driver readiness H(t) is: ； Where, for alertness; In the holding state; For visual attention; 、 、 They are alertness weight, grip state weight and sight attention weight respectively.

9. The autonomous driving takeover method based on cognition-execution dual-layer decision-making according to claim 7, characterized in that: Driver's adaptive control weights The expression is: ； Where, is the transition rate parameter; t is the current time; is the weight transfer center moment, and its value is ,in The takeover trigger moment.

10. An autonomous driving takeover system based on cognitive-execution dual-layer decision-making, characterized in that: The autonomous driving takeover system includes a data acquisition module, a data processing module, a multimodal large model, a takeover mode judgment module, a warning intensity calculation module, a perception reminder module, a readiness status assessment module, an adaptive weight distribution module, a control signal output module, and a guidance interface generation module; wherein the data acquisition module is connected to the vehicle sensor system to collect environmental perception information, vehicle motion status information, driver status information, and intelligent driving system health monitoring information; The data processing module is used to extract features from the collected data and perform multimodal feature fusion processing using a cross-attention mechanism; The multimodal large model is used to perform reasoning based on the fused features and output structured takeover decision information including risk assessment, takeover decision parameters, safe operation boundaries, and natural language prompts; The takeover mode judgment module is used to determine whether to trigger takeover and the takeover mode based on the takeover decision parameters output by the multimodal large model; The warning intensity calculation module is used to calculate the takeover warning intensity that changes over time based on the risk assessment, takeover decision parameters, and the determined takeover mode output by the multimodal large model; The perception reminder module is used to selectively activate visual, auditory, and tactile reminder modes according to the takeover warning intensity to remind the driver to prepare for takeover; The readiness assessment module is configured to assess the driver's readiness to take over after the reminder mode is activated; The adaptive weight allocation module is used to calculate the driver's adaptive control weight according to the driver's readiness; The control signal output module realizes smooth transfer of human-machine control rights under safety constraints through a control fusion mechanism based on the driver's adaptive control weight and outputs the final control signal; The guidance interface generation module is used to generate a guidance interface based on takeover decision parameters, safe operation boundaries, and natural language prompts, and present the drivable area, risk source location, and operation suggestions in real time.

Images