Automatic driving control method and device, electronic equipment and storage medium

Abstract

The application discloses an automatic driving control method and device, electronic equipment and storage medium, relates to the technical field of vehicles, and adopts a cognitive decision model, wherein the cognitive decision model comprises a cognitive layer and a decision layer; the automatic driving control method comprises the following steps: acquiring in-vehicle and out-vehicle monitoring data, extracting current cognitive layer features from the in-vehicle and out-vehicle monitoring data through the cognitive layer; inputting the current cognitive layer features into the decision layer to determine automatic driving control parameters; and generating driving behavior interpretation according to the current cognitive layer features and the automatic driving control parameters. The application improves the explainability of an end-to-end perception decision architecture, reduces negative emotions such as doubt, panic and unease of users, and improves user experience.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to an autonomous driving control method, device, electronic device, and storage medium. Background Technology

[0002] With the continuous development of technology and breakthroughs in computing power, autonomous driving is developing rapidly. Autonomous driving can provide people with a safer, more convenient and efficient way to travel, while also having a profound impact on the entire transportation system and urban planning.

[0003] The mainstream technical solutions for autonomous driving systems can be divided into two main types: rule-based modular architecture and data-driven end-to-end perception and decision-making architecture. While rule-based modular architectures have achieved mass production due to their advantages such as low computational requirements, simple deployment, and interpretable processes, they struggle to cover the entire scenario domain, and the results of modular solutions are not globally optimal. End-to-end solutions, on the other hand, use raw sensor data as input and generate planned and / or low-level control actions as output, enabling joint, global optimization and resulting in higher accuracy in autonomous driving decisions.

[0004] However, the end-to-end perception and decision-making architecture has poor interpretability. Therefore, when controlling the vehicle to perform sudden behaviors, it is easy for users riding in autonomous vehicles to be unable to understand the driving behavior of autonomous vehicles, resulting in negative emotions such as confusion, panic, and anxiety, and a poor user experience. Summary of the Invention

[0005] The main objective of this application is to provide an autonomous driving control method, device, electronic device, and storage medium, which aims to solve the technical problem of poor interpretability of end-to-end perception decision-making schemes in related technologies.

[0006] To achieve the above objectives, this application provides an autonomous driving control method. The autonomous driving control method employs a cognitive decision-making model, which includes a cognitive layer and a decision-making layer. The autonomous driving control method includes the following steps:

[0007] Acquire in-vehicle and out-of-vehicle monitoring data, and extract current cognitive layer features from the in-vehicle and out-of-vehicle monitoring data through the cognitive layer;

[0008] The current cognitive layer features are input into the decision layer to determine the autonomous driving control parameters;

[0009] Based on the current cognitive layer features and the autonomous driving control parameters, an explanation of driving behavior is generated.

[0010] This application also provides an autonomous driving control device, wherein a cognitive decision-making model is deployed on the autonomous driving control device, the cognitive decision-making model including a cognitive layer and a decision-making layer, and the autonomous driving control device includes:

[0011] The cognitive module is used to acquire monitoring data inside and outside the vehicle, and to extract current cognitive layer features from the monitoring data inside and outside the vehicle through the cognitive layer.

[0012] The decision module is used to input the current cognitive layer features into the decision layer to determine the autonomous driving control parameters;

[0013] The explanation module is used to generate an explanation of driving behavior based on the current cognitive layer features and the autonomous driving control parameters.

[0014] This application also provides an electronic device, which is a physical device, comprising: a memory, a processor, and a program of the autonomous driving control method stored in the memory and executable on the processor. When the program of the autonomous driving control method is executed by the processor, it can implement the steps of the autonomous driving control method as described above.

[0015] This application also provides a storage medium, which is a computer-readable storage medium, on which a program for implementing an autonomous driving control method is stored. When the program for the autonomous driving control method is executed by a processor, it implements the steps of the autonomous driving control method as described above.

[0016] This application provides an autonomous driving control method, device, electronic device, and storage medium. The autonomous driving control method employs a cognitive decision-making model, which includes a cognitive layer and a decision-making layer. First, by acquiring in-vehicle and out-of-vehicle monitoring data, the cognitive layer extracts current cognitive layer features from the monitoring data, achieving an understanding of the current situation of the autonomous vehicle. Then, by inputting the environmental cognitive features and user demand features into the decision-making layer, autonomous driving control parameters are determined, enabling accurate autonomous driving decisions based on the understanding of the current situation and determining the autonomous driving control parameters. Finally, by generating a driving behavior explanation based on the current cognitive layer features and the autonomous driving control parameters, the purpose of explaining the autonomous driving decisions is achieved. Since the autonomous driving control parameters are determined based on cognitive layer features, there is a certain causal relationship between the autonomous driving control parameters and the cognitive layer features. The reason why users cannot understand the driving behavior of autonomous vehicles is precisely because, before experiencing the sudden driving behavior of the autonomous vehicle, the user's attention may be focused on different things than the driver's attention. Therefore, when experiencing the sudden driving behavior of the autonomous vehicle, what the user perceives is the final decision made by the autonomous vehicle, and they may not see the factors influencing the decision, thus being unable to deduce the reason for the decision, resulting in negative emotions such as confusion, panic, and anxiety. Therefore, this application interprets autonomous driving control parameters based on cognitive layer features, generating explanations of driving behavior. It can proactively interact with users, explaining the reasons behind driving behavior and eliminating user confusion. This overcomes the poor interpretability of end-to-end perception decision-making architectures, which can lead to users experiencing confusion, panic, and anxiety when controlling vehicles to perform sudden actions, resulting in a poor user experience. This application improves the interpretability of end-to-end perception decision-making architectures, reduces negative emotions such as confusion, panic, and anxiety caused by the unknown, and enhances the user experience. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the first embodiment of the autonomous driving control method of this application;

[0020] Figure 2 This is a schematic diagram of the structure of one possible implementation of the cognitive decision-making model in the embodiments of this application;

[0021] Figure 3 This is a flowchart illustrating the second embodiment of the autonomous driving control method of this application.

[0022] Figure 4 This is a flowchart illustrating the third embodiment of the autonomous driving control method of this application;

[0023] Figure 5 This is a schematic diagram of the structure of the automatic driving control device in the embodiments of this application;

[0024] Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the autonomous driving control method in this application embodiment.

[0025] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0026] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Example 1

[0028] This application provides an autonomous driving control method, referring to... Figure 1 In the first embodiment of the autonomous driving control method of this application, the autonomous driving control method adopts a cognitive decision-making model, which includes a cognitive layer and a decision-making layer, and the autonomous driving control method includes the following steps:

[0029] Step S10: Acquire in-vehicle and out-of-vehicle monitoring data, and extract current cognitive layer features from the in-vehicle and out-of-vehicle monitoring data through the cognitive layer;

[0030] The execution subject of the method in this embodiment can be an autonomous driving control device, an autonomous driving control terminal device, or a server. This embodiment takes an autonomous driving control device as an example. The autonomous driving control device can be integrated into terminal devices such as vehicles, in-vehicle terminals, smartphones, and computers with data processing functions.

[0031] In this embodiment, it should be noted that the autonomous driving control method is applied to a cognitive decision model, which is used to realize end-to-end driving behavior control of autonomous vehicles. The cognitive decision model uses an attention mechanism and can be a deep learning model based on the Transformer structure or other deep learning neural network models that use attention mechanisms. The specific design can be made according to the actual situation, and this embodiment does not limit it.

[0032] The cognitive decision-making model includes at least a cognitive layer and a decision layer. The cognitive layer is used to extract, fuse, encode, and reason from the acquired in-vehicle and out-of-vehicle monitoring data to obtain the risk features required by the decision layer. The decision layer is used to plan the driving behavior of the autonomous vehicle in the next time step based on the risk features transmitted by the cognitive layer, so that the autonomous vehicle can avoid risks and drive safely to its destination.

[0033] The in-vehicle and out-of-vehicle monitoring data refers to the data required for autonomous driving control that can be acquired, including external monitoring data and cabin monitoring data. This data can be one or more modalities such as image data, text data, sound data, and signal data. For example, external monitoring data can be external image data collected by a camera, external sound data collected by a microphone, or traffic light countdown data obtained through a communication module. For example, cabin monitoring data can be vehicle speed, acceleration, and other vehicle driving data obtained from the vehicle controller or controller area network bus, or cabin image data collected by a camera, driver image data, cabin sound data collected by a microphone, text data converted from collected speech, or text data extracted from collected images. This in-vehicle and out-of-vehicle monitoring data can be collected in real time, thus carrying real-time information about the status of other road users, the driver's status, road conditions, and other potentially changing information.

[0034] For different modalities of vehicle interior and exterior monitoring data, different models can be used for feature extraction. For example, convolutional neural networks can be used for feature extraction of image data, and long short-term memory networks can be used for feature extraction of time series data. Different models can also be used for feature processing to extract different risk features required by the decision-making layer. Therefore, multiple models can be deployed in the cognitive layer for feature processing. The specific model structure and model type can be determined according to the actual situation. This embodiment does not impose any restrictions on this.

[0035] The cognitive layer features may include at least one of environmental cognitive features and user demand features.

[0036] The environmental perception features are used to characterize the autonomous vehicle's perception of its current environment, such as its perception of traffic facilities, ground obstacles, its own driving status, other traffic participants, the risks of the scene, and the risk items within the scene. The main purpose of autonomous vehicle behavior control is to ensure the autonomous vehicle safely reaches its destination; that is, to control the vehicle to its destination while avoiding risks. In one feasible approach, autonomous driving control can predict high-risk and / or low-risk driving areas for the autonomous vehicle in the next time step, thereby planning the autonomous vehicle's driving behavior to avoid high-risk areas or pass through low-risk areas in the next time step. Since the environmental perception features are an objective representation of the autonomous vehicle's current environment, the more accurate the extracted environmental perception features, the more accurate the risk assessment of the autonomous vehicle's current situation, and the higher the safety of the autonomous driving control. The environmental cognitive features may include scene cognitive features, map features, risk target detection and tracking features, risk target movement features, low-risk driving area features, etc., and may also include other features. The cognitive layer can be a multi-layer structure, that is, the input of any cognitive layer can be the output of other cognitive layers, and the output of any cognitive layer can also be the input of other cognitive layers, thereby enabling the fusion and reasoning of multiple features, capturing the correlation and dependence between the various features extracted by the cognitive layer, and realizing a more comprehensive understanding and deeper logical reasoning of the current driving situation.

[0037] The user demand features are used to characterize the needs of users riding in autonomous vehicles. Examples include user needs regarding the destination, the route to the destination, time constraints, and the desire for a smooth ride to alleviate motion sickness. These user demand features can be captured by detecting user actions and collecting user images, and then extracted from these images. For instance, user images can be captured by a camera; if image recognition technology identifies the user as asleep, this usually implies a need for smooth vehicle operation. Similarly, user voice information can be captured by a microphone; if the user says "drive faster," it indicates a need for acceleration.

[0038] The cognitive layer can employ an attention mechanism, and in one feasible approach, it can adopt a transformer architecture. Using an attention mechanism allows for effective feature extraction, fusion, encoding, and inference of various modalities and types of vehicle monitoring data. It extracts real-time information that may change at the current moment, combines it with historical information from a previous period to form an information sequence, and captures the correlations and dependencies between different types of information. Feature inference is then performed to extract risk features with deep logical information, which are then used by the decision-making layer for autonomous vehicle control. This allows for comprehensive consideration of potentially changing information, multiple risk items, and the interplay between user needs during the prediction of autonomous vehicle driving risks, improving the accuracy of risk prediction. This enables autonomous vehicles to avoid more risks while better meeting user needs, thus enhancing the safety and comfort of autonomous driving control. The correlations and dependencies between different types of information can significantly impact autonomous vehicle control. For example, interactions may occur between traffic participants, between traffic participants and traffic facilities, and between traffic participants and the surrounding environment. For example, at an intersection, this vehicle is traveling east-west, and there is a second vehicle traveling north-south and pedestrians walking east-west nearby. Assuming no interaction between traffic participants, the two vehicles will not collide at their current speeds. However, if the second vehicle slows down or stops to avoid a pedestrian, a collision may occur. Therefore, considering interactions between traffic participants, this vehicle still needs to slow down or plan an alternative route. If the second vehicle is in a turning lane, considering interactions between traffic participants and traffic facilities, if the second vehicle's turning trajectory does not intersect with this vehicle's, this vehicle does not need to slow down. Furthermore, considering interactions between traffic participants and the environment, the complexity of an intersection increases the risk to both people and vehicles, potentially requiring the vehicle to slow down.

[0039] As an example, step S10 includes: collecting sensor data at the current moment as in-vehicle and out-of-vehicle monitoring data using one or more sensors installed on the autonomous vehicle; and communicating with external devices via a communication module on the autonomous vehicle to obtain external data provided by the external devices as in-vehicle and out-of-vehicle monitoring data, such as traffic light signals, traffic accident information, and roadside monitoring data collected by roadside monitoring equipment. The in-vehicle and out-of-vehicle monitoring data is then input into the cognitive layer for multi-level progressive feature processing, including feature extraction, feature fusion, feature encoding, and feature reasoning, to obtain cognitive layer features.

[0040] Optionally, the cognitive layer includes a user feature extraction model; the current cognitive layer features include user demand features, wherein the user demand features include implicit user demand features and explicit user demand features; the in-vehicle and out-of-vehicle monitoring data includes user operation data and user status data;

[0041] The step of extracting current cognitive layer features from the vehicle interior and exterior monitoring data through the cognitive layer includes:

[0042] The user feature extraction model extracts user display requirement features from the user operation data, and / or extracts implicit user requirement features from the user status data.

[0043] In this embodiment, it should be noted that user operation data refers to data generated by the user actively interacting with the autonomous vehicle, such as issuing voice commands to the autonomous vehicle or actively clicking buttons on the in-vehicle terminal display interface. User status data refers to user-related information actively collected by the autonomous vehicle. For example, if the user is asleep and does not actively interact with the autonomous vehicle, the camera can collect the user's image data to identify the user's sleeping state. Similarly, if the user is on the phone with a friend expressing nervousness and apprehension about riding in the autonomous vehicle, and does not actively interact with the vehicle, the microphone can collect the user's voice data to identify the user's nervousness and apprehension.

[0044] As an example, the user operation data can be input into the user feature extraction model to perform multi-level progressive feature processing such as feature extraction, feature fusion, feature encoding, and feature reasoning to obtain user display requirement features. Alternatively, the user status data can be input into the user feature extraction model to perform multi-level progressive feature processing such as feature extraction, feature fusion, feature encoding, and feature reasoning to obtain user implicit requirement features.

[0045] In this embodiment, users can actively interact with autonomous vehicles and make requests. Autonomous vehicles can also respond to user interactions and meet user requests, improving the interactivity between autonomous vehicles and users. Furthermore, autonomous vehicles can proactively identify user needs and take autonomous driving actions that better meet those needs, thereby reducing user discomfort.

[0046] Optionally, the cognitive layer includes a scene cognition model, a map building model, a risk target detection and tracking model, and a motion prediction model; the current cognitive layer features include environmental cognition features, wherein the environmental cognition features include scene cognition features, map features, risk target detection and tracking features, and risk target motion features;

[0047] The step of extracting current cognitive layer features from the vehicle interior and exterior monitoring data through the cognitive layer includes:

[0048] Step A10: Extract scene recognition features from the vehicle interior and exterior monitoring data using the scene recognition model, extract map features from the vehicle interior and exterior monitoring data using the map construction model, and extract risk target detection and tracking features from the vehicle interior and exterior monitoring data using the risk target detection and tracking model.

[0049] Step A20: Using the motion prediction model and an attention mechanism, based on the scene cognitive features, the map features, and the risk target detection and tracking features, the motion prediction of the risk target is performed to obtain the motion features of the risk target.

[0050] In this embodiment, it should be noted that the cognitive layer includes a scene cognition model, a map building model, a risk target detection and tracking model, and a motion prediction model. A risk target refers to an object that may pose a risk to the autonomous vehicle's operation, and can be one or more of traffic participants, traffic infrastructure, etc. The scene cognition features, map features, and risk target detection and tracking features output by the scene cognition model, the map building model, and the risk target detection and tracking model all have a certain impact on predicting the motion of the risk target in the next time step. Furthermore, there are certain correlations and dependencies among the scene cognition features, map features, and risk target detection and tracking features. Therefore, they can all be used as inputs to the motion prediction model. The motion prediction model uses an attention mechanism to capture the correlations and dependencies between these features, and performs further feature fusion, encoding, and reasoning to determine the risk target's motion characteristics.

[0051] The risk features include scene recognition features, map features, risk target detection and tracking features, and risk target motion features. Scene recognition features are used to characterize the risk perception of the vehicle's current scene, such as attention distribution and semantic understanding of risk items in the scene. Map features are used to characterize map information such as ground facilities and road elements in the scene where the vehicle is located. Risk target detection and tracking features are used to characterize the identification of risk targets and the location information of risk targets within a certain period of time before the current moment. Risk target motion features are used to characterize the motion information of risk targets within a certain period of time after the current moment. Motion information can include location information, trajectory information, motion intention, and motion trend.

[0052] As an example, steps A10-A20 include: inputting the vehicle interior and exterior monitoring data into the scene cognition model, performing multi-level progressive feature processing such as feature extraction, feature fusion, feature encoding, and feature reasoning to extract scene cognition features; simultaneously, inputting the vehicle interior and exterior monitoring data into the map construction model, performing multi-level progressive feature processing such as feature extraction, feature fusion, feature encoding, and feature reasoning to extract map features; simultaneously, inputting the vehicle interior and exterior monitoring data into the risk target detection and tracking model, performing multi-level progressive feature processing such as feature extraction, feature fusion, feature encoding, and feature reasoning to extract risk target detection and tracking features. Furthermore, the scene cognition features, the map features, and the risk target detection and tracking features are input into the motion prediction model. Through the motion prediction model, using an attention mechanism, the correlation and dependency between the scene cognition features, the map features, and the risk target detection and tracking features are captured. Based on these correlations and dependencies, risk target movement prediction can be performed more accurately, determining the risk target movement features of at least one risk target.

[0053] In one feasible embodiment, the risk target detection and tracking feature includes at least one risk target detection and tracking sub-feature; the motion prediction model includes an attention layer, a multilayer perceptron layer, and a prediction layer;

[0054] The step of using the motion prediction model with an attention mechanism to predict the motion of risk targets based on the scene cognitive features, the map features, and the risk target detection and tracking features, and obtaining the motion features of the risk targets, includes:

[0055] Step A21: Input the scene cognition features, the map features, and the risk target detection and tracking features into the attention layer, and use the attention mechanism to determine the risk target interaction features between each of the risk target detection and tracking sub-features, the scene interaction features between each of the risk target detection and tracking sub-features and the scene cognition features, and the map interaction features between each of the risk target detection and tracking sub-features and the map features.

[0056] Step A22: Input the risk target interaction features, the scene interaction features, and the map interaction features into the multilayer perceptron layer to obtain scene risk features and motion query features;

[0057] Step A23: Input the scene risk features and the motion query features into the prediction layer to perform risk target motion prediction and obtain risk target motion features.

[0058] In this embodiment, it should be noted that for complex scenarios, multiple risk targets may be involved. When multiple risk targets are identified, the risk target detection and tracking model can identify and track them simultaneously. The resulting risk target detection and tracking features will contain relevant information about multiple risk targets; that is, the risk target detection and tracking features include at least one risk target detection and tracking sub-feature. In this case, motion prediction can be performed for each risk target separately. By employing an attention mechanism, the correlation and dependency between each risk target and other risk targets, between each risk target and map elements, and between each risk target and scene perception can be captured. Based on these correlations and dependencies, risk target motion prediction can be performed more accurately for each risk target, determining the risk target motion characteristics of each risk target.

[0059] Scene risk features and motion query features represent the risks that already exist in the current scene. Based on scene risk features and motion query features, we can further predict the motion information of the risk target after the current moment.

[0060] As an example, steps A21-A23 include: inputting the scene cognition features, the map features, and the risk target detection and tracking features into the attention layer; employing an attention mechanism to capture the dependencies between each risk target detection and tracking sub-feature to determine the mutual influence between each risk target and other risk targets, generating risk target interaction features; capturing the dependencies between each risk target detection and tracking sub-feature and the scene cognition features to determine the mutual influence between each risk target and scene cognition, generating scene interaction features; and capturing the dependencies between each risk target detection and tracking sub-feature and the map features to determine the mutual influence between each risk target and map elements, generating scene interaction features. Then, the risk target interaction features, the scene interaction features, and the map interaction features are concatenated and input into the multilayer perceptron layer to obtain scene risk features and motion query features. Finally, the scene risk features and the motion query features are input into the prediction layer to perform risk target motion prediction for each risk target, obtaining risk target motion features for each risk target.

[0061] In one feasible approach, the multilayer perceptron layer can also decode scene risk features and motion query features, and output the decoded scene risk prediction information and motion information of risk targets as intermediate results for users to view.

[0062] Step S20: Input the current cognitive layer features into the decision layer to determine the autonomous driving control parameters;

[0063] In this embodiment, it should be noted that the autonomous driving control parameters may include driving trajectory, vehicle control parameters, etc. The driving trajectory refers to the trajectory of the autonomous vehicle after a period of time following the current moment, and the autonomous vehicle can be controlled to drive based on the driving trajectory. The vehicle control parameters include speed, accelerator pedal opening, steering wheel angle, etc.

[0064] As an example, step S20 includes: inputting the environmental perception features and the user demand features into the decision layer to determine the autonomous driving control parameters of the autonomous vehicle at the current moment or for a period of time after the current moment.

[0065] Step S30: Generate an explanation of driving behavior based on the current cognitive layer features and the autonomous driving control parameters.

[0066] In this embodiment, it should be noted that the autonomous driving control parameters may include driving trajectory, vehicle control parameters, etc. The driving trajectory refers to the trajectory of the autonomous vehicle after a period of time following the current moment, and the autonomous vehicle can be controlled to drive based on the driving trajectory. The vehicle control parameters include speed, accelerator pedal opening, steering wheel angle, etc.

[0067] As an example, step S30 includes: decoding information about factors influencing the output of autonomous driving control parameters by the decision layer from the current cognitive layer features, and fusing the autonomous driving control parameters and the information about the influencing factors corresponding to the autonomous driving control parameters to generate an explanation of driving behavior.

[0068] In one feasible approach, after the step of generating a driving behavior explanation based on the current cognitive layer features and the autonomous driving control parameters, the method may further include: outputting the behavior explanation information. That is, the driving behavior explanation is output through images, sound, text, etc., so that the user can understand the reason for the driving behavior. For example, if an autonomous vehicle needs to stop at a red light intersection, the autonomous driving control parameters can be "stop," and the influencing factors can be the red light at the intersection ahead and the location information of the stop line. By aggregating these two, a driving behavior explanation can be generated: "Because the light ahead is red, the vehicle stops before the stop line." This can be output via a speaker as a voice announcement: "Because the light ahead is red, the vehicle stops before the stop line."

[0069] In one feasible approach, the driving behavior explanation can be output before the autonomous vehicle executes the autonomous driving control parameters. This allows the user to anticipate the driving behavior that the vehicle is about to perform, thus preventing panic caused by sudden driving behaviors of the autonomous vehicle.

[0070] Optionally, the autonomous driving control parameters include multiple autonomous driving control sub-parameters;

[0071] The step of generating an explanation of driving behavior based on the current cognitive layer features and the autonomous driving control parameters includes:

[0072] Step S31: Based on the current cognitive layer features, determine the target decision features corresponding to each of the autonomous driving control sub-parameters.

[0073] Step S32: Aggregate each of the autonomous driving control sub-parameters with their respective target decision features to obtain a step-by-step interpretation of driving behavior;

[0074] Step S33: Aggregate the step-by-step explanations of each driving behavior to obtain the driving behavior explanation.

[0075] In this embodiment, it should be noted that driving behavior may be achievable in one step or may require multiple steps. For driving behaviors requiring multiple steps, the corresponding autonomous driving control sub-parameters for each step can be determined and then arranged sequentially in chronological order to form an autonomous driving control parameter sequence. For example, for the driving behavior of accelerating and changing lanes, the autonomous driving control parameter sequence can be {τ1 (turn on right turn signal), τ2 (accelerate), τ3 (change lanes to the right)}, where τ represents the autonomous driving control sub-parameter. For driving behaviors requiring multiple steps, each step has its own reason for execution and key influencing factors; therefore, each step can be explained more accurately step by step.

[0076] As an example, steps S31-S33 include: decoding information from the current cognitive layer features that influences the output of each of the autonomous driving control sub-parameters of the decision layer; concatenating the information of the influencing factors corresponding to each of the autonomous driving control sub-parameters into target decision features; aggregating each of the autonomous driving control sub-parameters with their respective target decision features to generate a step-by-step explanation of driving behavior; and further aggregating each of the step-by-step explanations of driving behavior into a driving behavior explanation.

[0077] In one feasible approach, the target decision features may include shallow target decision features and deep target decision features. The step of determining the target decision features corresponding to each of the autonomous driving control sub-parameters based on the current cognitive layer features may include: determining the driving behavior step-by-step explanation type corresponding to each of the autonomous driving control sub-parameters based on the current cognitive layer features, using a classification model or an attention mechanism, wherein the driving behavior step-by-step explanation type includes shallow explanation and deep explanation; if the driving behavior step-by-step explanation type is determined to be shallow explanation, determining the shallow target decision features corresponding to each of the autonomous driving control sub-parameters based on the current cognitive layer features; if the driving behavior step-by-step explanation type is determined to be deep explanation, determining the deep target decision features corresponding to each of the autonomous driving control sub-parameters based on the current cognitive layer features. Among them, shallow explanation refers to the spatial explanation of driving behavior, while deep explanation refers to the spatiotemporal explanation of driving behavior. The series of steps corresponding to each driving behavior have different degrees of importance in the explanation of driving behavior. For example, in the driving behavior of accelerating to change lanes, since changing lanes usually involves decelerating first and then changing lanes, users may have great confusion about the "acceleration" step when accelerating to change lanes. Moreover, the user's need for explanation of driving behavior also varies depending on the actual situation encountered by autonomous vehicles in different scenarios. For example, the driving behavior of decelerating to change lanes is in line with conventional cognition, so it is not easy to cause confusion and a shallow explanation is sufficient. However, the driving behavior of accelerating to change lanes usually only occurs in special scenarios, so a deeper explanation of the cause and effect may be required.

[0078] Optionally, the step of aggregating the step-by-step interpretations of each driving behavior to obtain a driving behavior interpretation includes:

[0079] Step S331: Obtain the step-by-step interpretation weights corresponding to each of the autonomous driving control sub-parameters;

[0080] Step S332: Based on the weights of each step-by-step explanation, perform weighted aggregation on each step-by-step explanation of the driving behavior to obtain the driving behavior explanation.

[0081] In this embodiment, it should be noted that users have different actual needs for interpreting different steps. For example, in the automatic driving control parameter sequence {τ1 (turn on right turn signal), τ2 (accelerate), τ3 (change lanes to the right)} corresponding to the driving behavior of accelerating and changing lanes, the user's perception of whether the turn signal is turned on is not obvious. However, the reason for changing lanes to the right is usually quite obvious, and the user can understand it by looking up. Acceleration may not necessarily occur during the lane change process. For safety reasons, it is usually necessary to first decelerate and turn on the turn signal to warn the vehicle behind, then change lanes, and then accelerate after changing lanes. Therefore, accelerating before changing lanes is likely to arouse the user's alertness. Therefore, the interpretation of the driving behavior of accelerating and changing lanes is... Explaining acceleration is of higher importance. Therefore, by assigning weights, the explanation of driving behavior can be prioritized. The order and level of detail of the step-by-step explanation of driving behavior can also be determined based on the weights. The weights are positively correlated with the order and the level of detail. This allows the explanation of driving behavior with higher weights to be output earlier and in more detail, while the explanation of driving behavior with lower weights can be output later and more simply. This ensures that the more important steps are explained in detail and in a timely manner, while avoiding the user's dissatisfaction caused by explaining every step in detail.

[0082] The step-by-step interpretation weight refers to the importance of interpreting each autonomous driving control sub-parameter in the autonomous driving control parameter sequence relative to interpreting the driving behavior corresponding to the entire autonomous driving control parameter sequence. It can be pre-set manually or through a corresponding model based on test results and actual needs, or it can be determined using an attention mechanism based on the step-by-step interpretation of each driving behavior and the attention distribution characteristics determined in the previous time step. This embodiment does not impose any limitations on this.

[0083] As an example, steps S331-S332 include: obtaining the step-by-step interpretation weights corresponding to each of the currently determined autonomous driving control sub-parameters, and performing weighted aggregation of each step-by-step interpretation of driving behavior based on each step-by-step interpretation weight to obtain the driving behavior interpretation.

[0084] In one feasible approach, the step of employing an attention mechanism to determine the step-by-step interpretation weights corresponding to each of the autonomous driving control sub-parameters, based on the step-by-step interpretation of each of the driving behaviors and the attention distribution characteristics of the previous time step, can be expressed as follows:

[0085]

[0086] Where J refers to the interpretation of driving behavior, β tyt refers to the step-by-step interpretation weight corresponding to action time t, and yt refers to the step-by-step interpretation of driving behavior corresponding to action time t, where t∈[t0,T].

[0087] Optionally, the step of obtaining the step-by-step interpretation weights corresponding to each of the autonomous driving control sub-parameters includes:

[0088] Step S3311: Obtain the attention distribution features of the previous time step;

[0089] Step S3312: Using an attention mechanism, based on the step-by-step interpretation of each driving behavior and the attention distribution characteristics of the previous time step, determine the step-by-step interpretation weights corresponding to each of the autonomous driving control sub-parameters.

[0090] As an example, steps S3311-S3312 include: obtaining the attention distribution features of the previous time step, and then using an attention mechanism to perform feature fusion, feature encoding, feature reasoning, etc. on the step-by-step interpretation of each driving behavior and the attention distribution features of the previous time step, determining the step-by-step interpretation weight of the current time step, and determining the step-by-step interpretation weight of the current time step as the step-by-step interpretation weight corresponding to each of the autonomous driving control sub-parameters in the currently determined autonomous driving control parameters.

[0091] In one feasible approach, refer to Figure 2 The autonomous driving control device includes a cockpit domain and an intelligent driving domain human-machine interface, a cognitive decision model, and a vehicle driving behavior representation module. The cockpit domain and intelligent driving domain human-machine interface are used to receive user demand information and environmental cognition information obtained from the cockpit domain and intelligent driving domain. The cognitive decision model is used to determine autonomous driving control parameters. The vehicle driving behavior representation module is used to control the autonomous vehicle to perform corresponding driving behaviors based on the autonomous driving control parameters. After performing a driving behavior, the device can obtain user feedback information on the driving behavior through the cockpit domain and intelligent driving domain human-machine interface. This cycle allows the autonomous vehicle to perform driving behaviors adapted to user needs.

[0092] In this embodiment, the autonomous driving control method employs a cognitive decision-making model, which includes a cognitive layer and a decision-making layer. First, by acquiring in-vehicle and out-of-vehicle monitoring data, the cognitive layer extracts current cognitive layer features from this data, achieving an understanding of the current situation of the autonomous vehicle. Then, by inputting the environmental cognitive features and user demand features into the decision-making layer, autonomous driving control parameters are determined, enabling accurate autonomous driving decisions based on the understanding of the current situation and the determination of autonomous driving control parameters. Finally, by generating an explanation of driving behavior based on the current cognitive layer features and the autonomous driving control parameters, the purpose of explaining the autonomous driving decisions is achieved. Since the autonomous driving control parameters are determined based on cognitive layer features, there is a certain causal relationship between the autonomous driving control parameters and the cognitive layer features. The reason why users cannot understand the driving behavior of autonomous vehicles is precisely because, before experiencing the sudden driving behavior of the autonomous vehicle, their attention may be focused on different things than the driver's attention. Therefore, when experiencing the sudden driving behavior of the autonomous vehicle, what they perceive is the final decision made by the autonomous vehicle, and they may not see the factors influencing the decision, thus failing to deduce the reason for the decision, leading to negative emotions such as confusion, panic, and anxiety. Therefore, this application interprets autonomous driving control parameters based on cognitive layer features, generating explanations of driving behavior. It can proactively interact with users, explaining the reasons behind driving behavior and eliminating user confusion. This overcomes the poor interpretability of end-to-end perception decision-making architectures, which can lead to users experiencing confusion, panic, and anxiety when controlling vehicles to perform sudden actions, resulting in a poor user experience. This application improves the interpretability of end-to-end perception decision-making architectures, reduces negative emotions such as confusion, panic, and anxiety caused by the unknown, and enhances the user experience.

[0093] Example 2

[0094] Furthermore, in the second embodiment of this application, content that is the same as or similar to the above embodiments can be referred to the above description, and will not be repeated hereafter. Based on this, refer to... Figure 3 The current cognitive layer features include external environmental features; the target decision features include shallow target decision features.

[0095] The step of determining the target decision features corresponding to each of the autonomous driving control sub-parameters based on the current cognitive layer features includes:

[0096] Step B10: Based on the current cognitive layer features, determine at least one shallow sub-parameter to be explained from each of the autonomous driving control sub-parameters, and the spatial attention weight distribution corresponding to each of the shallow sub-parameters to be explained.

[0097] In this embodiment, it should be noted that more explanations of autonomous driving behavior are not necessarily better. Explaining the driving behavior of an autonomous vehicle when the user cannot understand it can effectively alleviate the user's negative emotions. However, if detailed explanations are still given for easily understood driving behaviors, on the one hand, more detailed explanations will require more analysis and consume more computing power. On the other hand, giving very detailed explanations for easily understood driving behaviors may easily lead to users feeling that it is verbose, noisy, and unable to distinguish the key points, thus generating negative emotions.

[0098] The spatial attention weight distribution is used to characterize the attention distribution required for an autonomous vehicle to achieve safe driving in the current scenario and current step. Positions with high attention weights are significant for the task.

[0099] As an example, step B10 includes: determining at least one shallow sub-parameter to be explained from each of the autonomous driving control sub-parameters based on the current cognitive layer features by means of a classification model or an attention mechanism; simultaneously, outputting the spatial attention weight distribution corresponding to each of the shallow sub-parameters to be explained based on the current cognitive layer features by an activation function.

[0100] Step B20: Determine the shallow target decision features corresponding to each of the shallow sub-parameters to be explained based on the spatial attention weight distribution and the external environment features.

[0101] As an example, step B20 includes: fusing the spatial attention weight distribution and the corresponding external environment features of each of the shallow sub-parameters to be explained to obtain the shallow target decision features corresponding to each of the shallow sub-parameters to be explained.

[0102] In one possible implementation, the step of determining the shallow target decision features corresponding to each of the shallow sub-parameters to be explained based on the spatial attention weight distribution and the external environment features can be expressed as:

[0103] Where 's' represents the shallow layer. This refers to the shallow, uninterpreted sub-parameters at time t. Let be the spatial attention weight distribution map of the i-th autonomous driving control sub-parameter at time t, and satisfy . x t,iLet i be the external environmental feature corresponding to the i-th autonomous driving control sub-parameter at time t, where i = {1, 2, ..., l} and l refers to the number of autonomous driving control sub-parameters.

[0104] Optionally, before the step of acquiring in-vehicle and out-of-vehicle monitoring data and extracting current cognitive layer features from the in-vehicle and out-of-vehicle monitoring data through the cognitive layer, the method further includes:

[0105] Step C10: Obtain the monitoring data inside and outside the training sample vehicle and the autonomous driving behavior label, and extract the current cognitive layer features of the training sample from the monitoring data inside and outside the training sample vehicle through the cognitive layer;

[0106] Step C20: Input the current cognitive layer features of the training samples into the decision layer to determine multiple training sample autonomous driving control sub-parameters;

[0107] Step C30: Based on the current cognitive layer features of the training samples, determine at least one shallow unexplained sub-parameter of each training sample from the autonomous driving control sub-parameters of each training sample, and the spatial attention weight distribution of each training sample corresponding to the shallow unexplained sub-parameter of each training sample.

[0108] Step C40: Determine a shallow attention loss based on the autonomous driving control sub-parameters of each training sample, the autonomous driving behavior labels of each training sample, and the spatial attention weight distribution of each training sample; and iteratively optimize the cognitive decision-making model based on the shallow attention loss.

[0109] In this embodiment, it should be noted that before applying the cognitive decision-making model, model training is required. During each iteration of model training, the cognitive decision-making model can be updated based on the calculated model loss until the model converges. The training process of the cognitive decision-making model can be performed on the final application device. Considering the computational power requirements of model training, the trained cognitive decision-making model can also be deployed to the application device after training on other devices. For example, the cognitive decision-making model can be trained on a server and then deployed on an autonomous vehicle. The method of making cognitive decisions based on training samples during model training is similar to the model application process. Content that is the same as or similar to the above embodiment can be referred to the above description and will not be repeated hereafter. When the target decision features include shallow target decision features, since autonomous driving behavior prediction is closely related to the spatial attention weight distribution, the detection accuracy of the spatial attention weight distribution of the training samples can be used to correct the autonomous driving behavior prediction loss, reducing the error of calculating the driving behavior prediction loss alone.

[0110] As an example, steps C10-C40 include: acquiring training sample in-vehicle and out-of-vehicle monitoring data and autonomous driving behavior labels; extracting current cognitive layer features of the training samples from the training sample in-vehicle and out-of-vehicle monitoring data through the cognitive layer; inputting the current cognitive layer features of the training samples into the decision layer to determine multiple training sample autonomous driving control sub-parameters; determining at least one shallow unexplained sub-parameter of the training samples from each of the training sample autonomous driving control sub-parameters based on the current cognitive layer features of the training samples, and the spatial attention weight distribution of the training samples corresponding to each of the shallow unexplained sub-parameters; and determining at least one... The initial prediction loss is calculated for each action moment, and the spatial attention loss is determined based on the spatial attention weight distribution of the training samples corresponding to each action moment. The initial prediction loss is then corrected based on the spatial attention loss to obtain the shallow attention loss. Subsequently, the convergence of the cognitive decision model is determined based on the shallow attention loss. If the cognitive decision model is determined to be converged, the model training is considered complete, and the trained cognitive decision model is obtained. If the cognitive decision model is determined not to be converged, the cognitive decision model can be updated once based on the shallow attention loss using the gradient descent method, and the process returns to the step of obtaining the training sample vehicle interior and exterior monitoring data and autonomous driving behavior labels for the next round of training.

[0111] In one feasible approach, the correction of the corresponding initial prediction loss based on the spatial attention loss can be achieved by calculating the attention weight entropy value based on the spatial attention weight distribution of the training samples corresponding to each action time, determining the attention weight entropy value as the spatial attention loss, and aggregating the spatial attention loss with each of the initial prediction losses to correct the initial prediction losses. It should be noted that the attention weight distribution represents the importance of each element in the input image, and the attention weight entropy value refers to the entropy value of the attention weight distribution, used to measure the uncertainty of the model's attention to each element in the input image. If the attention weight distribution is concentrated on one or a few elements, the calculated attention weight entropy value will be relatively low, indicating that the model's attention to the input image is highly certain. Conversely, if the attention weight distribution is relatively uniform, the calculated attention weight entropy value will be relatively high, indicating that the model's attention to the input image is uncertain.

[0112] Optionally, the step of determining the shallow attention loss based on the autonomous driving control sub-parameters of each training sample, the autonomous driving behavior labels of each training sample, and the spatial attention weight distribution of each training sample includes:

[0113] The autonomous driving control sub-parameters of each training sample, the autonomous driving behavior labels of each training sample, and the spatial attention weight distribution of each training sample are input into a shallow attention loss function to determine the shallow attention loss, wherein the shallow attention loss function is:

[0114]

[0115] in, For shallow attention loss, τ t The label for the autonomous driving behavior at time t. Let λ be the autonomous driving control sub-parameters of the training sample at time t, H be the entropy function used to calculate the entropy of the spatial attention weight distribution of the training sample, and λ be the value of λ. s To preset the first hyperparameter, Let be the spatial attention weight distribution of the training samples at time t, where t∈[t0,T].

[0116] In this embodiment, a shallow explanation mechanism can be used to provide a shallow spatial explanation of driving behaviors that are easy for users to understand. On the one hand, spatial explanations only need to utilize the cognitive features of the current moment without the need for temporal analysis. On the other hand, this reduces computational consumption and improves the efficiency of generating explanations for driving behaviors. Furthermore, it simplifies the explanations and avoids causing users to feel annoyed by the explanations being verbose, noisy, or unclear about the key points. For example, in the driving behavior of "stopping at a traffic light intersection," the key influencing factor for the decision is the red light ahead. A large weight in the spatial attention weight distribution will also be concentrated at the location of the traffic light. Therefore, a shallow spatial explanation of "because the light ahead is red, the vehicle stops before the stop line" is sufficient.

[0117] Example 3

[0118] Furthermore, in the third embodiment of this application, content that is the same as or similar to the above embodiments can be referred to the above description, and will not be repeated hereafter. Based on this, refer to... Figure 4 The current cognitive layer features include external environmental features; the target decision features include shallow target decision features.

[0119] The step of determining the target decision features corresponding to each of the autonomous driving control sub-parameters based on the current cognitive layer features includes:

[0120] Step D10: Based on the current cognitive layer features, determine at least one deep-level unexplained sub-parameter from each of the autonomous driving control sub-parameters;

[0121] In this embodiment, it should be noted that if the driving behavior decisions made in complex scenarios are only explained at a superficial spatial level, users need to think about and associate the spatiotemporal logic themselves. Furthermore, due to the lack of some historical information, such as a cat suddenly running across the road, by the time the user feels the brakes and looks up, the cat may have already run away. As a result, the user may not be able to accurately understand the driving behavior and may experience negative emotions such as confusion, panic, and anxiety, leading to a poor user experience.

[0122] As an example, step D10 includes: determining at least one deep-level unexplained sub-parameter from each of the autonomous driving control sub-parameters based on the current cognitive layer features, using a classification model or an attention mechanism.

[0123] Step D20: Obtain the historical cognitive layer features extracted before the current moment;

[0124] Step D30: Using a spatiotemporal attention mechanism, based on the current cognitive layer features and the historical cognitive layer features, extract the deep target decision features corresponding to each of the deep unexplained sub-parameters.

[0125] In this embodiment, it should be noted that the historical cognitive layer features refer to the cognitive layer features extracted before the current moment. Deep interpretation addresses driving behaviors that cannot be clearly and intuitively explained through spatial dimensions, providing explanations in both time and space. The deep target decision features contain spatial and temporal information about factors influencing decision-making, such as motion trend information and trajectory information. For temporal interpretation, whether analyzing the past to the present or predicting the future, trend analysis based on historical cognitive layer features is necessary to more accurately identify the key factors influencing driving behavior. Therefore, it is necessary to acquire historical cognitive layer features and combine them with current cognitive layer features for analysis. For example, in a scenario where a vehicle accelerates and changes lanes to avoid an obstacle in a mixed-traffic situation, there are many influencing factors involved, and the vehicle performs an unconventional operation of accelerating before changing lanes. If only a shallow spatial explanation is given, such as "accelerating to change lanes to avoid an obstacle because there is someone in front," users will still have doubts about the autonomous vehicle's driving behavior, since the vehicle should usually slow down to give way when there is someone in front. In this case, an explanation in both time and space is needed. The generated deeper explanation could be, "Because the pedestrian's trajectory will intersect with the vehicle's trajectory, there is a risk of collision. The vehicle cannot slow down to avoid a collision, so it accelerates to change lanes to avoid the obstacle." It can also output a spatial attention weight distribution map with a larger weight concentrated on the location of the pedestrian who cannot be avoided, allowing users to see the pedestrian who cannot be avoided more intuitively and thus better understand the driving behavior.

[0126] As an example, step D20 includes: using a spatiotemporal attention mechanism to perform feature fusion, feature encoding, feature reasoning, and other analyses on the current cognitive layer features, the historical cognitive layer features, and each of the deep unexplained sub-parameters, to determine the deep target decision features corresponding to each of the deep unexplained sub-parameters.

[0127] Optionally, the step of determining a shallow attention loss based on the autonomous driving control sub-parameters of each training sample, the autonomous driving behavior labels of each training sample, and the spatial attention weight distribution of each training sample, and iteratively optimizing the cognitive decision-making model based on the shallow attention loss, includes:

[0128] Step C41: Based on the current cognitive layer features of the training samples, determine at least one deep unexplained sub-parameter of the training sample from the autonomous driving control sub-parameters of each training sample, and the spatial attention weight distribution of the training sample corresponding to each deep unexplained sub-parameter of the training sample.

[0129] Step C42: Obtain the historical cognitive layer features of the training samples extracted before the current moment, and use a spatiotemporal attention mechanism to determine the spatiotemporal attention weight distribution of each deep unexplained sub-parameter of the training sample based on the current cognitive layer features of the training sample and the historical cognitive layer features of the training sample.

[0130] Step C43: Determine the deep attention loss based on the difference between the spatial attention weight distribution of each training sample and the spatiotemporal attention weight distribution of each training sample corresponding to the deep unexplained sub-parameters; determine the shallow attention loss based on the difference between the spatial attention weight distribution of each training sample and the autonomous driving control sub-parameters of each training sample and their corresponding autonomous driving behavior labels.

[0131] Step C44: Aggregate the deep attention loss and the shallow attention loss to obtain the autonomous driving behavior prediction loss, and iteratively optimize the cognitive decision-making model based on the autonomous driving behavior prediction loss.

[0132] In this embodiment, it should be noted that when the target decision features include deep target decision features, since it is necessary to determine the key factors affecting the decision whether the analysis is performed in the spatial dimension or in both the temporal and spatial dimensions, this key factor is usually determined based on the attention weight distribution. Therefore, by comparing the difference between the spatial attention weight distribution determined from the spatial dimension and the spatiotemporal attention weight distribution determined from the spatiotemporal dimension, the spatiotemporal attention mechanism can refer to the significant attention object, that is, the position with higher weight in the spatial attention weight distribution determined in the spatial dimension, and interpret the vehicle driving behavior output by the cognitive decision model in both time and space dimensions in the spatiotemporal dimension.

[0133] As an example, steps C41-C44 include: determining at least one deep-level unexplained sub-parameter of each training sample from the autonomous driving control sub-parameters of each training sample, and the spatial attention weight distribution of each deep-level unexplained sub-parameter of the training sample, based on the current cognitive layer features of the training sample; obtaining the historical cognitive layer features of the training sample extracted before the current moment, and using a spatiotemporal attention mechanism, determining the spatiotemporal attention weight distribution of each deep-level unexplained sub-parameter of the training sample based on the current cognitive layer features and the historical cognitive layer features of the training sample; and determining deep attention based on the difference between the spatial attention weight distribution of each deep-level unexplained sub-parameter of the training sample and the spatiotemporal attention weight distribution of each deep-level unexplained sub-parameter of the training sample. The loss is determined based on the spatial attention weight distribution of each training sample and the difference between the autonomous driving control sub-parameters of each training sample and their corresponding autonomous driving behavior labels. The deep attention loss and the shallow attention loss are then aggregated to obtain the autonomous driving behavior prediction loss. Next, based on the autonomous driving behavior prediction loss, it is determined whether the cognitive decision-making model has converged. If the cognitive decision-making model has converged, the model training is considered complete, and a trained cognitive decision-making model is obtained. If the cognitive decision-making model has not converged, the cognitive decision-making model can be updated once using the gradient descent method based on the autonomous driving behavior prediction loss, and the process returns to the step of obtaining the in-vehicle and out-of-vehicle monitoring data and autonomous driving behavior labels of the training samples for the next round of training.

[0134] Optionally, the step of determining the deep attention loss based on the difference between the spatial attention weight distribution of the training samples corresponding to each deep unexplained sub-parameter of the training samples and the spatiotemporal attention weight distribution of the training samples corresponding to each deep unexplained sub-parameter includes:

[0135] The spatial attention weight distribution and the spatiotemporal attention weight distribution of each training sample are input into the deep attention loss function to determine the deep attention loss, wherein the deep attention loss function is:

[0136]

[0137] Among them, D KL Represents the divergence function. Let be the spatial attention weight distribution of the deep unexplained sub-parameters of the training sample at time t. Let λ be the spatiotemporal attention weight distribution of the training sample corresponding to the deep unexplained sub-parameters of the training sample at time t. d The second hyperparameter is a preset, i∈[1,l].

[0138] Optionally, the step of aggregating the deep attention loss and the shallow attention loss to obtain the autonomous driving behavior prediction loss includes:

[0139] Step E10: Based on the shallow unexplained sub-parameters of each training sample and the current cognitive layer features of the training sample, generate at least one step-by-step explanation of the shallow driving behavior of the training sample; based on the deep unexplained sub-parameters of each training sample and the current cognitive layer features of the training sample, generate at least one step-by-step explanation of the deep driving behavior of the training sample.

[0140] Step E20: Aggregate the step-by-step explanation of shallow driving behavior of each training sample, the sub-parameters to be explained of shallow driving behavior of each training sample, the step-by-step explanation of deep driving behavior of each training sample, and the sub-parameters to be explained of deep driving behavior of each training sample to obtain the explanation of driving behavior of the training sample.

[0141] Step E30: Obtain the historical training sample driving behavior explanation, historical training sample spatial attention weight distribution, and historical training sample spatiotemporal attention weight distribution from the previous time step. Based on the historical training sample driving behavior explanation, the historical training sample spatial attention weight distribution, and the historical training sample spatiotemporal attention weight distribution, as well as the training sample driving behavior explanation, training sample spatial attention weight distribution, and training sample spatiotemporal attention weight distribution from the current time step, determine the minimum negative log-likelihood loss.

[0142] Step E40: Aggregate the deep attention loss, the shallow attention loss, and the minimized negative log-likelihood loss to obtain the autonomous driving behavior prediction loss.

[0143] In this embodiment, it should be noted that, in order to uniformly optimize the cognitive decision-making model, minimize the overall loss of the cognitive decision-making model, and obtain the best explanatory performance, shallow attention loss and deep attention loss are combined for training, and the model is fine-tuned by minimizing the negative log-likelihood function.

[0144] As an example, steps E10-E40 include: generating a step-by-step explanation of shallow driving behavior for each training sample based on the shallow unexplained sub-parameters of each training sample and the current cognitive layer features of the training sample; generating a step-by-step explanation of deep driving behavior for each training sample based on the deep unexplained sub-parameters of each training sample and the current cognitive layer features of the training sample; further, aggregating the step-by-step explanations of shallow driving behavior of each training sample with their corresponding shallow unexplained sub-parameters of the training sample into a shallow step-by-step explanation of driving behavior of the training sample; and aggregating the step-by-step explanations of deep driving behavior of each training sample with their corresponding deep unexplained sub-parameters of the training sample into a deep step-by-step explanation of driving behavior of the training sample; and further aggregating the step-by-step explanations of deep driving behavior of each training sample with their corresponding deep unexplained sub-parameters of the training sample into a deep step-by-step explanation of driving behavior of the training sample. The shallow step-by-step explanation of driving behavior and the deep step-by-step explanation of driving behavior of each training sample are aggregated to obtain the driving behavior explanation of the training samples. Then, the historical driving behavior explanation of the training samples, the spatial attention weight distribution of the historical training samples, and the spatiotemporal attention weight distribution of the historical training samples from the previous time step are obtained. The historical driving behavior explanation of the training samples from the previous time step, the spatial attention weight distribution of the historical training samples, and the spatiotemporal attention weight distribution of the historical training samples, along with the driving behavior explanation of the training samples from the current time step, the spatial attention weight distribution of the training samples, and the spatiotemporal attention weight distribution of the training samples, are input into the minimum negative log-likelihood function to calculate the minimum negative log-likelihood loss. Then, the deep attention loss, the shallow attention loss, and the minimum negative log-likelihood loss are aggregated to obtain the autonomous driving behavior prediction loss.

[0145] In one feasible approach, the loss function for predicting the autonomous driving behavior can be expressed as:

[0146]

[0147] in,

[0148]

[0149] in, For predicting losses in autonomous driving behavior, This indicates superficial attention loss. This represents deep attention loss, where k represents the current time step and k-1 represents the previous time step. J represents the attention weight distribution at the current time step, including the spatial attention weight distribution and the spatiotemporal attention weight distribution. k This indicates the interpretation of driving behavior at the current time step, o k The interpretability output for the current time step includes... and J k o k-1 This indicates the interpretability output of the previous time step. This represents the probability of determining the interpretable output of the current time step given the interpretable output of the previous time step.

[0150] In this embodiment, through the deep explanation mechanism, driving behaviors that are difficult for users to understand in relatively complex scenarios can be explained in depth in terms of time and space dimensions. This explains the causes and consequences of driving behaviors to users, eliminates users' doubts about driving behaviors, improves the explainability of the end-to-end perception decision architecture, reduces negative emotions such as doubt, panic, and anxiety caused by the unknown, and improves the user experience.

[0151] Example 4

[0152] Furthermore, embodiments of this application also provide an automatic driving control device, referring to... Figure 5 The autonomous driving control device is equipped with a cognitive decision-making model, which includes a cognitive layer and a decision-making layer. The autonomous driving control device includes:

[0153] Cognitive module 10 is used to acquire in-vehicle and out-of-vehicle monitoring data and extract current cognitive layer features from the in-vehicle and out-of-vehicle monitoring data through the cognitive layer;

[0154] Decision module 20 is used to input the current cognitive layer features into the decision layer to determine the autonomous driving control parameters;

[0155] The explanation module 30 is used to generate an explanation of driving behavior based on the current cognitive layer features and the autonomous driving control parameters.

[0156] Optionally, the autonomous driving control parameters include multiple autonomous driving control sub-parameters;

[0157] The interpretation module 30 is further configured to:

[0158] Based on the current cognitive layer features, determine the target decision features corresponding to each of the autonomous driving control sub-parameters;

[0159] Each of the aforementioned autonomous driving control sub-parameters is aggregated with its corresponding target decision features to obtain a step-by-step explanation of driving behavior;

[0160] The individual explanations of each driving behavior are aggregated to obtain the driving behavior explanation.

[0161] Optionally, the current cognitive layer features include external environment features; the target decision features include shallow target decision features;

[0162] The interpretation module 30 is further configured to:

[0163] Based on the current cognitive layer features, at least one shallow sub-parameter to be explained is determined from each of the autonomous driving control sub-parameters, and the spatial attention weight distribution corresponding to each of the shallow sub-parameters to be explained is determined.

[0164] Based on the spatial attention weight distribution and the external environment features, the shallow target decision features corresponding to each of the shallow sub-parameters to be explained are determined.

[0165] Optionally, the autonomous driving control device further includes a model training module, the model training module being used for:

[0166] Acquire training sample vehicle interior and exterior monitoring data and autonomous driving behavior labels, and extract the current cognitive layer features of the training sample from the training sample vehicle interior and exterior monitoring data through the cognitive layer;

[0167] The current cognitive layer features of the training samples are input into the decision layer to determine multiple autonomous driving control sub-parameters for the training samples.

[0168] Based on the current cognitive layer features of the training samples, at least one shallow unexplained sub-parameter of the training sample is determined from the autonomous driving control sub-parameters of each training sample, and the spatial attention weight distribution of the training sample corresponding to each shallow unexplained sub-parameter of the training sample is determined.

[0169] The shallow attention loss is determined based on the autonomous driving control sub-parameters of each training sample, the autonomous driving behavior labels of each training sample, and the spatial attention weight distribution of each training sample. The cognitive decision-making model is then iteratively optimized based on the shallow attention loss.

[0170] Optionally, the model training module is further configured to:

[0171] The autonomous driving control sub-parameters of each training sample, the autonomous driving behavior labels of each training sample, and the spatial attention weight distribution of each training sample are input into a shallow attention loss function to determine the shallow attention loss, wherein the shallow attention loss function is:

[0172]

[0173] in, For shallow attention loss, τ t The label for the autonomous driving behavior at time t. Let H be the training sample parameters for autonomous driving control at time t, and λ be the entropy function. s To preset the first hyperparameter, Let be the spatial attention weight distribution of the training samples at time t, where t∈[t0,T].

[0174] Optionally, the driving behavior interpretation includes deep driving behavior interpretation; the target decision features include deep target decision features;

[0175] The interpretation module 30 is further configured to:

[0176] Based on the current cognitive layer features, at least one deep sub-parameter to be explained is determined from each of the autonomous driving control sub-parameters;

[0177] Obtain the historical cognitive layer features extracted up to the current moment;

[0178] A spatiotemporal attention mechanism is employed to extract deep target decision features corresponding to each of the deep unexplained sub-parameters based on the current cognitive layer features and the historical cognitive layer features.

[0179] Optionally, the model training module is further configured to:

[0180] Based on the current cognitive layer features of the training samples, at least one deep unexplained sub-parameter of the training sample is determined from the autonomous driving control sub-parameters of each training sample, and the spatial attention weight distribution of the training sample corresponding to each deep unexplained sub-parameter of the training sample is determined.

[0181] The historical cognitive layer features of the training samples extracted before the current moment are obtained. A spatiotemporal attention mechanism is adopted to determine the spatiotemporal attention weight distribution of each deep unexplained sub-parameter of the training sample based on the current cognitive layer features and the historical cognitive layer features of the training sample.

[0182] The deep attention loss is determined based on the difference between the spatial attention weight distribution of each training sample and the spatiotemporal attention weight distribution of each training sample corresponding to the deep unexplained sub-parameters. The shallow attention loss is determined based on the difference between the spatial attention weight distribution of each training sample and the difference between the autonomous driving control sub-parameters of each training sample and their corresponding autonomous driving behavior labels.

[0183] The deep attention loss and the shallow attention loss are aggregated to obtain the autonomous driving behavior prediction loss, and the cognitive decision-making model is iteratively optimized based on the autonomous driving behavior prediction loss.

[0184] Optionally, the model training module is further configured to:

[0185] The spatial attention weight distribution and the spatiotemporal attention weight distribution of each training sample are input into the deep attention loss function to determine the deep attention loss, wherein the deep attention loss function is:

[0186]

[0187] Among them, D KL Represents the divergence function. Let be the spatial attention weight distribution of the deep unexplained sub-parameters of the training sample at time t. Let λ be the spatiotemporal attention weight distribution of the training sample corresponding to the deep unexplained sub-parameters of the training sample at time t. d The second hyperparameter is a preset, i∈[1,l].

[0188] Optionally, the model training module is further configured to:

[0189] Based on the shallow unexplained sub-parameters of each training sample and the current cognitive layer features of the training sample, at least one step-by-step explanation of shallow driving behavior of the training sample is generated; based on the deep unexplained sub-parameters of each training sample and the current cognitive layer features of the training sample, at least one step-by-step explanation of deep driving behavior of the training sample is generated.

[0190] The shallow driving behavior explanations, shallow sub-parameters to be explained, deep driving behavior explanations, and deep sub-parameters to be explained of each training sample are aggregated to obtain the driving behavior explanation of the training sample.

[0191] Obtain the historical training sample driving behavior explanation, historical training sample spatial attention weight distribution, and historical training sample spatiotemporal attention weight distribution from the previous time step. Based on the historical training sample driving behavior explanation, the historical training sample spatial attention weight distribution, and the historical training sample spatiotemporal attention weight distribution, as well as the training sample driving behavior explanation, training sample spatial attention weight distribution, and training sample spatiotemporal attention weight distribution from the current time step, determine the minimum negative log-likelihood loss.

[0192] The deep attention loss, the shallow attention loss, and the minimized negative log-likelihood loss are aggregated to obtain the autonomous driving behavior prediction loss.

[0193] Optionally, the interpretation module 30 is further configured to:

[0194] Obtain the step-by-step interpretation weights corresponding to each of the aforementioned autonomous driving control sub-parameters;

[0195] The driving behavior explanations are obtained by weighting and aggregating the step-by-step explanations based on the weights of each step-by-step explanation.

[0196] Optionally, the interpretation module 30 is further configured to:

[0197] Obtain the attention distribution features from the previous time step;

[0198] An attention mechanism is employed to determine the step-by-step interpretation weights for each of the aforementioned driving behaviors and the attention distribution characteristics of the previous time step.

[0199] Optionally, the cognitive layer includes a user feature extraction model; the current cognitive layer features include user demand features, wherein the user demand features include implicit user demand features and explicit user demand features; the in-vehicle and out-of-vehicle monitoring data includes user operation data and user status data;

[0200] The cognitive module 10 is also used for:

[0201] The user feature extraction model extracts user display requirement features from the user operation data, and / or extracts implicit user requirement features from the user status data.

[0202] Optionally, the cognitive layer includes a scene cognition model, a map building model, a risk target detection and tracking model, and a motion prediction model; the current cognitive layer features include environmental cognition features, wherein the environmental cognition features include scene cognition features, map features, risk target detection and tracking features, and risk target motion features;

[0203] The cognitive module 10 is also used for:

[0204] The scene recognition model extracts scene recognition features from the vehicle interior and exterior monitoring data, the map construction model extracts map features from the vehicle interior and exterior monitoring data, and the risk target detection and tracking model extracts risk target detection and tracking features from the vehicle interior and exterior monitoring data.

[0205] The motion prediction model employs an attention mechanism to predict the motion of risk targets based on the scene cognitive features, the map features, and the risk target detection and tracking features, thereby obtaining the motion features of the risk targets.

[0206] The autonomous driving control device provided by this invention employs the autonomous driving control method in the above embodiments, solving the technical problem of poor interpretability of end-to-end perception decision-making schemes in related technologies. Compared with related technologies, the autonomous driving control device provided by this invention has the same advantages as the autonomous driving control method provided in the above embodiments, and other technical features in this autonomous driving control device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0207] Example 5

[0208] Furthermore, embodiments of this application also provide an autonomous driving control device, on which a cognitive decision-making model is deployed. The cognitive decision-making model includes a cognitive layer and a decision layer. The cognitive layer is pre-trained using the autonomous driving control method described above. The autonomous driving control device includes:

[0209] The second acquisition module is used to acquire the monitoring data inside and outside the vehicle to be predicted;

[0210] The second cognitive module is used to extract the risk features to be predicted from the monitoring data inside and outside the vehicle through the cognitive layer, wherein the risk features to be predicted include cognitive features of the scenario to be predicted.

[0211] The decision module is used to input the risk characteristics to be predicted into the decision layer to determine the autonomous driving control parameters.

[0212] Optionally, the cognitive layer includes a scene cognition model, a map building model, a risk target detection and tracking model, and a motion prediction model; the risk features to be predicted also include map features to be predicted, risk target detection and tracking features to be predicted, and risk target motion features to be predicted.

[0213] The second cognitive model is also used for:

[0214] The scene cognition model extracts the scene cognition features to be predicted from the monitoring data inside and outside the vehicle to be predicted; the map construction model extracts the map features to be predicted from the monitoring data inside and outside the vehicle to be predicted; and the risk target detection and tracking model extracts the risk target detection and tracking features to be predicted from the monitoring data inside and outside the vehicle to be predicted.

[0215] The motion prediction model employs an attention mechanism to predict the motion of risk targets based on the cognitive features of the scene to be predicted, the features of the map to be predicted, and the detection and tracking features of the risk targets to be predicted, thereby obtaining the motion features of the risk targets to be predicted.

[0216] Optionally, the risk target detection and tracking feature to be predicted includes at least one risk target detection and tracking sub-feature; the motion prediction model includes an attention layer, a multilayer perceptron layer, and a prediction layer;

[0217] The second cognitive model is also used for:

[0218] The scene cognitive features to be predicted, the map features to be predicted, and the risk target detection and tracking features to be predicted are input into the attention layer. An attention mechanism is used to determine the risk target interaction features between each risk target detection and tracking sub-feature, the scene interaction features between each risk target detection and tracking sub-feature and the scene cognitive features to be predicted, and the map interaction features between each risk target detection and tracking sub-feature and the map features to be predicted.

[0219] The risk target interaction features, the scene interaction features, and the map interaction features are input into the multilayer perceptron layer to obtain scene risk features and motion query features.

[0220] The scenario risk features and the motion query features are input into the prediction layer to perform risk target motion prediction and obtain the motion features of the risk target to be predicted.

[0221] The autonomous driving control device provided by this invention employs the autonomous driving control method in the above embodiments, solving the technical problem of poor interpretability of end-to-end perception decision-making schemes in related technologies. Compared with related technologies, the autonomous driving control device provided by this invention has the same advantages as the autonomous driving control method provided in the above embodiments, and other technical features in this autonomous driving control device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0222] Example 6

[0223] Furthermore, embodiments of the present invention provide an electronic device, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute the autonomous driving control method in the above embodiments.

[0224] The following is for reference. Figure 6 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as Bluetooth headsets, mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0225] like Figure 6 As shown, an electronic device may include a processing unit (such as a central processing unit, graphics processing unit, etc.) that can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) or a program loaded from a storage device into random access memory (RAM). The RAM also stores various programs and arrays required for the operation of the electronic device. The processing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0226] Typically, the following systems can be connected to the I / O interface: input devices including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices including, for example, magnetic tapes, hard disks, etc.; and communication devices. Communication devices allow electronic devices to communicate wirelessly or wiredly with other devices to exchange arrays. Although electronic devices with various systems are shown in the figures, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems may be implemented alternatively.

[0227] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, it performs the functions defined above in the methods of embodiments of this disclosure.

[0228] The electronic device provided by this invention employs the autonomous driving control method in the above embodiments, solving the technical problem of poor interpretability of end-to-end perception decision-making schemes in related technologies. Compared with related technologies, the electronic device provided by this invention has the same advantages as the autonomous driving control method provided in the above embodiments, and other technical features in this electronic device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0229] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0230] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0231] Example 7

[0232] Furthermore, this embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon, the computer-readable program instructions being used to execute the autonomous driving control method in the above embodiment.

[0233] The computer-readable storage medium provided in this embodiment of the invention may be, for example, a USB flash drive, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0234] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0235] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by an electronic device, the electronic device causes the electronic device to: acquire in-vehicle and out-of-vehicle monitoring data and scene labels, and acquire the target human driving experience scene cognitive features corresponding to the scene labels; extract risk features from the in-vehicle and out-of-vehicle monitoring data through the cognitive layer; and iteratively optimize the cognitive layer based on the risk features and the target human driving experience scene cognitive features.

[0236] Alternatively, the aforementioned computer-readable storage medium carries one or more programs that, when executed by an electronic device, cause the electronic device to: acquire in-vehicle and out-of-vehicle monitoring data to be predicted; extract in-vehicle and out-of-vehicle risk features from the monitoring data through the cognitive layer, wherein the in-vehicle and out-of-vehicle risk features include in-vehicle scenario cognitive features; and input the in-vehicle and out-of-vehicle risk features into the decision layer to determine autonomous driving control parameters.

[0237] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0238] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0239] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0240] The computer-readable storage medium provided by this invention stores computer-readable program instructions for executing the above-described autonomous driving control method, thus solving the technical problem of poor interpretability of end-to-end perception decision-making schemes in related technologies. Compared with related technologies, the advantages of the computer-readable storage medium provided in this invention are the same as those of the autonomous driving control method provided in the above embodiments, and will not be repeated here.

[0241] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent scope of this application.

Claims

1. An automatic driving control method, characterized in that, The autonomous driving control method employs a cognitive decision-making model, which includes a cognitive layer and a decision-making layer; the autonomous driving control method includes the following steps: Acquire in-vehicle and out-of-vehicle monitoring data, and extract current cognitive layer features from the in-vehicle and out-of-vehicle monitoring data through the cognitive layer; The current cognitive layer features are input into the decision layer to determine the autonomous driving control parameters; The autonomous driving control parameters include multiple autonomous driving control sub-parameters. Based on the current cognitive layer features, the target decision features corresponding to each of the autonomous driving control sub-parameters are determined. Each of the aforementioned autonomous driving control sub-parameters is aggregated with its corresponding target decision features to obtain a step-by-step explanation of driving behavior; The individual explanations of each driving behavior are aggregated to obtain the driving behavior explanation; The current cognitive layer features include external vehicle environment features; the target decision features include shallow target decision features; The step of determining the target decision features corresponding to each of the autonomous driving control sub-parameters based on the current cognitive layer features includes: Based on the current cognitive layer features, at least one shallow sub-parameter to be explained is determined from each of the autonomous driving control sub-parameters, and the spatial attention weight distribution corresponding to each of the shallow sub-parameters to be explained is determined. Based on the spatial attention weight distribution and the external environment features, the shallow target decision features corresponding to each of the shallow sub-parameters to be explained are determined.

2. The autonomous driving control method as described in claim 1, characterized in that, Before the step of acquiring in-vehicle and out-of-vehicle monitoring data and extracting current cognitive layer features from the in-vehicle and out-of-vehicle monitoring data through the cognitive layer, the method further includes: Acquire training sample vehicle interior and exterior monitoring data and autonomous driving behavior labels, and extract the current cognitive layer features of the training sample from the training sample vehicle interior and exterior monitoring data through the cognitive layer; The current cognitive layer features of the training samples are input into the decision layer to determine multiple autonomous driving control sub-parameters for the training samples. Based on the current cognitive layer features of the training samples, at least one shallow unexplained sub-parameter of the training sample is determined from the autonomous driving control sub-parameters of each training sample, and the spatial attention weight distribution of the training sample corresponding to each shallow unexplained sub-parameter of the training sample is determined. The shallow attention loss is determined based on the autonomous driving control sub-parameters of each training sample, the autonomous driving behavior labels of each training sample, and the spatial attention weight distribution of each training sample. The cognitive decision-making model is then iteratively optimized based on the shallow attention loss.

3. The automatic driving control method as described in claim 2, characterized in that, The step of determining the shallow attention loss based on the autonomous driving control sub-parameters of each training sample, the autonomous driving behavior labels of each training sample, and the spatial attention weight distribution of each training sample includes: The autonomous driving control sub-parameters of each training sample, the autonomous driving behavior labels of each training sample, and the spatial attention weight distribution of each training sample are input into a shallow attention loss function to determine the shallow attention loss, wherein the shallow attention loss function is: in, s This is a superficial loss of attention. The label for the autonomous driving behavior at time t. Let be the training sample parameters for the autonomous driving control at time t, and H be the entropy function. To preset the first hyperparameter, Let be the spatial attention weight distribution of the training samples at time t, where t∈[t0,T].

4. The automatic driving control method as described in claim 3, characterized in that, The driving behavior interpretation includes deep driving behavior interpretation; the target decision features include deep target decision features; The step of determining the target decision features corresponding to each of the autonomous driving control sub-parameters based on the current cognitive layer features includes: Based on the current cognitive layer features, at least one deep sub-parameter to be explained is determined from each of the autonomous driving control sub-parameters; Obtain the historical cognitive layer features extracted up to the current moment; A spatiotemporal attention mechanism is employed to extract deep target decision features corresponding to each of the deep unexplained sub-parameters based on the current cognitive layer features and the historical cognitive layer features.

5. The automatic driving control method as described in claim 4, characterized in that, The step of determining a shallow attention loss based on the autonomous driving control sub-parameters of each training sample, the autonomous driving behavior labels of each training sample, and the spatial attention weight distribution of each training sample, and iteratively optimizing the cognitive decision-making model based on the shallow attention loss, includes: Based on the current cognitive layer features of the training samples, at least one deep unexplained sub-parameter of the training sample is determined from the autonomous driving control sub-parameters of each training sample, and the spatial attention weight distribution of the training sample corresponding to each deep unexplained sub-parameter of the training sample is determined. The historical cognitive layer features of the training samples extracted before the current moment are obtained. A spatiotemporal attention mechanism is adopted to determine the spatiotemporal attention weight distribution of each deep unexplained sub-parameter of the training sample based on the current cognitive layer features and the historical cognitive layer features of the training sample. The deep attention loss is determined based on the difference between the spatial attention weight distribution of each training sample and the spatiotemporal attention weight distribution of each training sample corresponding to the deep unexplained sub-parameters. The shallow attention loss is determined based on the difference between the spatial attention weight distribution of each training sample and the difference between the autonomous driving control sub-parameters of each training sample and their corresponding autonomous driving behavior labels. The deep attention loss and the shallow attention loss are aggregated to obtain the autonomous driving behavior prediction loss, and the cognitive decision-making model is iteratively optimized based on the autonomous driving behavior prediction loss.

6. The automatic driving control method as described in claim 5, characterized in that, The step of determining the deep attention loss based on the difference between the spatial attention weight distribution of each training sample and the spatiotemporal attention weight distribution of each training sample for each deep unexplained sub-parameter includes: The spatial attention weight distribution and the spatiotemporal attention weight distribution of each training sample are input into the deep attention loss function to determine the deep attention loss, wherein the deep attention loss function is: Among them, D KL Represents the divergence function. Let be the spatial attention weight distribution of the deep unexplained sub-parameters of the training sample at time t. Let be the spatiotemporal attention weight distribution of the training sample corresponding to the deep unexplained sub-parameters of the training sample at time t. For the preset second hyperparameter, i∈[1, l ].

7. The automatic driving control method as described in claim 5, characterized in that, The step of aggregating the deep attention loss and the shallow attention loss to obtain the autonomous driving behavior prediction loss includes: Based on the shallow unexplained sub-parameters of each training sample and the current cognitive layer features of the training sample, at least one step-by-step explanation of shallow driving behavior of the training sample is generated; based on the deep unexplained sub-parameters of each training sample and the current cognitive layer features of the training sample, at least one step-by-step explanation of deep driving behavior of the training sample is generated. The shallow driving behavior explanations, shallow sub-parameters to be explained, deep driving behavior explanations, and deep sub-parameters to be explained of each training sample are aggregated to obtain the driving behavior explanation of the training sample. Obtain the historical training sample driving behavior explanation, historical training sample spatial attention weight distribution, and historical training sample spatiotemporal attention weight distribution from the previous time step. Based on the historical training sample driving behavior explanation, the historical training sample spatial attention weight distribution, and the historical training sample spatiotemporal attention weight distribution, as well as the training sample driving behavior explanation, training sample spatial attention weight distribution, and training sample spatiotemporal attention weight distribution from the current time step, determine the minimum negative log-likelihood loss. The deep attention loss, the shallow attention loss, and the minimized negative log-likelihood loss are aggregated to obtain the autonomous driving behavior prediction loss.

8. The automatic driving control method as described in claim 1, characterized in that, The step of aggregating the step-by-step interpretations of each driving behavior to obtain a driving behavior interpretation includes: Obtain the step-by-step interpretation weights corresponding to each of the aforementioned autonomous driving control sub-parameters; The driving behavior explanations are obtained by weighting and aggregating the step-by-step explanations based on the weights of each step-by-step explanation.

9. The automatic driving control method as described in claim 8, characterized in that, The step of obtaining the step-by-step interpretation weights corresponding to each of the autonomous driving control sub-parameters includes: Obtain the attention distribution features from the previous time step; An attention mechanism is employed to determine the step-by-step interpretation weights for each of the aforementioned driving behaviors and the attention distribution characteristics of the previous time step.

10. The automatic driving control method as described in claim 1, characterized in that, The cognitive layer includes a user feature extraction model; the current cognitive layer features include user demand features, wherein the user demand features include implicit user demand features and explicit user demand features; the in-vehicle and out-of-vehicle monitoring data includes user operation data and user status data; The step of extracting current cognitive layer features from the vehicle interior and exterior monitoring data through the cognitive layer includes: The user feature extraction model extracts user display requirement features from the user operation data, and / or extracts implicit user requirement features from the user status data.

11. The automatic driving control method as described in claim 1, characterized in that, The cognitive layer includes a scene cognition model, a map building model, a risk target detection and tracking model, and a motion prediction model; the current cognitive layer features include environmental cognition features, wherein the environmental cognition features include scene cognition features, map features, risk target detection and tracking features, and risk target motion features; The step of extracting current cognitive layer features from the vehicle interior and exterior monitoring data through the cognitive layer includes: The scene recognition model extracts scene recognition features from the vehicle interior and exterior monitoring data, the map construction model extracts map features from the vehicle interior and exterior monitoring data, and the risk target detection and tracking model extracts risk target detection and tracking features from the vehicle interior and exterior monitoring data. The motion prediction model employs an attention mechanism to predict the motion of risk targets based on the scene cognitive features, the map features, and the risk target detection and tracking features, thereby obtaining the motion features of the risk targets.

12. An automatic driving control device, characterized in that, The autonomous driving control device is equipped with a cognitive decision-making model, which includes a cognitive layer and a decision-making layer. The autonomous driving control device includes: The cognitive module is used to acquire monitoring data inside and outside the vehicle, and to extract current cognitive layer features from the monitoring data inside and outside the vehicle through the cognitive layer. The decision module is used to input the current cognitive layer features into the decision layer to determine the autonomous driving control parameters; the autonomous driving control parameters include multiple autonomous driving control sub-parameters; The explanation module is used to determine the target decision features corresponding to each of the autonomous driving control sub-parameters based on the current cognitive layer features; aggregate each of the autonomous driving control sub-parameters and their corresponding target decision features to obtain a step-by-step explanation of driving behavior; and aggregate the step-by-step explanations of driving behavior to obtain a driving behavior explanation. The current cognitive layer features include external vehicle environment features; the target decision features include shallow target decision features; The step of determining the target decision features corresponding to each of the autonomous driving control sub-parameters based on the current cognitive layer features includes: Based on the current cognitive layer features, at least one shallow sub-parameter to be explained is determined from each of the autonomous driving control sub-parameters, and the spatial attention weight distribution corresponding to each of the shallow sub-parameters to be explained is determined. Based on the spatial attention weight distribution and the external environment features, the shallow target decision features corresponding to each of the shallow sub-parameters to be explained are determined.

13. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by at least one of the processors to enable at least one of the processors to perform the steps of the autonomous driving control method as described in any one of claims 1 to 11.

14. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and the computer-readable storage medium stores a program for implementing the autonomous driving control method. The program for implementing the autonomous driving control method is executed by a processor to implement the steps of the autonomous driving control method as described in any one of claims 1 to 11.

Images