Automatic driving control method and device, electronic equipment and storage medium

Abstract

The application discloses an automatic driving control method and device, electronic equipment and storage medium. The automatic driving control method adopts a cognitive decision model, the cognitive decision model comprises a cognitive layer and a plurality of decision layers, and the decision layers are connected in series. The automatic driving control method comprises the following steps: acquiring in-vehicle and out-vehicle monitoring data; extracting environmental cognitive features and user demand features from the in-vehicle and out-vehicle monitoring data through the cognitive layer; inputting the environmental cognitive features and the user demand features into each decision layer in turn for multi-layer decision reasoning to determine automatic driving control parameters. The application improves the interaction between the automatic driving vehicle and the user, reduces the negative emotions such as doubt, panic and unease of the user, and improves the user experience.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to an autonomous driving control method, device, electronic device, and storage medium. Background Technology

[0002] With the continuous development of technology and breakthroughs in computing power, autonomous driving is developing rapidly. Autonomous driving can provide people with a safer, more convenient and efficient way to travel, while also having a profound impact on the entire transportation system and urban planning.

[0003] The mainstream technical solutions for autonomous driving systems can be divided into two main types: rule-based modular architecture and data-driven end-to-end perception and decision-making architecture. While rule-based modular architectures have achieved mass production due to their advantages such as low computational requirements, simple deployment, and interpretable processes, they struggle to cover the entire scenario domain, and the results of modular solutions are not globally optimal. End-to-end solutions, on the other hand, use raw sensor data as input and generate planned and / or low-level control actions as output, enabling joint, global optimization and resulting in higher accuracy in autonomous driving decisions.

[0004] However, the end-to-end perception and decision-making architecture has poor interpretability and cannot conduct real-time human-machine interaction with the autonomous driving system. Therefore, when controlling the vehicle to perform sudden behaviors, it is easy for users riding in autonomous vehicles to be unable to understand the driving behavior of autonomous vehicles, resulting in negative emotions such as confusion, panic, and anxiety. At the same time, users find it difficult to express their own needs in the intelligent driving domain, resulting in a poor user experience. Summary of the Invention

[0005] The main objective of this application is to provide an autonomous driving control method, device, electronic device, and storage medium, aiming to solve the technical problem in related technologies where the interpretability of end-to-end perception decision-making schemes is poor due to the lack of hierarchical decision-making.

[0006] To achieve the above objectives, this application provides an autonomous driving control method. The autonomous driving control method employs a cognitive decision-making model, which includes a cognitive layer and multiple decision layers connected in series. The autonomous driving control method includes the following steps:

[0007] Acquire monitoring data inside and outside the vehicle;

[0008] The cognitive layer extracts environmental cognitive features and user demand features from the monitoring data inside and outside the vehicle.

[0009] The environmental perception features and user demand features are sequentially input into each of the decision layers for multi-level decision reasoning to determine the autonomous driving control parameters.

[0010] Optionally, the decision-making layer includes a first decision-making layer and a second decision-making layer; the step of sequentially inputting the environmental perception features and the user demand features into each of the decision-making layers for multi-level decision-making reasoning to determine the autonomous driving control parameters includes:

[0011] The first decision layer performs decision reasoning based on the environmental cognitive characteristics and the user demand characteristics to determine the sequence of tasks to be executed, wherein the sequence of tasks to be executed includes at least one task to be executed.

[0012] The target task to be executed is determined sequentially from each of the tasks to be executed, and the target task to be executed is input into the second decision layer;

[0013] The second decision layer determines the autonomous driving control parameters based on each target task to be executed and the corresponding environmental cognitive characteristics of each target task to be executed.

[0014] Optionally, the step of determining the autonomous driving control parameters through the second decision layer based on each of the target tasks to be executed and the environmental cognitive features corresponding to each of the target tasks to be executed includes:

[0015] The second decision layer determines autonomous driving control parameters based on the target task to be executed and the environmental cognitive characteristics.

[0016] The target task to be executed is identified as an executed task;

[0017] Acquire new in-vehicle and out-of-vehicle monitoring data, and extract new environmental cognitive features from the new in-vehicle and out-of-vehicle monitoring data through the cognitive layer;

[0018] Return to the step of sequentially determining the target task from each of the tasks to be executed and inputting the target task into the second decision layer.

[0019] Optionally, the second decision layer includes an inference layer and a decoding layer; the step of determining the autonomous driving control parameters based on each of the target tasks to be executed and the environmental cognitive features corresponding to each of the target tasks to be executed through the second decision layer includes:

[0020] Acquire environmental cognitive features and vehicle driving status data that match the time of each of the target tasks to be executed;

[0021] Through the inference layer, the driving behavior characteristics corresponding to each of the target tasks to be executed are determined based on the environmental cognitive characteristics corresponding to each of the target tasks to be executed.

[0022] The decoding layer decodes the autonomous driving control parameters based on each target task to be executed, the driving behavior characteristics corresponding to each target task to be executed, and the corresponding vehicle driving status data.

[0023] Optionally, after the step of sequentially inputting the environmental perception features and the user demand features into each of the decision layers for multi-level decision reasoning to determine the autonomous driving control parameters, the method further includes:

[0024] Based on the vehicle's internal and external monitoring data and the autonomous driving control parameters, an explanation of driving behavior is generated;

[0025] Output an explanation of the driving behavior.

[0026] Optionally, the cognitive layer includes a user feature extraction model; the in-vehicle and out-of-vehicle monitoring data includes user operation data and user status data;

[0027] The steps for extracting user demand characteristics from the in-vehicle and out-of-vehicle monitoring data through the cognitive layer include:

[0028] The user feature extraction model extracts user display requirement features from the user operation data, and / or extracts implicit user requirement features from the user status data.

[0029] Optionally, the cognitive layer includes a scene cognition model, a map building model, a risk target detection and tracking model, and a motion prediction model; the environmental cognition features include scene cognition features, map features, risk target detection and tracking features, and risk target motion features.

[0030] The steps for extracting environmental cognitive features from the vehicle interior and exterior monitoring data through the cognitive layer include:

[0031] The scene recognition model extracts scene recognition features from the vehicle interior and exterior monitoring data, the map construction model extracts map features from the vehicle interior and exterior monitoring data, and the risk target detection and tracking model extracts risk target detection and tracking features from the vehicle interior and exterior monitoring data.

[0032] The motion prediction model uses the scene cognitive features, the map features, and the risk target detection and tracking features to predict the motion of risk targets and obtain the motion features of risk targets. The motion prediction model employs an attention mechanism.

[0033] This application also provides an autonomous driving control device, wherein a cognitive decision-making model is deployed on the autonomous driving control device, the cognitive decision-making model includes a cognitive layer and multiple decision layers connected in series, and the autonomous driving control device includes:

[0034] The first acquisition module is used to acquire monitoring data inside and outside the vehicle;

[0035] The first cognitive module is used to extract environmental cognitive features and user demand features from the vehicle interior and exterior monitoring data through the cognitive layer;

[0036] The hierarchical decision module is used to sequentially input the environmental cognitive features and the user demand features into each of the decision layers to perform multi-level decision reasoning and determine the autonomous driving control parameters.

[0037] This application also provides an electronic device, which is a physical device, comprising: a memory, a processor, and a program of the autonomous driving control method stored in the memory and executable on the processor. When the program of the autonomous driving control method is executed by the processor, it can implement the steps of the autonomous driving control method as described above.

[0038] This application also provides a storage medium, which is a computer-readable storage medium, on which a program for implementing an autonomous driving control method is stored. When the program for the autonomous driving control method is executed by a processor, it implements the steps of the autonomous driving control method as described above.

[0039] This application provides an autonomous driving control method, device, electronic device, and storage medium. The autonomous driving control method employs a cognitive decision-making model, which includes a cognitive layer and multiple decision layers connected in series. First, by acquiring in-vehicle and out-of-vehicle monitoring data, the cognitive layer extracts environmental cognitive features and user demand features from the monitoring data, thus achieving the extraction of environmental cognitive features and user demand features. Then, by sequentially inputting the environmental cognitive features and user demand features into each of the decision layers for multi-level decision reasoning, the autonomous driving control parameters are determined, realizing the integration of environmental cognitive features and user demand features for multi-level decision reasoning. On the one hand, the progressive fulfillment of user needs aligns with the user's logical reasoning process, resulting in more orderly and logically sound autonomous driving control parameters that are easier for users to understand. On the other hand, autonomous driving control based on environmental perception characteristics allows for objective decision-making, ensuring the safe operation of autonomous vehicles. The autonomous driving control parameters that ensure safe operation are typically allowed to fluctuate within a certain safe range. For example, it is safe to pass through intersections at speeds within a certain range, and turning angles within a certain range will prevent collisions with other road users. Therefore, based on this, and combined with user needs, a more comfortable range that adapts to user requirements can be further identified from this safe range. By catering to user needs and enhancing interactivity, the final driving behavior of the autonomous vehicle is easier for users to understand, thereby improving the interpretability of the end-to-end perception and decision-making architecture, reducing negative emotions such as user confusion, panic, and anxiety, and enhancing the user experience. Attached Figure Description

[0040] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating the first embodiment of the autonomous driving control method of this application;

[0043] Figure 2 This is a schematic diagram of one possible implementation of the automatic driving control device in the embodiments of this application;

[0044] Figure 3This is a flowchart illustrating the second embodiment of the autonomous driving control method of this application;

[0045] Figure 4 This is a schematic diagram of the structure of the automatic driving control device in the embodiments of this application;

[0046] Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the autonomous driving control method in this application embodiment.

[0047] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0048] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Example 1

[0050] This application provides an autonomous driving control method, referring to... Figure 1 In the first embodiment of the autonomous driving control method of this application, the autonomous driving control method adopts a cognitive decision model, which includes a cognitive layer and multiple decision layers connected in series. The autonomous driving control method includes the following steps:

[0051] Step S10: Acquire monitoring data inside and outside the vehicle;

[0052] The execution subject of the method in this embodiment can be an autonomous driving control device, an autonomous driving control terminal device, or a server. This embodiment takes an autonomous driving control device as an example. The autonomous driving control device can be integrated into terminal devices such as vehicles, in-vehicle terminals, smartphones, and computers with data processing functions.

[0053] In this embodiment, it should be noted that the autonomous driving control method is applied to a cognitive decision model, which is used to realize end-to-end driving behavior control of autonomous vehicles. The cognitive decision model uses an attention mechanism and can be a deep learning model based on the Transformer structure or other deep learning neural network models that use attention mechanisms. The specific design can be made according to the actual situation, and this embodiment does not limit it.

[0054] The cognitive decision-making model includes at least a cognitive layer and a decision layer. The cognitive layer is used to extract, fuse, encode, and reason from the acquired in-vehicle and out-of-vehicle monitoring data to obtain the risk features required by the decision layer. The decision layer is used to plan the driving behavior of the autonomous vehicle in the next time step based on the risk features transmitted by the cognitive layer, so that the autonomous vehicle can avoid risks and drive safely to the destination. The decision layer is a multi-layered serial structure. That is, the environmental cognitive features and the user demand features are input into the first layer to obtain the output of the first layer, and then the output of the first layer is input into the second layer, and so on, until the last layer outputs the autonomous driving control parameters. In this way, reasoning and decision-making can be carried out step by step from the user demand until the final autonomous driving control parameters are obtained. This process conforms to the user's logical reasoning process. Therefore, the determined autonomous driving control parameters are more orderly and logical, and easier for the user to understand.

[0055] The in-vehicle and out-of-vehicle monitoring data refers to the data required for autonomous driving control that can be acquired, including external vehicle monitoring data and cabin monitoring data. This data can be one or more modalities selected from image data, text data, sound data, and signal data. For example, external vehicle monitoring data can include external image data collected by a camera, external point cloud data collected by LiDAR and millimeter-wave radar, vehicle location information data collected by a vehicle positioning system, external sound data collected by a microphone, and traffic light countdown data obtained through a communication module. For example, cabin monitoring data can include vehicle speed, acceleration, and other vehicle driving data obtained from the vehicle controller or controller area network bus, as well as cabin image data collected by a camera, driver image data, cabin sound data collected by a microphone, text data converted from collected speech, text data extracted from collected images, and user demand signal data collected through a human-machine interface. This in-vehicle and out-of-vehicle monitoring data can be collected in real time, thus carrying real-time information on the status of other road users, the driver's status, road conditions, and other potentially changing information.

[0056] As an example, step S10 includes: collecting sensor data at the current moment as in-vehicle and out-of-vehicle monitoring data through one or more sensors installed on the autonomous vehicle; and communicating with external devices through the communication module on the autonomous vehicle to obtain external data provided by the external devices as in-vehicle and out-of-vehicle monitoring data, such as traffic light signals, traffic accident information, and roadside monitoring data collected by roadside monitoring devices.

[0057] Step S20: Extract environmental cognitive features and user demand features from the vehicle interior and exterior monitoring data through the cognitive layer;

[0058] In this embodiment, it should be noted that different models can be used for feature extraction of different modalities of vehicle interior and exterior monitoring data. For example, convolutional neural networks can be used for feature extraction of image data, and long short-term memory networks can be used for feature extraction of time series data. Different models can also be used for feature processing to extract different risk features required by the decision-making layer. Therefore, multiple models can be deployed in the cognitive layer for feature processing. The specific model structure and model type can be determined according to the actual situation, and this embodiment does not impose any restrictions on this.

[0059] The environmental perception features are used to characterize the autonomous vehicle's perception of its current environment, such as its perception of traffic facilities, ground obstacles, its own driving status, other traffic participants, the risks of the scene, and the risk items within the scene. The main purpose of autonomous vehicle behavior control is to ensure the autonomous vehicle safely reaches its destination; that is, to control the vehicle to its destination while avoiding risks. In one feasible approach, autonomous driving control can predict high-risk and / or low-risk driving areas for the autonomous vehicle in the next time step, thereby planning the autonomous vehicle's driving behavior to avoid high-risk areas or pass through low-risk areas in the next time step. Since the environmental perception features are an objective representation of the autonomous vehicle's current environment, the more accurate the extracted environmental perception features, the more accurate the risk assessment of the autonomous vehicle's current situation, and the higher the safety of the autonomous driving control. The environmental cognitive features may include scene cognitive features, map features, risk target detection and tracking features, risk target movement features, low-risk driving area features, etc., and may also include other features. The cognitive layer can be a multi-layer structure, that is, the input of any cognitive layer can be the output of other cognitive layers, and the output of any cognitive layer can also be the input of other cognitive layers, thereby enabling the fusion and reasoning of multiple features, capturing the correlation and dependence between the various features extracted by the cognitive layer, and realizing a more comprehensive understanding and deeper logical reasoning of the current driving situation.

[0060] The user demand features are used to characterize the needs of users riding in autonomous vehicles. Examples include user needs regarding the destination, the route to the destination, time constraints, and the desire for a smooth ride to alleviate motion sickness. These user demand features can be captured by detecting user actions and collecting user images, and then extracted from these images. For instance, user images can be captured by a camera; if image recognition technology identifies the user as asleep, this usually implies a need for smooth vehicle operation. Similarly, user voice information can be captured by a microphone; if the user says "drive faster," it indicates a need for acceleration.

[0061] The cognitive layer can employ an attention mechanism, and in one feasible approach, it can adopt a transformer architecture. Using an attention mechanism allows for effective feature extraction, fusion, encoding, and inference of various modalities and types of vehicle monitoring data. It extracts real-time information that may change at the current moment, combines it with historical information from a previous period to form an information sequence, and captures the correlations and dependencies between different types of information. Feature inference is then performed to extract risk features with deep logical information, which are then used by the decision-making layer for autonomous vehicle control. This allows for comprehensive consideration of potentially changing information, multiple risk items, and the interplay between user needs during the prediction of autonomous vehicle driving risks, improving the accuracy of risk prediction. This enables autonomous vehicles to avoid more risks while better meeting user needs, thus enhancing the safety and comfort of autonomous driving control. The correlations and dependencies between different types of information can significantly impact autonomous vehicle control. For example, interactions may occur between traffic participants, between traffic participants and traffic facilities, and between traffic participants and the surrounding environment. For example, at an intersection, this vehicle is traveling east-west, and there is a second vehicle traveling north-south and pedestrians walking east-west nearby. Assuming no interaction between traffic participants, the two vehicles will not collide at their current speeds. However, if the second vehicle slows down or stops to avoid a pedestrian, a collision may occur. Therefore, considering interactions between traffic participants, this vehicle still needs to slow down or plan an alternative route. If the second vehicle is in a turning lane, considering interactions between traffic participants and traffic facilities, if the second vehicle's turning trajectory does not intersect with this vehicle's, this vehicle does not need to slow down. Furthermore, considering interactions between traffic participants and the environment, the complexity of an intersection increases the risk to both people and vehicles, potentially requiring the vehicle to slow down.

[0062] As an example, step S20 includes: inputting the vehicle interior and exterior monitoring data into the cognitive layer, performing multi-level progressive feature processing such as feature extraction, feature fusion, feature encoding, and feature reasoning to obtain cognitive layer features, wherein the cognitive layer features include at least environmental cognitive features and user demand features.

[0063] Optionally, the cognitive layer includes a user feature extraction model; the in-vehicle and out-of-vehicle monitoring data includes user operation data and user status data;

[0064] The steps for extracting user demand characteristics from the in-vehicle and out-of-vehicle monitoring data through the cognitive layer include:

[0065] The user feature extraction model extracts user display requirement features from the user operation data, and / or extracts implicit user requirement features from the user status data.

[0066] In this embodiment, it should be noted that user operation data refers to data generated by the user actively interacting with the autonomous vehicle, such as issuing voice commands to the autonomous vehicle or actively clicking buttons on the in-vehicle terminal display interface. User status data refers to user-related information actively collected by the autonomous vehicle. For example, if the user is asleep and does not actively interact with the autonomous vehicle, the camera can collect the user's image data to identify the user's sleeping state. Similarly, if the user is on the phone with a friend expressing nervousness and apprehension about riding in the autonomous vehicle, and does not actively interact with the vehicle, the microphone can collect the user's voice data to identify the user's nervousness and apprehension.

[0067] As an example, the user operation data can be input into the user feature extraction model to perform multi-level progressive feature processing such as feature extraction, feature fusion, feature encoding, and feature reasoning to obtain user display requirement features. Alternatively, the user status data can be input into the user feature extraction model to perform multi-level progressive feature processing such as feature extraction, feature fusion, feature encoding, and feature reasoning to obtain user implicit requirement features.

[0068] In this embodiment, users can actively interact with autonomous vehicles and make requests. Autonomous vehicles can also respond to user interactions and meet user requests, improving the interactivity between autonomous vehicles and users. Furthermore, autonomous vehicles can proactively identify user needs and take autonomous driving actions that better meet those needs, thereby reducing user discomfort.

[0069] Optionally, the cognitive layer includes a scene cognition model, a map building model, a risk target detection and tracking model, and a motion prediction model; the environmental cognition features include scene cognition features, map features, risk target detection and tracking features, and risk target motion features.

[0070] The steps for extracting environmental cognitive features from the vehicle interior and exterior monitoring data through the cognitive layer include:

[0071] Step A10: Extract scene recognition features from the vehicle interior and exterior monitoring data using the scene recognition model, extract map features from the vehicle interior and exterior monitoring data using the map construction model, and extract risk target detection and tracking features from the vehicle interior and exterior monitoring data using the risk target detection and tracking model.

[0072] Step A20: Based on the scene cognitive features, the map features, and the risk target detection and tracking features, the motion prediction model performs risk target motion prediction to obtain risk target motion features. The motion prediction model employs an attention mechanism.

[0073] In this embodiment, it should be noted that the cognitive layer includes a scene cognition model, a map building model, a risk target detection and tracking model, and a motion prediction model. A risk target refers to an object that may pose a risk to the autonomous vehicle's operation, and can be one or more of traffic participants, traffic infrastructure, etc. The scene cognition features, map features, and risk target detection and tracking features output by the scene cognition model, the map building model, and the risk target detection and tracking model all have a certain impact on predicting the motion of the risk target in the next time step. Furthermore, there are certain correlations and dependencies among the scene cognition features, map features, and risk target detection and tracking features. Therefore, they can all be used as inputs to the motion prediction model. The motion prediction model uses an attention mechanism to capture the correlations and dependencies between these features, and performs further feature fusion, encoding, and reasoning to determine the risk target's motion characteristics.

[0074] The risk features include scene recognition features, map features, risk target detection and tracking features, and risk target motion features. Scene recognition features are used to characterize the risk perception of the vehicle's current scene, such as attention distribution and semantic understanding of risk items in the scene. Map features are used to characterize map information such as ground facilities and road elements in the scene where the vehicle is located. Risk target detection and tracking features are used to characterize the identification of risk targets and the location information of risk targets within a certain period of time before the current moment. Risk target motion features are used to characterize the motion information of risk targets within a certain period of time after the current moment. Motion information can include location information, trajectory information, motion intention, and motion trend.

[0075] As an example, steps A10-A20 include: inputting the vehicle interior and exterior monitoring data into the scene cognition model, performing multi-level progressive feature processing such as feature extraction, feature fusion, feature encoding, and feature reasoning to extract scene cognition features; simultaneously, inputting the vehicle interior and exterior monitoring data into the map construction model, performing multi-level progressive feature processing such as feature extraction, feature fusion, feature encoding, and feature reasoning to extract map features; simultaneously, inputting the vehicle interior and exterior monitoring data into the risk target detection and tracking model, performing multi-level progressive feature processing such as feature extraction, feature fusion, feature encoding, and feature reasoning to extract risk target detection and tracking features. Furthermore, the scene cognition features, the map features, and the risk target detection and tracking features are input into the motion prediction model. The motion prediction model employs an attention mechanism to capture the correlation and dependency between the scene cognition features, the map features, and the risk target detection and tracking features. Based on these correlations and dependencies, risk target movement prediction can be performed more accurately, determining the risk target motion features of at least one risk target.

[0076] In one implementable manner, the risk target detection and tracking feature includes at least one risk target detection and tracking sub-feature; the motion prediction model includes an attention layer, a multilayer perceptron layer, and a prediction layer;

[0077] The step of using the motion prediction model to predict the motion of risk targets based on the scene cognitive features, the map features, and the risk target detection and tracking features, to obtain the risk target motion features, wherein the motion prediction model employs an attention mechanism includes:

[0078] Step A21: Input the scene cognition features, the map features, and the risk target detection and tracking features into the attention layer, and use the attention mechanism to determine the risk target interaction features between each of the risk target detection and tracking sub-features, the scene interaction features between each of the risk target detection and tracking sub-features and the scene cognition features, and the map interaction features between each of the risk target detection and tracking sub-features and the map features.

[0079] Step A22: Input the risk target interaction features, the scene interaction features, and the map interaction features into the multilayer perceptron layer to obtain scene risk features and motion query features;

[0080] Step A23: Input the scene risk features and the motion query features into the prediction layer to perform risk target motion prediction and obtain risk target motion features.

[0081] In this embodiment, it should be noted that for complex scenarios, multiple risk targets may be involved. When multiple risk targets are identified, the risk target detection and tracking model can identify and track them simultaneously. The resulting risk target detection and tracking features will contain relevant information about multiple risk targets; that is, the risk target detection and tracking features include at least one risk target detection and tracking sub-feature. In this case, motion prediction can be performed for each risk target separately. By employing an attention mechanism, the correlation and dependency between each risk target and other risk targets, between each risk target and map elements, and between each risk target and scene perception can be captured. Based on these correlations and dependencies, risk target motion prediction can be performed more accurately for each risk target, determining the risk target motion characteristics of each risk target.

[0082] Scene risk features and motion query features represent the risks that already exist in the current scene. Based on scene risk features and motion query features, we can further predict the motion information of the risk target after the current moment.

[0083] As an example, steps A21-A23 include: inputting the scene cognition features, the map features, and the risk target detection and tracking features into the attention layer; the attention layer employs an attention mechanism to capture the dependencies between each risk target detection and tracking sub-feature to determine the mutual influence between each risk target and other risk targets, generating risk target interaction features; capturing the dependencies between each risk target detection and tracking sub-feature and the scene cognition features to determine the mutual influence between each risk target and scene cognition, generating scene interaction features; and capturing the dependencies between each risk target detection and tracking sub-feature and the map features to determine the mutual influence between each risk target and map elements, generating scene interaction features. Then, the risk target interaction features, the scene interaction features, and the map interaction features are concatenated and input into a multilayer perceptron layer to obtain scene risk features and motion query features. Finally, the scene risk features and the motion query features are input into the prediction layer to perform risk target motion prediction for each risk target, obtaining risk target motion features for each risk target.

[0084] In one feasible approach, the multilayer perceptron layer can also decode scene risk features and motion query features, and output the decoded scene risk prediction information and motion information of risk targets as intermediate results for users to view.

[0085] Step S30: The environmental perception features and the user demand features are sequentially input into each of the decision layers for multi-level decision reasoning to determine the autonomous driving control parameters.

[0086] In this embodiment, it should be noted that the autonomous driving control parameters may include driving trajectory, vehicle control parameters, etc. The driving trajectory refers to the trajectory of the autonomous vehicle after a period of time following the current moment, and the autonomous vehicle can be controlled to drive based on the driving trajectory. The vehicle control parameters include speed, accelerator pedal opening, steering wheel angle, etc.

[0087] As an example, step S30 includes: sequentially inputting the environmental perception features and the user demand features into each of the decision layers to perform multi-level decision reasoning, and determining the autonomous driving control parameters of the autonomous vehicle at the current moment or for a period of time after the current moment.

[0088] Optionally, after the step of sequentially inputting the environmental perception features and the user demand features into each of the decision layers for multi-level decision reasoning to determine the autonomous driving control parameters, the method further includes:

[0089] Step S40: Generate a driving behavior explanation based on the vehicle interior and exterior monitoring data and the autonomous driving control parameters;

[0090] Step S50: Output the explanation of the driving behavior.

[0091] In this embodiment, it should be noted that while catering to user needs can help users better understand the autonomous driving decisions made by the cognitive decision-making model, the decisions made by the cognitive decision-making model after integrating a large amount of in-vehicle and out-of-vehicle monitoring data and conducting comprehensive and multi-level reasoning have a long logical chain between them and the root causes of the decisions. Users can only perceive the final decision, and may even lose sight of the root causes affecting the decision over time. For example, if a cat suddenly runs across the road, causing the autonomous vehicle to stop abruptly, by the time the user perceives the sudden stop and tries to find the cause, the cat may have already disappeared. Another example is that the autonomous vehicle may accelerate and change lanes to avoid a collision because the trajectory of a pedestrian is likely to intersect with its own trajectory at some point, posing a collision risk. Since the vehicle cannot slow down to avoid a collision, it may do so. When the user perceives this acceleration and tries to find the cause, they cannot predict the trajectory and cannot accurately know the reason for the autonomous vehicle's acceleration and lane-changing behavior, at least in the short term. In such cases, users still find it difficult to understand the behavior of the autonomous vehicle, thus experiencing negative emotions such as confusion, panic, and anxiety, resulting in a poor user experience.

[0092] By generating and outputting explanations of driving behavior, it is possible to proactively interact with users, explain the reasons for driving behavior, eliminate users' doubts about driving behavior, reduce negative emotions such as doubt, panic, and anxiety, increase users' trust in autonomous vehicles, and improve the user experience.

[0093] As an example, steps S40-S50 include: decoding information about influencing factors affecting the decision-making layer's output of autonomous driving control parameters from the vehicle's internal and external monitoring data; fusing the autonomous driving control parameters and the information about the corresponding influencing factors to generate a driving behavior explanation; and then outputting the driving behavior explanation through images, sound, text, etc., so that users can understand the reasons for the driving behavior. For example, if an autonomous vehicle needs to stop at a red light intersection, the autonomous driving control parameter can be "stop," and the influencing factors can be the location of the red light and the stop line. By aggregating these two, a driving behavior explanation can be generated: "Because the light ahead is red, the vehicle stops before the stop line." This can be output via a speaker as a voice announcement: "Because the light ahead is red, the vehicle stops before the stop line."

[0094] In one feasible approach, the driving behavior explanation can be output before the autonomous vehicle executes the autonomous driving control parameters. This allows the user to anticipate the driving behavior the vehicle is about to perform, thus preventing panic caused by sudden driving behaviors of the autonomous vehicle.

[0095] In one feasible approach, refer to Figure 2 The autonomous driving control device includes a cockpit domain and an intelligent driving domain human-machine interface, a cognitive decision model, and a vehicle driving behavior representation module. The cockpit domain and intelligent driving domain human-machine interface are used to receive user demand information and environmental cognition information obtained from the cockpit domain and intelligent driving domain. The cognitive decision model is used to determine autonomous driving control parameters. The vehicle driving behavior representation module is used to control the autonomous vehicle to perform corresponding driving behaviors based on the autonomous driving control parameters. After performing a driving behavior, the device can obtain user feedback information on the driving behavior through the cockpit domain and intelligent driving domain human-machine interface. This cycle allows the autonomous vehicle to perform driving behaviors adapted to user needs.

[0096] In this embodiment, the autonomous driving control method employs a cognitive decision-making model, which includes a cognitive layer and multiple decision layers connected in series. First, by acquiring in-vehicle and out-of-vehicle monitoring data, the cognitive layer extracts environmental cognitive features and user demand features from the monitoring data, thus achieving the extraction of these features. Then, by sequentially inputting the environmental cognitive features and user demand features into each decision layer for multi-level decision-making reasoning, the autonomous driving control parameters are determined, achieving the integration of environmental cognitive features and user demand features for multi-level decision-making reasoning. On the one hand, the progressive fulfillment of user needs aligns with the user's logical reasoning process, resulting in more orderly and logically sound autonomous driving control parameters that are easier for users to understand. On the other hand, autonomous driving control based on environmental perception characteristics allows for objective decision-making, ensuring the safe operation of autonomous vehicles. The autonomous driving control parameters that ensure safe operation are typically allowed to fluctuate within a certain safe range. For example, it is safe to pass through intersections at speeds within a certain range, and turning angles within a certain range will prevent collisions with other road users. Therefore, based on this, and combined with user needs, a more comfortable range that adapts to user requirements can be further identified from this safe range. By catering to user needs and enhancing interactivity, the final driving behavior of the autonomous vehicle is easier for users to understand, thereby improving the interpretability of the end-to-end perception and decision-making architecture, reducing negative emotions such as user confusion, panic, and anxiety, and enhancing the user experience.

[0097] Example 2

[0098] Furthermore, in the second embodiment of this application, content that is the same as or similar to the above embodiments can be referred to the above description, and will not be repeated hereafter. Based on this, refer to... Figure 3 The decision-making layer includes a first decision-making layer and a second decision-making layer; the step of sequentially inputting the environmental cognitive features and the user demand features into each of the decision-making layers for multi-level decision-making reasoning to determine the autonomous driving control parameters includes:

[0099] Step S31: Through the first decision layer, decision reasoning is performed based on the environmental cognitive characteristics and the user demand characteristics to determine the sequence of tasks to be executed, wherein the sequence of tasks to be executed includes at least one task to be executed;

[0100] In this embodiment, it should be noted that user needs are diverse. For user needs that require multiple steps to achieve, if the overall control strategy corresponding to the user need is output, for example, if the user needs to go to a certain place, the driving trajectory or destination is output; or if the user needs to drive more smoothly, the switch to gentle driving mode is output. Although this can meet the user needs to a certain extent, it requires cooperation with other functional modules of the vehicle, places certain requirements on the vehicle configuration, and requires a lot of communication during the process, which may affect the overall control efficiency of the autonomous vehicle.

[0101] The decision-making layer consists of two levels. The first level makes task-level decisions based on user needs, dividing the data into a series of tasks to be executed. Since the collected user needs are usually fragmented and subjective, the first level needs to infer the precise and complete driving tasks that the user wants the autonomous vehicle to perform based on user need characteristics and environmental awareness characteristics. For driving tasks that require multiple steps or a long time to complete, they can be further divided into multiple tasks to be executed sequentially. The second level, based on the tasks to be executed and environmental awareness characteristics, sequentially generates autonomous driving control parameters for each task in the current environment. These generated autonomous driving control parameters can be determined as parameters that the autonomous vehicle can directly execute, based on the actual situation of the vehicle.

[0102] In one feasible approach, the autonomous driving control parameters can be driving parameters such as accelerator pedal opening and steering wheel angle. Outputting driving parameters can enable more refined control of the autonomous vehicle and achieve comprehensive regulation of the autonomous vehicle.

[0103] The sequence of tasks to be executed consists of at least one task to be executed arranged in chronological order. The autonomous vehicle can fulfill the user's needs by executing the tasks in the sequence of tasks to be executed in sequence.

[0104] As an example, step S31 includes: inputting the environmental perception features and the user demand features into the first decision layer for decision reasoning, and determining the sequence of tasks to be executed by the autonomous vehicle in chronological order over a period of time after the current moment.

[0105] Step S32: Sequentially determine the target task to be executed from each of the tasks to be executed, and input the target task to be executed into the second decision layer;

[0106] As an example, step S32 includes: after the first decision layer determines the sequence of tasks to be executed, sequentially identifying the task that has not yet been executed and has the earliest execution time as the target task to be executed, and inputting the target task to be executed into the second decision layer so that the second decision layer can further generate corresponding autonomous driving control parameters. It should be noted that executed and unexecuted tasks can be distinguished by adding a flag indicating whether the task to be executed has been executed, or by deleting the corresponding task from the sequence of tasks to be executed after each task is identified as a target task to be executed.

[0107] Step S33: The second decision layer determines the autonomous driving control parameters based on each of the target tasks to be executed and the environmental cognitive characteristics corresponding to each of the target tasks to be executed.

[0108] In this embodiment, it should be noted that the environmental cognitive features corresponding to each of the target tasks to be executed can be the same as the environmental cognitive features used by the first decision layer to determine the sequence of tasks to be executed; that is, the same environmental cognitive features are used for the entire sequence of tasks to be executed. The environmental cognitive features corresponding to each of the target tasks to be executed can also be different from the environmental cognitive features used by the first decision layer to determine the sequence of tasks to be executed; that is, new environmental cognitive features are obtained to adapt to changes in the environment over time. For example, during autonomous driving, the collection of in-vehicle and out-of-vehicle monitoring data and the extraction of environmental cognitive features can be performed continuously or at intervals, in parallel with the autonomous driving control method; alternatively, the collection of in-vehicle and out-of-vehicle monitoring data and the extraction of environmental cognitive features can be performed again before the second decision layer determines the autonomous driving control parameters each time.

[0109] As an example, step S33 includes: inputting each of the target tasks to be executed and the environmental cognitive features corresponding to each of the target tasks to be executed into the second decision layer to generate autonomous driving control parameters corresponding to each of the target tasks to be executed.

[0110] Optionally, after the step of determining the autonomous driving control parameters based on each of the target tasks to be executed and the environmental cognitive characteristics corresponding to each of the target tasks to be executed by the second decision layer, the method further includes:

[0111] Step B10: The second decision layer determines the autonomous driving control parameters based on the target task to be executed and the environmental cognitive characteristics.

[0112] Step B20: Determine the target task to be executed as an executed task;

[0113] Step B30: Acquire new in-vehicle and out-of-vehicle monitoring data, and extract new environmental cognitive features from the new in-vehicle and out-of-vehicle monitoring data through the cognitive layer;

[0114] Step B40: Return to the step of sequentially determining the target task to be executed from each of the tasks to be executed and inputting the target task to be executed into the second decision layer.

[0115] As an example, steps B10-B40 include: inputting the target task to be executed and the environmental cognitive features into the second decision layer to generate autonomous driving control parameters corresponding to the target task to be executed. In one feasible manner, after generating the autonomous driving control parameters corresponding to the target task to be executed, the autonomous driving control parameters can be decoded immediately to control the autonomous vehicle to execute the driving behavior corresponding to the autonomous driving control parameters. Then, the target task to be executed is determined as an executed task, new in-vehicle and out-of-vehicle monitoring data is acquired again, and the new in-vehicle and out-of-vehicle monitoring data is input into the cognitive layer to extract new environmental cognitive features, thereby updating the environmental cognitive features. After updating the environmental cognitive features, the process returns to the step of sequentially determining the target task to be executed from each of the tasks to be executed and inputting the target task to be executed into the second decision layer, so that the autonomous vehicle continues to execute the next task to be executed until all tasks to be executed are completed.

[0116] Optionally, the second decision layer includes an inference layer and a decoding layer; the step of determining the autonomous driving control parameters based on each of the target tasks to be executed and the environmental cognitive features corresponding to each of the target tasks to be executed through the second decision layer includes:

[0117] Step C10: Obtain environmental cognitive features and vehicle driving status data that match the time of each target's pending task;

[0118] In this embodiment, it should be noted that the second decision layer includes an inference layer and a decoding layer. The inference layer is used to infer driving behavior that conforms to the current actual situation, and the decoding layer is used to decode the autonomous driving control parameters that the autonomous vehicle can directly execute.

[0119] During the execution of each of the tasks to be executed, the external environment and vehicle status may be constantly changing. Therefore, it is necessary to continuously understand the external environment and vehicle driving status, and then through time matching, the vehicle can make autonomous driving decisions that adapt to the actual situation. In one feasible approach, time matching with the target task to be executed can be achieved by ensuring that the generation time or acquisition time is closest to the execution time of the target task to be executed.

[0120] As an example, step C10 includes: acquiring environmental cognitive features and vehicle driving status data that match the time of each of the target tasks to be performed.

[0121] Step C20: Through the inference layer, determine the driving behavior characteristics corresponding to each of the target tasks to be executed based on the environmental cognitive characteristics corresponding to each of the target tasks to be executed.

[0122] As an example, step C20 includes: inputting each of the target tasks to be executed and the environmental cognitive features corresponding to each of the target tasks to be executed into the inference layer, performing feature inference, and determining the driving behavior features corresponding to each of the target tasks to be executed.

[0123] Step C30: Through the decoding layer, the autonomous driving control parameters are decoded based on each of the target tasks to be executed, the driving behavior characteristics corresponding to each of the target tasks to be executed, and the corresponding vehicle driving state data.

[0124] As an example, step C30 includes: inputting each of the target tasks to be executed, the driving behavior features corresponding to each of the target tasks to be executed, and the vehicle driving state data corresponding to each of the target tasks to be executed into the decoding layer, and decoding the autonomous driving control parameters corresponding to each of the target tasks to be executed.

[0125] In one feasible approach, steps C10-C30 can be performed sequentially. Specifically, the first decision layer determines a target task to be executed each time. The second decision layer acquires environmental cognitive features and vehicle driving state data that are closest in time to the currently determined target task. Through the inference layer, driving behavior features are determined based on the environmental cognitive features corresponding to the currently determined target task. Then, through the decoding layer, the autonomous driving control parameters corresponding to the currently determined target task are decoded based on the currently determined target task, environmental cognitive features, and vehicle driving state data. A completion signal can then be sent to the first decision layer, causing it to re-determine a target task. This process is repeated until all tasks are completed.

[0126] In this embodiment, hierarchical decision-making enables the subdivision of driving tasks required by user needs, thereby achieving comprehensive control over the entire sequence of tasks to be executed. On the one hand, it eliminates the need for other functional modules of the vehicle and reduces communication time, improving the control efficiency of autonomous vehicles. On the other hand, for driving tasks with long execution times or many steps, adaptive adjustments can be made continuously throughout the control process based on changes in the external environment and vehicle status. This better adapts to changing realities while meeting user needs, reducing driving risks and improving the safety of autonomous driving.

[0127] Example 3

[0128] Furthermore, embodiments of this application also provide an automatic driving control device, referring to... Figure 4 The autonomous driving control device is equipped with a cognitive decision-making model, which includes a cognitive layer and multiple decision layers connected in series. The autonomous driving control device includes:

[0129] The first acquisition module 10 is used to acquire monitoring data inside and outside the vehicle;

[0130] The first cognitive module 20 is used to extract environmental cognitive features and user demand features from the vehicle interior and exterior monitoring data through the cognitive layer;

[0131] The hierarchical decision module 30 is used to sequentially input the environmental cognitive features and the user demand features into each of the decision layers to perform multi-level decision reasoning and determine the autonomous driving control parameters.

[0132] Optionally, the decision-making layer includes a first decision-making layer and a second decision-making layer; the hierarchical decision-making module 30 is further configured to:

[0133] The first decision layer performs decision reasoning based on the environmental cognitive characteristics and the user demand characteristics to determine the sequence of tasks to be executed, wherein the sequence of tasks to be executed includes at least one task to be executed.

[0134] The target task to be executed is determined sequentially from each of the tasks to be executed, and the target task to be executed is input into the second decision layer;

[0135] The second decision layer determines the autonomous driving control parameters based on each target task to be executed and the corresponding environmental cognitive characteristics of each target task to be executed.

[0136] Optionally, the hierarchical decision module 30 is further configured to:

[0137] The second decision layer determines autonomous driving control parameters based on the target task to be executed and the environmental cognitive characteristics.

[0138] The target task to be executed is identified as an executed task;

[0139] Acquire new in-vehicle and out-of-vehicle monitoring data, and extract new environmental cognitive features from the new in-vehicle and out-of-vehicle monitoring data through the cognitive layer;

[0140] Return to the step of sequentially determining the target task from each of the tasks to be executed and inputting the target task into the second decision layer.

[0141] Optionally, the second decision layer includes an inference layer and a decoding layer; the hierarchical decision module 30 is further configured to:

[0142] Acquire environmental cognitive features and vehicle driving status data that match the time of each of the target tasks to be executed;

[0143] Through the inference layer, the driving behavior characteristics corresponding to each of the target tasks to be executed are determined based on the environmental cognitive characteristics corresponding to each of the target tasks to be executed.

[0144] The decoding layer decodes the autonomous driving control parameters based on each target task to be executed, the driving behavior characteristics corresponding to each target task to be executed, and the corresponding vehicle driving status data.

[0145] Optionally, after the step of sequentially inputting the environmental perception features and the user demand features into each of the decision layers for multi-level decision reasoning to determine the autonomous driving control parameters, the autonomous driving control device further includes an interpretation module, which is used for:

[0146] Based on the vehicle's internal and external monitoring data and the autonomous driving control parameters, an explanation of driving behavior is generated;

[0147] Output an explanation of the driving behavior.

[0148] Optionally, the cognitive layer includes a user feature extraction model; the in-vehicle and out-of-vehicle monitoring data includes user operation data and user status data;

[0149] The first cognitive module 10 is further configured to:

[0150] The user feature extraction model extracts user display requirement features from the user operation data, and / or extracts implicit user requirement features from the user status data.

[0151] Optionally, the cognitive layer includes a scene cognition model, a map building model, a risk target detection and tracking model, and a motion prediction model; the environmental cognition features include scene cognition features, map features, risk target detection and tracking features, and risk target motion features.

[0152] The first cognitive module 10 is further configured to:

[0153] The scene recognition model extracts scene recognition features from the vehicle interior and exterior monitoring data, the map construction model extracts map features from the vehicle interior and exterior monitoring data, and the risk target detection and tracking model extracts risk target detection and tracking features from the vehicle interior and exterior monitoring data.

[0154] The motion prediction model uses the scene cognitive features, the map features, and the risk target detection and tracking features to predict the motion of risk targets and obtain the motion features of risk targets. The motion prediction model employs an attention mechanism.

[0155] The autonomous driving control device provided by this invention employs the autonomous driving control method in the above embodiments, solving the technical problem in related technologies where the end-to-end perception decision-making scheme still has poor interpretability due to the lack of hierarchical decision-making. Compared with related technologies, the autonomous driving control device provided by this invention has the same advantages as the autonomous driving control method provided in the above embodiments, and other technical features in this autonomous driving control device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0156] Example 4

[0157] Furthermore, embodiments of the present invention provide an electronic device, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute the autonomous driving control method in the above embodiments.

[0158] The following is for reference. Figure 5 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as Bluetooth headsets, mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0159] like Figure 5 As shown, an electronic device may include a processing unit (such as a central processing unit, graphics processing unit, etc.) that can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) or a program loaded from a storage device into random access memory (RAM). The RAM also stores various programs and arrays required for the operation of the electronic device. The processing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0160] Typically, the following systems can be connected to the I / O interface: input devices including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices including, for example, magnetic tapes, hard disks, etc.; and communication devices. Communication devices allow electronic devices to communicate wirelessly or wiredly with other devices to exchange arrays. Although electronic devices with various systems are shown in the figures, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems may be implemented alternatively.

[0161] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, it performs the functions defined above in the methods of embodiments of this disclosure.

[0162] The electronic device provided by this invention employs the autonomous driving control method in the above embodiments, solving the technical problem in related technologies where the end-to-end perception decision-making scheme still suffers from poor interpretability due to the lack of hierarchical decision-making. Compared with related technologies, the electronic device provided by this invention has the same advantages as the autonomous driving control method provided in the above embodiments, and other technical features in this electronic device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0163] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0164] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0165] Example 5

[0166] Furthermore, this embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon, the computer-readable program instructions being used to execute the autonomous driving control method in the above embodiment.

[0167] The computer-readable storage medium provided in this embodiment of the invention may be, for example, a USB flash drive, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0168] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0169] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by an electronic device, the electronic device causes the electronic device to: acquire in-vehicle and out-of-vehicle monitoring data; extract environmental cognitive features and user demand features from the in-vehicle and out-of-vehicle monitoring data through the cognitive layer; and sequentially input the environmental cognitive features and the user demand features into each of the decision layers for multi-level decision reasoning to determine the autonomous driving control parameters.

[0170] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0171] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0172] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0173] The computer-readable storage medium provided by this invention stores computer-readable program instructions for executing the above-described autonomous driving control method, solving the technical problem in related technologies where the end-to-end perception decision-making scheme still has poor interpretability due to the lack of hierarchical decision-making. Compared with related technologies, the advantages of the computer-readable storage medium provided in the embodiments of this invention are the same as those of the autonomous driving control method provided in the above embodiments, and will not be repeated here.

[0174] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent scope of this application.

Claims

1. An automatic driving control method, characterized in that, The autonomous driving control method employs a cognitive decision-making model, which includes a cognitive layer and multiple decision layers connected in series. The autonomous driving control method includes the following steps: Acquire monitoring data inside and outside the vehicle; The cognitive layer extracts environmental cognitive features and user demand features from the monitoring data inside and outside the vehicle. The environmental perception features and the user demand features are sequentially input into each of the decision layers for multi-level decision reasoning to determine the autonomous driving control parameters. The cognitive layer includes a scene cognition model, a map building model, a risk target detection and tracking model, and a motion prediction model; the environmental cognition features include scene cognition features, map features, risk target detection and tracking features, and risk target motion features. The steps for extracting environmental cognitive features from the vehicle interior and exterior monitoring data through the cognitive layer include: The scene recognition model extracts scene recognition features from the vehicle interior and exterior monitoring data, the map construction model extracts map features from the vehicle interior and exterior monitoring data, and the risk target detection and tracking model extracts risk target detection and tracking features from the vehicle interior and exterior monitoring data. The motion prediction model uses the scene cognitive features, the map features, and the risk target detection and tracking features to predict the motion of risk targets and obtain the motion features of risk targets. The motion prediction model employs an attention mechanism.

2. The automatic driving control method as described in claim 1, characterized in that, The decision-making layer includes a first decision-making layer and a second decision-making layer; the step of sequentially inputting the environmental cognitive features and the user demand features into each of the decision-making layers for multi-level decision-making reasoning to determine the autonomous driving control parameters includes: The first decision layer performs decision reasoning based on the environmental cognitive characteristics and the user demand characteristics to determine the sequence of tasks to be executed, wherein the sequence of tasks to be executed includes at least one task to be executed. The target task to be executed is determined sequentially from each of the tasks to be executed, and the target task to be executed is input into the second decision layer; The second decision layer determines the autonomous driving control parameters based on each target task to be executed and the corresponding environmental cognitive characteristics of each target task to be executed.

3. The automatic driving control method as described in claim 2, characterized in that, The step of determining autonomous driving control parameters through the second decision layer based on each of the target tasks to be executed and the environmental cognitive features corresponding to each of the target tasks to be executed includes: The second decision layer determines autonomous driving control parameters based on the target task to be executed and the environmental cognitive characteristics. The target task to be executed is identified as an executed task; Acquire new in-vehicle and out-of-vehicle monitoring data, and extract new environmental cognitive features from the new in-vehicle and out-of-vehicle monitoring data through the cognitive layer; Return to the step of sequentially determining the target task from each of the tasks to be executed and inputting the target task into the second decision layer.

4. The automatic driving control method as described in claim 2, characterized in that, The second decision layer includes an inference layer and a decoding layer; the step of determining the autonomous driving control parameters based on each of the target tasks to be executed and the environmental cognitive features corresponding to each of the target tasks to be executed through the second decision layer includes: Acquire environmental cognitive features and vehicle driving status data that match the time of each of the target tasks to be executed; Through the inference layer, the driving behavior characteristics corresponding to each of the target tasks to be executed are determined based on the environmental cognitive characteristics corresponding to each of the target tasks to be executed. The decoding layer decodes the autonomous driving control parameters based on each target task to be executed, the driving behavior characteristics corresponding to each target task to be executed, and the corresponding vehicle driving status data.

5. The automatic driving control method as described in claim 1, characterized in that, After the step of sequentially inputting the environmental perception features and the user demand features into each of the decision layers for multi-level decision reasoning to determine the autonomous driving control parameters, the method further includes: Based on the vehicle's internal and external monitoring data and the autonomous driving control parameters, an explanation of driving behavior is generated; Output an explanation of the driving behavior.

6. The automatic driving control method as described in claim 1, characterized in that, The cognitive layer includes a user feature extraction model; the vehicle interior and exterior monitoring data includes user operation data and user status data. The steps for extracting user demand characteristics from the in-vehicle and out-of-vehicle monitoring data through the cognitive layer include: The user feature extraction model extracts explicit user demand features from the user operation data, and / or extracts implicit user demand features from the user status data.

7. An automatic driving control device, characterized in that, The autonomous driving control device is equipped with a cognitive decision-making model, which includes a cognitive layer and multiple decision layers connected in series. The autonomous driving control device includes: The first acquisition module is used to acquire monitoring data inside and outside the vehicle; The first cognitive module is used to extract environmental cognitive features and user demand features from the vehicle interior and exterior monitoring data through the cognitive layer; The hierarchical decision-making module is used to sequentially input the environmental cognitive features and the user demand features into each of the decision layers to perform multi-level decision-making reasoning and determine the autonomous driving control parameters. The cognitive layer includes a scene cognition model, a map building model, a risk target detection and tracking model, and a motion prediction model; the environmental cognition features include scene cognition features, map features, risk target detection and tracking features, and risk target motion features. The first cognitive module is further used for: The scene recognition model extracts scene recognition features from the vehicle interior and exterior monitoring data, the map construction model extracts map features from the vehicle interior and exterior monitoring data, and the risk target detection and tracking model extracts risk target detection and tracking features from the vehicle interior and exterior monitoring data. The motion prediction model uses the scene cognitive features, the map features, and the risk target detection and tracking features to predict the motion of risk targets and obtain the motion features of risk targets. The motion prediction model employs an attention mechanism.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by at least one of the processors to enable at least one of the processors to perform the steps of the autonomous driving control method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and the computer-readable storage medium stores a program for implementing the autonomous driving control method. The program for implementing the autonomous driving control method is executed by a processor to implement the steps of the autonomous driving control method as described in any one of claims 1 to 6.

Images