End-to-end autonomous driving control method based on bird's eye view and deep reinforcement learning

Abstract

This application relates to the field of autonomous driving control technology, and in particular to an end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning. The method includes: when a target vehicle is detected to be in autonomous driving mode, acquiring multi-view image data of the target vehicle's environment, converting the multi-view image data into a bird's-eye view representation to generate a bird's-eye view feature map of the environment, and classifying each pixel in the bird's-eye view feature map to obtain a semantic segmentation map of the environment; inputting the bird's-eye view feature map and the semantic segmentation map into a target deep reinforcement learning network to output an autonomous driving control signal for the target vehicle, thereby controlling the target vehicle's movement. This solves the problems of high computational complexity, low adaptability to complex environments, and insufficient interpretability in related autonomous driving control systems, which reduce the overall performance of the system and fail to meet the user's driving needs.

Description

Technical Field

[0001] This application relates to the field of autonomous driving control technology, and in particular to an end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning. Background Technology

[0002] The autonomous driving control systems in related technologies rely on a range of advanced technologies, including sensor fusion such as LiDAR (Light Detection and Ranging), cameras, and radar; high-precision positioning such as GPS (Global Positioning System), IMU (Inertial Measurement Unit), and SLAM (Simultaneous Localization and Mapping); behavior prediction; and path planning and control algorithms such as MPC (Model Predictive Control) and PID (Proportional-Integral-Derivative Control) to achieve environmental perception, decision-making, and action control.

[0003] However, the autonomous driving control systems in related technologies have limitations, and the multiple independent modules increase the computational complexity of the system. The system has a low ability to adapt to complex environments and insufficient interpretability, which reduces the overall performance of the system, fails to meet the user's driving needs, and reduces the user's driving experience. These issues urgently need to be addressed. Summary of the Invention

[0004] This application is based on the inventor's understanding and insights into the following issues:

[0005] Autonomous driving control systems are one of the core technologies supporting autonomous vehicle navigation and operation. They utilize various sensors and algorithms to perceive, make decisions, and control actions based on the surrounding environment. An autonomous driving control system typically comprises several main components: a perception module, a localization module, a prediction module, a planning module, and a control module. The perception module uses various sensors such as LiDAR (Light Detection and Ranging), cameras, millimeter-wave radar, and ultrasonic sensors, combined with computer vision and deep learning technologies, to identify and classify surrounding objects and acquire real-time environmental information, including roads, obstacles, and traffic signals. The localization module uses GPS (Global Positioning System), IMU (Inertial Measurement Unit), and high-precision maps for vehicle localization. Commonly used technologies also include SLAM (Simultaneous Localization and Mapping) to determine the vehicle's accurate location on the map. The prediction module analyzes the movement patterns of traffic participants based on historical data and deep learning models to make safer decisions and predict the future behavior of surrounding traffic participants (such as vehicles and pedestrians). The planning module calculates the optimal driving path from the current location to the target location using global and local path planning algorithms, while also considering dynamic obstacles and traffic rules to formulate driving paths and strategies for the vehicle. The control module uses algorithms such as MPC (Model Predictive Control) and PID (Proportional-Integral-Derivative Control) to precisely control the vehicle's acceleration, braking, and steering to achieve smooth driving, translating the planned path into specific operating commands.

[0006] However, this modular approach suffers from four drawbacks: error propagation, high computational complexity, lack of end-to-end learning capability, and insufficient interpretability. Error propagation refers to the fact that the independence between modules means that an error in one module can affect the performance of subsequent modules, reducing the overall system reliability. High computational complexity arises because multiple independent modules require complex data transfer and processing, increasing the computational burden. The lack of end-to-end learning capability means that existing methods cannot fully utilize the advantages of deep learning technology, resulting in limited understanding of the environment and decision-making capabilities. Insufficient interpretability refers to the blurred boundaries between modules, making it difficult to explain the system's decision-making process as a whole, thus limiting user trust in autonomous driving systems and urgently requiring improvement.

[0007] This application provides an end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning to address the limitations of autonomous driving control systems in related technologies. Furthermore, multiple independent modules increase the computational complexity of the system, resulting in low adaptability to complex environments, insufficient interpretability, reduced overall system performance, and an inability to meet user driving needs and experience.

[0008] The first aspect of this application provides an end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning, comprising the following steps: when a target vehicle is detected to be in an autonomous driving condition, acquiring multi-view image data of the environment in which the target vehicle is located; based on a bird's-eye view feature extraction network, converting the multi-view image data into a bird's-eye view representation to generate a bird's-eye view feature map of the environment, and classifying each pixel in the bird's-eye view feature map to obtain a semantic segmentation map of the environment; inputting the bird's-eye view feature map and the semantic segmentation map into a target deep reinforcement learning network to output an autonomous driving control signal for the target vehicle, and controlling the target vehicle to drive according to the autonomous driving control signal.

[0009] Optionally, in one embodiment of this application, the step of converting the multi-view image data into a bird's-eye view representation based on the bird's-eye view feature extraction network includes: extracting target features from the multi-view image data to generate a target feature map; calculating the depth distribution of each pixel in the target feature map to obtain the target depth value corresponding to each pixel in the target feature map; determining the probability distribution of each depth value and generating a depth prediction map of the target feature map according to the probability distribution; determining the 3D coordinates of each pixel based on the depth prediction map and generating a 3D point cloud using the 3D coordinates of each pixel; and projecting the 3D point cloud onto the bird's-eye view grid, so that the multi-view image data is converted into the bird's-eye view representation.

[0010] Optionally, in one embodiment of this application, classifying each pixel in the bird's-eye view feature map to obtain a semantic segmentation map of the environment includes: extracting abstract features of the bird's-eye view feature map using an encoder in a pre-trained target network; and classifying each pixel in the bird's-eye view feature map based on the decoder in the pre-trained target network and the abstract features to obtain a semantic segmentation map of the environment.

[0011] Optionally, in one embodiment of this application, the step of inputting the bird's-eye view feature map and the semantic segmentation map into a target deep reinforcement learning network and outputting the autonomous driving control signal of the target vehicle includes: performing preliminary processing on the bird's-eye view feature map, the semantic segmentation map, and the state information of the target vehicle to obtain target-level features; using the target-level features to determine the target probability distribution of the target vehicle performing each action, and determining the autonomous driving control signal of the target vehicle based on the target probability distribution.

[0012] Optionally, in one embodiment of this application, determining the target probability distribution of the target vehicle performing each action using the features of the target level includes: determining the action space dimension of the target vehicle based on the features of the target level; and generating the target probability distribution of the target vehicle performing each action using the action space dimension.

[0013] A second aspect of this application provides an end-to-end autonomous driving control device based on bird's-eye view and deep reinforcement learning, comprising: a data acquisition module, configured to acquire multi-view image data of the environment in which the target vehicle is located when the target vehicle is detected to be in autonomous driving mode; an acquisition module, configured to convert the multi-view image data into a bird's-eye view representation based on a bird's-eye view feature extraction network to generate a bird's-eye view feature map of the environment, and classify each pixel in the bird's-eye view feature map to obtain a semantic segmentation map of the environment; and a control module, configured to input the bird's-eye view feature map and the semantic segmentation map into a target deep reinforcement learning network to output an autonomous driving control signal for the target vehicle, and control the target vehicle to drive according to the autonomous driving control signal.

[0014] Optionally, in one embodiment of this application, the acquisition module includes: a first extraction unit, configured to extract target features from the multi-view image data to generate a target feature map; a first acquisition unit, configured to calculate the depth distribution of each pixel in the target feature map to obtain the target depth value corresponding to each pixel in the target feature map; a first determination unit, configured to determine the probability distribution of each depth value and generate a depth prediction map of the target feature map based on the probability distribution; a generation unit, configured to determine the 3D coordinates of each pixel based on the depth prediction map and generate a 3D point cloud using the 3D coordinates of each pixel; and a second acquisition unit, configured to project the 3D point cloud onto a bird's-eye view grid, so that the multi-view image data is converted into the bird's-eye view representation.

[0015] Optionally, in one embodiment of this application, the acquisition module includes: a second extraction unit, used to extract abstract features of the bird's-eye view feature map using an encoder in a pre-trained target network; and a third acquisition unit, used to classify each pixel in the bird's-eye view feature map based on the decoder in the pre-trained target network and the abstract features, so as to obtain a semantic segmentation map of the environment.

[0016] Optionally, in one embodiment of this application, the control module includes: a fourth acquisition unit, configured to perform preliminary processing on the bird's-eye view feature map, the semantic segmentation map, and the state information of the target vehicle to obtain target-level features; and a second determination unit, configured to use the target-level features to determine the target probability distribution of the target vehicle performing each action, and to determine the autonomous driving control signal of the target vehicle based on the target probability distribution.

[0017] Optionally, in one embodiment of this application, the second determining unit includes: a determining subunit, configured to determine the action space dimension of the target vehicle based on the features of the target level; and a generating subunit, configured to generate a target probability distribution of the target vehicle performing each action using the action space dimension.

[0018] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning as described in the above embodiments.

[0019] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning.

[0020] A fifth aspect of this application provides a computer program product, including a computer program that, when executed, is used to implement the above-described end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning.

[0021] This application embodiment can, when the target vehicle is detected to be in autonomous driving mode, convert multi-view image data of the target vehicle's environment into a bird's-eye view representation, and classify each pixel in the generated bird's-eye view feature map of the environment to obtain a semantic segmentation map of the environment. Then, the bird's-eye view feature map and the semantic segmentation map are input into the target deep reinforcement learning network to output the autonomous driving control signal for the target vehicle, thereby controlling the vehicle's movement. This solves the problems of high computational complexity, low adaptability to complex environments, and insufficient interpretability in related technologies' autonomous driving control systems, which reduce the overall performance of the system, fail to meet user driving needs, and diminish the user's driving experience.

[0022] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0023] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0024] Figure 1 This is a flowchart of an end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning, according to an embodiment of this application.

[0025] Figure 2 This is a schematic diagram of an end-to-end autonomous driving control system based on bird's-eye view and deep reinforcement learning, according to an embodiment of this application.

[0026] Figure 3 This is a schematic diagram of the structure of an end-to-end autonomous driving control device based on bird's-eye view and deep reinforcement learning, according to an embodiment of this application.

[0027] Figure 4 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0028] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0029] The following describes an end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning, according to embodiments of this application, with reference to the accompanying drawings. Addressing the limitations of the autonomous driving control systems mentioned in the background section, including the increased computational complexity due to multiple independent modules, low adaptability to complex environments, insufficient interpretability, and reduced overall system performance, thus failing to meet user driving needs, this application provides an end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning. In this method, when the target vehicle is detected to be in autonomous driving mode, multi-view image data of the target vehicle's environment is converted into a bird's-eye view representation. Each pixel in the generated bird's-eye view feature map of the environment is then classified to obtain a semantic segmentation map of the environment. The bird's-eye view feature map and the semantic segmentation map are then input into the target deep reinforcement learning network to output the autonomous driving control signal for the target vehicle, thereby controlling the vehicle's movement. This solves the problems of high computational complexity, low adaptability to complex environments, insufficient interpretability, reduced overall system performance, and failure to meet user driving needs and experience in the related autonomous driving control systems.

[0030] This application establishes a hierarchical decoupled end-to-end autonomous driving control system based on BEV feature extraction. The system includes four modules: a multi-sensor input module, a BEV feature extraction network, a deep reinforcement learning policy network, and a semantic segmentation and decoding module, as detailed below:

[0031] (1) Multi-sensor input module: This module consists of multiple RGB cameras surrounding the vehicle, capable of capturing environmental images from different directions. This module provides real-time environmental perception data, ensuring the comprehensiveness and accuracy of the information, and providing a foundation for subsequent feature extraction and decision-making.

[0032] (2) BEV Feature Extraction Network: Employing a Lift-Splat architecture, this network transforms multi-view image data into a unified BEV representation. First, it upscales the 2D image into a 3D point cloud by predicting the depth distribution of each pixel (Lift step). Then, it projects these point clouds onto the BEV mesh (Splat step). This feature extraction network effectively integrates viewpoint information from different cameras, reducing information errors and improving the accuracy of environmental understanding.

[0033] (3) Deep Reinforcement Learning Policy Network: Combining the Actor-Critic architecture, the system takes high-dimensional features extracted from BEV features as input and outputs control signals (such as acceleration, braking, and steering angle). Through the DRL method, the system can automatically learn the optimal driving strategy in a dynamic environment, adjust vehicle behavior in real time, and enhance decision-making capabilities.

[0034] (4) Semantic Segmentation Decoding Module: This module uses a pre-trained network (such as ResNet) to decode BEV features, generating semantic segmentation results and providing interpretability information about objects in the environment. This module transforms high-dimensional features into easily understandable environmental information, improving the interpretability of the autonomous driving system and making the vehicle's decision-making process more transparent.

[0035] He2eAD (Hierarchical End-to-End Autonomous Driving) achieves an efficient and interpretable end-to-end autonomous driving solution by combining BEV (Bird's Eye View) feature extraction with DRL (Deep Reinforcement Learning). Unlike traditional modular approaches, this system utilizes inputs from multiple cameras to construct a BEV representation of the vehicle's surrounding environment, integrating perception, prediction, planning, and control into a unified framework. By directly mapping sensor inputs to control outputs, it simplifies information processing and reduces error propagation. Compared to existing autonomous driving control systems, the hierarchical decoupled end-to-end autonomous driving control system, through BEV representation, can capture environmental information more comprehensively, reducing collision rates and improving driving safety. It also enhances the control system's decision-making capabilities: integrating BEV feature extraction and DRL makes the system's decisions more accurate and flexible in complex urban environments.

[0036] Specifically, Figure 1 This is a flowchart illustrating an end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning, provided as an embodiment of this application.

[0037] like Figure 1 As shown, this end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning includes the following steps:

[0038] In step S101, when the target vehicle is detected to be in an autonomous driving mode, multi-view image data of the environment in which the target vehicle is located are collected.

[0039] It is understood that, in the case of detecting that the target vehicle is in an autonomous driving mode, for example, when the current driving area of ​​the vehicle meets the conditions for autonomous driving, the vehicle can be controlled to enter the autonomous driving mode, or the user can control the vehicle to enter the autonomous driving mode through physical buttons. When the vehicle enters the autonomous driving mode, the present application example can collect multi-angle environmental information of the vehicle's environment, i.e., multi-view image data, through the multi-sensor input module of the above system. The collected multi-view image data can also be calibrated, denoised and enhanced, thereby effectively improving the quality of the image data.

[0040] In step S102, based on the bird's-eye view feature extraction network, the multi-view image data is transformed into a bird's-eye view representation to generate a bird's-eye view feature map of the environment. Each pixel in the bird's-eye view feature map is then classified to obtain a semantic segmentation map of the environment.

[0041] It is understood that the embodiments of this application can use the BEV feature extraction network in the above system to transform multi-view image data into a bird's-eye view representation. For example, firstly, the 2D image is upscaled into a 3D point cloud, and then these 3D point clouds are projected onto the BEV grid to generate a bird's-eye view feature map of the environment. This can effectively integrate the perspective information from different cameras, reduce information errors, and improve the accuracy of environmental understanding. In addition, the semantic segmentation decoding module in the above system can be used to classify each pixel in the bird's-eye view feature map to obtain a semantic segmentation map of the environment, effectively providing interpretable information about objects in the environment.

[0042] In one embodiment of this application, multi-view image data is converted into a bird's-eye view representation based on a bird's-eye view feature extraction network, including: extracting target features from the multi-view image data to generate a target feature map; calculating the depth distribution of each pixel in the target feature map to obtain the target depth value corresponding to each pixel in the target feature map; determining the probability distribution of each depth value and generating a depth prediction map of the target feature map based on the probability distribution; determining the 3D coordinates of each pixel based on the depth prediction map and generating a 3D point cloud using the 3D coordinates of each pixel; and projecting the 3D point cloud onto the bird's-eye view grid, so that the multi-view image data is converted into a bird's-eye view representation.

[0043] In actual implementation, such as Figure 2 As shown, the embodiments of this application can construct a BEV feature extraction network. The BEV feature extraction network in the above system aims to convert multi-view image data into a bird's-eye view (BEV) representation to achieve a comprehensive understanding of the surrounding environment. The BEV feature extraction network consists of two main steps: the Lift step and the Splat step.

[0044] Specifically, the Lift step aims to convert 2D multi-view image data into 3D point clouds, predicting the depth information of each pixel in the target feature map extracted from the multi-view image data. The input is an image captured by a surround-view camera (such as front-view, rear-view, and side-view), usually in RGB format. Multiple convolutional layers are used to extract image features, and the ReLU activation function is used to increase non-linear expressive power. Through a series of convolutional operations, a target feature map is generated, capturing image information at different levels. For each pixel in the target feature map, a depth distribution is calculated through a fully connected layer to generate the depth value corresponding to each pixel. A Softmax layer is used to output the probability distribution of each depth value, forming a depth prediction map. The output generates the 3D coordinates (x, y, z) of each pixel, forming a 3D point cloud.

[0045] The Splat step aims to project the generated 3D point cloud onto the BEV mesh to form a unified BEV representation. This involves determining the size and resolution of the BEV mesh, typically centered on the vehicle and covering a certain area. Using the camera's intrinsic and extrinsic parameters, each point in the 3D point cloud is projected onto the BEV plane. A transformation matrix converts the coordinates of the 3D points to the BEV coordinate system. Sum pooling is then used to map 3D points from different viewpoints to their corresponding positions on the BEV mesh. For multiple points projected onto the same mesh cell, summation or averaging methods can be used to integrate features, generating the final BEV feature map. This effectively integrates data from multiple sensors, providing a unified and high-dimensional environmental representation while reducing information loss and error propagation.

[0046] Optionally, in one embodiment of this application, classifying each pixel in the bird's-eye view feature map to obtain a semantic segmentation map of the environment includes: extracting abstract features of the bird's-eye view feature map using an encoder in a pre-trained target network; and classifying each pixel in the bird's-eye view feature map based on the decoder and abstract features in the pre-trained target network to obtain a semantic segmentation map of the environment.

[0047] As one possible way to achieve this, such as Figure 2As shown in the embodiments of this application, a semantic segmentation network can be built. The goal of the semantic segmentation module in the above system is to classify each pixel in the input image (or bird's-eye view BEV features) into a specific category (such as vehicle, pedestrian, road, etc.). The semantic segmentation module typically adopts a convolutional neural network architecture, which can effectively capture spatial features and contextual information in the image. The input layer receives feature maps from the BEV feature extraction module or directly inputs the image. The input size is usually fixed (such as 256x256 or 512x512). The encoder part consists of convolutional layers and pooling layers. The convolutional layers consist of multiple convolutional layers, using the ReLU activation function to extract features. Each convolutional layer is followed by Batch Normalization to accelerate training and improve stability. The pooling layers use max pooling to gradually reduce the spatial dimension of the feature map and extract higher-level features. Generally, a 2×2 pooling window with a stride of 2 is used. The feature decoding and visualization part inputs the BEV features into the semantic segmentation network to generate a semantic segmentation map of the environment, implements visualization tools, and displays the segmentation results to enhance interpretability.

[0048] In step S103, the bird's-eye view feature map and semantic segmentation map are input into the target deep reinforcement learning network to output the autonomous driving control signal of the target vehicle, and the target vehicle is controlled to drive according to the autonomous driving control signal.

[0049] It is understood that, in the embodiments of this application, the bird's-eye view feature map and semantic segmentation map can be input into the target deep reinforcement learning network, such as the policy network of DRL (Deep Reinforcement Learning), to output the autonomous driving control signals of the target vehicle, such as acceleration, braking and steering angle, and control the target vehicle to drive according to the autonomous driving control signals. Thus, the embodiments of this application can automatically learn the optimal driving strategy in a dynamic environment, adjust vehicle behavior in real time, and enhance decision-making ability.

[0050] In one embodiment of this application, the bird's-eye view feature map and semantic segmentation map are input into the target deep reinforcement learning network to output the autonomous driving control signal of the target vehicle. This includes: performing preliminary processing on the bird's-eye view feature map, semantic segmentation map and the state information of the target vehicle to obtain target-level features; using the target-level features to determine the target probability distribution of each action performed by the target vehicle; and determining the autonomous driving control signal of the target vehicle based on the target probability distribution.

[0051] In this embodiment of the application, the features at the target level are the higher-level features extracted in this embodiment of the application.

[0052] In some embodiments, this application can implement the construction of a control policy network based on deep reinforcement learning. For example, the DRL-based policy network in the above system aims to learn the optimal control policy from the environmental state. This network typically adopts an Actor-Critic architecture, transforming the environmental state input into action policies and state value estimates. The input features of the input layer come from the feature map (such as BEV representation) of the BEV feature extraction module, vehicle state information (such as speed, acceleration, direction, etc.), and environmental information (such as road conditions, traffic signals, etc.). The input features are preliminarily processed through convolutional layers or fully connected layers to extract higher-level features. These higher-level features are used to determine the probability distribution of each action performed by the target vehicle, and the autonomous driving control signal of the vehicle is determined based on the target probability distribution, effectively improving vehicle stability and enhancing user trust.

[0053] Optionally, in one embodiment of this application, determining the target probability distribution of each action performed by the target vehicle using the features of the target level includes: determining the action space dimension of the target vehicle based on the features of the target level; and generating the target probability distribution of each action performed by the target vehicle using the action space dimension.

[0054] In some embodiments, the present application can utilize an Actor network to generate an action policy under a given state and output the probability distribution of each action. Convolutional layers are used to extract spatial features from the image, and fully connected layers use multiple fully connected layers to learn a high-dimensional representation from the input features. The output layer outputs the probability distribution of the action policy using a Softmax activation function based on the dimension of the action space. In addition, for continuous action spaces, a Gaussian distribution can be used to output the mean and standard deviation of the actions, which can flexibly handle various types of continuous actions and improve the stability and reliability of the vehicle.

[0055] Furthermore, in this embodiment, model training is a crucial step in deep reinforcement learning, involving data preparation, model construction, training configuration, and subsequent evaluation and optimization. The following is a detailed training process for the policy network and semantic segmentation module based on deep reinforcement learning:

[0056] The data preparation and collection process involves collecting RGB image data in the CARLA simulation environment to ensure the dataset covers various driving scenarios, such as urban roads, highways, and rural roads. Data augmentation techniques (such as rotation, scaling, translation, and color transformation) are applied to expand the training set and improve the model's robustness and generalization ability. In deep reinforcement learning, corresponding actions and rewards are recorded for each state. A suitable framework is selected, such as using a deep learning framework (such as TensorFlow or PyTorch) to build semantic segmentation and policy networks. Model layers are defined, including the input, output, and activation functions of each layer according to the aforementioned design. An initial learning rate is set, typically 0.001 or 0.0001, and a learning rate decay strategy may be used. The batch size for training is set (e.g., 16, 32) to balance training speed and memory usage. The number of training epochs is set, usually adjusted based on the performance of the validation set. The cross-entropy loss function is used to evaluate the difference between the predicted class distribution and the true label. Loss functions for Actors and Critics are set, including policy loss and value loss, and mean squared error (MSE) is used to evaluate state values.

[0057] Therefore, the introduction of the semantic segmentation module in this embodiment enables the embodiment to provide visualized results of environmental understanding, improves users' understanding and trust in autonomous driving decisions, and through end-to-end design, this embodiment achieves seamless connection between perception, decision-making and control, greatly simplifies the calculation process and improves real-time response capabilities.

[0058] The end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning proposed in this application can, when the target vehicle is detected to be in autonomous driving mode, convert multi-view image data of the target vehicle's environment into a bird's-eye view representation, and classify each pixel in the generated bird's-eye view feature map of the environment to obtain a semantic segmentation map of the environment. Then, the bird's-eye view feature map and the semantic segmentation map are input into the target deep reinforcement learning network to output the autonomous driving control signal of the target vehicle, thereby controlling the target vehicle's movement. This solves the problems of high computational complexity, low adaptability to complex environments, and insufficient interpretability in related technologies' autonomous driving control systems, which reduce the overall performance of the system, fail to meet user driving needs, and reduce the user's driving experience.

[0059] Next, referring to the accompanying drawings, an end-to-end autonomous driving control device based on bird's-eye view and deep reinforcement learning, according to an embodiment of this application, is described.

[0060] Figure 3 This is a block diagram of an end-to-end autonomous driving control device based on bird's-eye view and deep reinforcement learning, according to an embodiment of this application.

[0061] like Figure 3 As shown, the end-to-end autonomous driving control device 10 based on bird's-eye view and deep reinforcement learning includes: a data acquisition module 100, an acquisition module 200, and a control module 300.

[0062] Specifically, the acquisition module 100 is used to acquire multi-view image data of the environment in which the target vehicle is located when the target vehicle is detected to be in an autonomous driving mode.

[0063] The acquisition module 200 is used to transform multi-view image data into a bird's-eye view representation based on a bird's-eye view feature extraction network to generate a bird's-eye view feature map of the environment, and classify each pixel in the bird's-eye view feature map to obtain a semantic segmentation map of the environment.

[0064] The control module 300 is used to input the bird's-eye view feature map and semantic segmentation map into the target deep reinforcement learning network to output the autonomous driving control signal of the target vehicle, and control the target vehicle to drive according to the autonomous driving control signal.

[0065] Optionally, in one embodiment of this application, the acquisition module 200 includes: a first extraction unit, a first acquisition unit, a first determination unit, a generation unit, and a second acquisition unit.

[0066] The first extraction unit is used to extract target features from multi-view image data to generate a target feature map.

[0067] The first acquisition unit is used to calculate the depth distribution of each pixel in the target feature map to obtain the target depth value corresponding to each pixel in the target feature map.

[0068] The first determining unit is used to determine the probability distribution of each depth value and generate a depth prediction map of the target feature map based on the probability distribution.

[0069] The generation unit is used to determine the 3D coordinates of each pixel based on the depth prediction map and generate a 3D point cloud using the 3D coordinates of each pixel.

[0070] The second acquisition unit is used to project 3D point clouds onto a bird's-eye view grid, so that multi-view image data is converted into a bird's-eye view representation.

[0071] Optionally, in one embodiment of this application, the acquisition module includes a second extraction unit and a third acquisition unit.

[0072] The second extraction unit is used to extract abstract features from the bird's-eye view feature map using the encoder in the pre-trained target network.

[0073] The third acquisition unit is used to classify each pixel in the bird's-eye view feature map based on the decoder and abstract features in the pre-trained target network to obtain a semantic segmentation map of the environment.

[0074] Optionally, in one embodiment of this application, the control module includes: a fourth acquisition unit and a second determination unit.

[0075] The fourth acquisition unit is used to perform preliminary processing on the bird's-eye view feature map, semantic segmentation map and the state information of the target vehicle to obtain target-level features.

[0076] The second determining unit is used to determine the target probability distribution of the target vehicle performing each action by utilizing the features at the target level, and to determine the autonomous driving control signal of the target vehicle based on the target probability distribution.

[0077] Optionally, in one embodiment of this application, the second determining unit includes a determining subunit and a generating subunit.

[0078] Among them, the determination sub-unit is used to determine the motion space dimension of the target vehicle based on the characteristics of the target level.

[0079] The generation sub-unit is used to generate the target probability distribution of each action performed by the target vehicle using the action space dimension.

[0080] It should be noted that the foregoing explanation of the embodiment of the end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning also applies to the end-to-end autonomous driving control device based on bird's-eye view and deep reinforcement learning in this embodiment, and will not be repeated here.

[0081] The end-to-end autonomous driving control device based on bird's-eye view and deep reinforcement learning proposed in this application can, when the target vehicle is detected to be in autonomous driving mode, convert multi-view image data of the target vehicle's environment into a bird's-eye view representation, and classify each pixel in the generated bird's-eye view feature map of the environment to obtain a semantic segmentation map of the environment. Then, the bird's-eye view feature map and the semantic segmentation map are input into the target deep reinforcement learning network to output the autonomous driving control signal of the target vehicle, thereby controlling the target vehicle's movement. This solves the problems of high computational complexity, low adaptability to complex environments, and insufficient interpretability in related autonomous driving control systems, which reduce the overall performance of the system, fail to meet user driving needs, and reduce the user's driving experience.

[0082] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0083] The memory 401, the processor 402, and the computer program stored on the memory 401 and capable of running on the processor 402.

[0084] When the processor 402 executes the program, it implements the end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning provided in the above embodiments.

[0085] Furthermore, electronic devices also include:

[0086] Communication interface 403 is used for communication between memory 401 and processor 402.

[0087] The memory 401 is used to store computer programs that can run on the processor 402.

[0088] The memory 401 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0089] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized into address buses, data buses, control buses, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0090] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0091] Processor 402 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0092] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning.

[0093] This embodiment also provides a computer program product, including a computer program that, when executed, is used to implement the above-described end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning.

[0094] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0095] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0096] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0097] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0098] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0099] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0100] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0101] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. An end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning, characterized in that, Includes the following steps: When the target vehicle is detected to be in autonomous driving mode, multi-view image data of the environment in which the target vehicle is located are collected; Based on the bird's-eye view feature extraction network, the multi-view image data is transformed into a bird's-eye view representation to generate a bird's-eye view feature map of the environment. Each pixel in the bird's-eye view feature map is then classified to obtain a semantic segmentation map of the environment. The bird's-eye view feature map and the semantic segmentation map are input into the target deep reinforcement learning network to output the autonomous driving control signal of the target vehicle, and the target vehicle is controlled to drive according to the autonomous driving control signal; The step of inputting the bird's-eye view feature map and the semantic segmentation map into the target deep reinforcement learning network and outputting the autonomous driving control signal of the target vehicle includes: The bird's-eye view feature map, the semantic segmentation map, and the state information of the target vehicle are preliminarily processed to obtain target-level features. The target probability distribution for each action performed by the target vehicle is determined using the features of the target level, and the autonomous driving control signal of the target vehicle is determined based on the target probability distribution; Determining the target probability distribution for each action performed by the target vehicle using the features of the target level includes: The motion space dimension of the target vehicle is determined based on the characteristics of the target level. The target probability distribution for the target vehicle to perform each action is generated using the action space dimension.

2. The method according to claim 1, characterized in that, The bird's-eye view feature extraction network transforms the multi-view image data into a bird's-eye view representation, including: Extract target features from the multi-view image data to generate a target feature map; Calculate the depth distribution of each pixel in the target feature map to obtain the target depth value corresponding to each pixel in the target feature map; Determine the probability distribution for each depth value, and generate a depth prediction map of the target feature map based on the probability distribution; Based on the depth prediction map, the 3D coordinates of each pixel are determined, and a 3D point cloud is generated using the 3D coordinates of each pixel. The 3D point cloud is projected onto the bird's-eye view grid, so that the multi-view image data is transformed into the bird's-eye view representation.

3. The method according to claim 1, characterized in that, The step of classifying each pixel in the bird's-eye view feature map to obtain a semantic segmentation map of the environment includes: Abstract features of the bird's-eye view feature map are extracted using an encoder in a pre-trained target network; Based on the decoder in the pre-trained target network and the abstract features, each pixel in the bird's-eye view feature map is classified to obtain a semantic segmentation map of the environment.

4. An end-to-end autonomous driving control device based on bird's-eye view and deep reinforcement learning, characterized in that, include: The acquisition module is used to acquire multi-view image data of the environment in which the target vehicle is located when the target vehicle is detected to be in an autonomous driving mode. The acquisition module is used to convert the multi-view image data into a bird's-eye view representation based on a bird's-eye view feature extraction network to generate a bird's-eye view feature map of the environment, and classify each pixel in the bird's-eye view feature map to obtain a semantic segmentation map of the environment. The control module is used to input the bird's-eye view feature map and the semantic segmentation map into the target deep reinforcement learning network to output the autonomous driving control signal of the target vehicle, and control the target vehicle to drive according to the autonomous driving control signal; The control module includes: a fourth acquisition unit, used to perform preliminary processing on the bird's-eye view feature map, the semantic segmentation map, and the state information of the target vehicle to obtain target-level features; and a second determination unit, used to determine the target probability distribution of the target vehicle performing each action using the target-level features, and to determine the autonomous driving control signal of the target vehicle based on the target probability distribution. The second determining unit includes: a determining subunit, configured to determine the action space dimension of the target vehicle based on the features of the target level; and a generating subunit, configured to generate a target probability distribution of the target vehicle performing each action using the action space dimension.

5. The apparatus according to claim 4, characterized in that, The acquisition module includes: The first extraction unit is used to extract target features from the multi-view image data to generate a target feature map; The first acquisition unit is used to calculate the depth distribution of each pixel in the target feature map to obtain the target depth value corresponding to each pixel in the target feature map; The first determining unit is used to determine the probability distribution of each depth value and generate a depth prediction map of the target feature map based on the probability distribution. The generation unit is used to determine the 3D coordinates of each pixel based on the depth prediction map, and to generate a 3D point cloud using the 3D coordinates of each pixel. The second acquisition unit is used to project the 3D point cloud onto the bird's-eye view grid, so that the multi-view image data is converted into the bird's-eye view representation.

6. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, the processor executing the program to implement the end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning as described in any one of claims 1-3.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning as described in any one of claims 1-3.

8. A computer program product, comprising a computer program, characterized in that, The computer program is executed by a processor to implement the end-to-end autonomous driving control method based on bird's-eye view and deep reinforcement learning as described in any one of claims 1-3.

Images