Unmanned vehicle reinforcement learning unknown environment perception method and system based on bird's-eye view

Abstract

The application provides an aerial view-based unknown environment perception method and system for unmanned vehicle reinforcement learning, comprising: collecting a depth image and converting the depth image to generate an aerial view image; inputting the aerial view image data into a convolution layer and a full connection layer of a neural network to generate a high-dimensional feature vector of the image data; collecting local point cloud data around the unmanned vehicle and inputting the local point cloud data into a multilayer perceptron to generate a feature vector of the terrain point cloud data; inputting pose data of the unmanned vehicle into two continuous full connection layers of the neural network through an IMU inertial sensor to generate a pose data encoding vector; decoding three modal feature information through the full connection layer and fusing the three modal feature information into a high-dimensional feature vector; performing action evaluation through the high-dimensional feature vector; outputting an action with the highest score according to the environmental characteristics; accepting and executing the selected action through a decision module of the unmanned vehicle; and interacting with the environment again through a perception module of the unmanned vehicle, and repeating the above steps until the destination is reached, so as to finally realize autonomous navigation of the unmanned vehicle.

Description

A reinforcement learning-based method and system for unmanned vehicles to perceive unknown environments based on bird's-eye view. Technical Field

[0001] This invention relates to the field of reinforcement learning technology, and more specifically, to a method and system for unmanned vehicles to perceive unknown environments based on bird's-eye view reinforcement learning. Background Technology

[0002] Reinforcement learning has significant advantages in handling autonomous vehicle action decisions, as it can consider the vehicle's expected actions and states in response to the environment, and is therefore widely used in the field of autonomous navigation. However, when autonomous vehicles need to move autonomously in unknown environments to complete tasks, finding the safest traversable area in their surroundings remains a very challenging task. Especially when facing complex environments containing vegetation, various terrain features, and dynamic obstacles, the different traversability of grass, vegetation, rocks, potholes, and unstructured obstacles makes it very difficult for the autonomous vehicle to correctly understand the traversability of its area ahead.

[0003] Reinforcement learning-based autonomous vehicle navigation requires processing and analyzing sensor data to perceive the environment. Current environmental perception methods often rely on prior human knowledge. For example, they might use visual semantic segmentation techniques to segment the input from visual sensors into pixels, then feed this into a reinforcement learning network to understand traversable terrain; or they might use visual and laser sensors to add human constraints or optimizations based on the geometric characteristics of the terrain, combining this with a probability distribution model to perceive the traversability probability of the surrounding environment, and then inputting an elevation map of the terrain into the reinforcement learning network to make decisions. These methods suffer from two main problems:

[0004] 1) Reliance on prior knowledge. Both deep learning-based terrain semantic segmentation and terrain geometric feature analysis rely on manually set prior knowledge (such as classification labels and scalar values). The network outputs perception results with the help of prior knowledge. This leads to the fact that when the difference between the unknown environment and the labeled environment is large, the perceived terrain information is often not accurate enough, and there will be problems of misjudgment and information loss. As a result, autonomous vehicles cannot complete navigation tasks well based on perception.

[0005] 2) Inconvenience of model transfer. Due to safety and economic considerations, reinforcement learning autonomous vehicle navigation training mostly needs to train models in simulation environments. Therefore, when deployed to the real environment, there is a problem of model transfer from virtual to reality. Complex perception models need to provide two different label values ​​for the corresponding virtual and real environments, which brings inconvenience to model transfer and affects the perception efficiency during actual operation. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a reinforcement learning-based method and system for unmanned vehicles to perceive unknown environments based on bird's-eye views.

[0007] According to the present invention, an unmanned vehicle reinforcement learning-based unknown environment perception method and system based on bird's-eye view is provided, the scheme of which is as follows:

[0008] Firstly, a reinforcement learning-based method for unmanned vehicles to perceive unknown environments based on bird's-eye views is provided, the method comprising:

[0009] Step S1: Acquire depth images and convert them into bird's-eye view images;

[0010] Step S2: Input the bird's-eye view image data into the convolutional layer and fully connected layer of the neural network to generate a high-dimensional feature vector of the image data;

[0011] Step S3: Local point cloud data around the unmanned vehicle is collected by LiDAR and input into a multilayer perceptron composed of convolutional layers and max pooling layers. First, the point cloud is normalized according to its spatial distribution. Then, the spatial distribution feature information of the point cloud is extracted by convolution and pooling operations to generate the feature vector of the terrain point cloud data.

[0012] Step S4: Input the pose data of the autonomous vehicle into two consecutive fully connected layers of the neural network through the IMU inertial sensor to generate a pose data encoding vector;

[0013] Step S5: Decode the feature vectors of image data, terrain point cloud data, and pose data through a fully connected layer, and fuse them into a high-dimensional feature vector;

[0014] Step S6: Construct an action value network and an optimal advantage function network. The two networks share the parameters of the neural network layer that generates high-dimensional feature vectors. The action value network updates its parameters in real time, while the optimal advantage function network does not update them in real time. The high-dimensional feature vectors are fed into the two networks respectively. The current environment is evaluated through the high-dimensional feature vectors, and an action set is output, in which each action contains a score.

[0015] Step S7: Select the highest-scoring action and output it in the form of angular velocity and linear velocity;

[0016] Step S8: The unmanned vehicle chassis receives the selected action and converts it into control commands for execution;

[0017] Step S9: Start executing from step S1 again, repeating the above steps until the destination is reached.

[0018] Preferably, the step S1 of converting and generating the bird's-eye view image includes:

[0019] Step S1.1: Before running, calibrate the camera, obtain the camera matrix and distance coefficients to eliminate all image distortions from the camera;

[0020] Step S1.2: Apply the Sobel operator to the depth map in the X direction of the image to calculate the absolute value, separate the gradient in the image, and give an image that can be used to separate obstacles;

[0021] Step S1.3: Perform bidirectional Sobel operator operations in the X and Y directions of the image to remove noise from the previous image;

[0022] Step S1.4: Calculate the gradient direction as the y-direction gradient divided by the arctangent of the x-direction gradient to further eliminate the remaining noise;

[0023] Step S1.5: Separate the luminance and saturation channels of the color image, and then take an absolute Sobel operator in the X direction of the luminance channel to activate the pixel.

[0024] Preferably, step S1.5 includes setting a threshold to activate a pixel if its saturation channel is within a hard-coded threshold, or if its luminance channel's scaled Sobel pixel is within a separate hard-coded threshold.

[0025] Preferably, step S6 includes: the high-dimensional feature vectors are respectively fed into the value function network and the optimal advantage function network, the network shares and fuses the parameters of the neural network layer that generates the high-dimensional feature vectors, and finally scores the actions in the action set.

[0026] Secondly, a reinforcement learning-based unknown environment perception system for autonomous vehicles based on bird's-eye view is provided, the system comprising:

[0027] Module M1: Acquires depth images and converts them into bird's-eye view images;

[0028] Module M2: Inputs the bird's-eye view image data into the convolutional and fully connected layers of the neural network to generate a high-dimensional feature vector of the image data;

[0029] Module M3: Local point cloud data around the unmanned vehicle is collected by LiDAR and input into a multilayer perceptron consisting of convolutional layers and max pooling layers. The data is first normalized according to the spatial distribution of the point cloud, and then the spatial distribution feature information of the point cloud is extracted by convolution and pooling operations to generate feature vectors of terrain point cloud data.

[0030] Module M4: Inputs the pose data of the autonomous vehicle into two consecutive fully connected layers of the neural network through the IMU inertial sensor to generate a pose data encoding vector;

[0031] Module M5: Decodes the feature vectors of image data, terrain point cloud data, and pose data through a fully connected layer, and fuses them into a high-dimensional feature vector;

[0032] Module M6: Constructs an action value network and an optimal advantage function network. The two networks share and fuse the parameters of the neural network layer that generates high-dimensional feature vectors. The action value network updates its parameters in real time, while the optimal advantage function network does not update them in real time. The high-dimensional feature vectors are fed into both networks respectively, and the current environment is evaluated through the high-dimensional feature vectors. A set of actions is output, in which each action contains a score.

[0033] Module M7: Selects the highest-scoring action and outputs it in the form of angular velocity and linear velocity;

[0034] Module M8: The unmanned vehicle chassis receives the selected action and converts it into control commands for execution;

[0035] Module M9: Start executing from module M1 again, looping through the above modules until the destination is reached.

[0036] Preferably, the conversion and generation of the bird's-eye view image in module M1 includes:

[0037] Module M1.1: Before running, calibrate the camera, obtain the camera matrix and distance coefficients to eliminate all image distortions from the camera;

[0038] Module M1.2: Applies the Sobel operator to the depth map in the X direction of the image to calculate the absolute value, separates the gradient in the image, and provides an image that can be used to separate obstacles;

[0039] Module M1.3: Performs bidirectional Sobel operator operations in the X and Y directions of the image to remove noise from the previous image;

[0040] Module M1.4: Calculates the gradient direction as the y-direction gradient divided by the arctangent of the x-direction gradient, further eliminating other noise;

[0041] Module M1.5: Separates the luminance and saturation channels of the color image, and then takes an absolute Sobel operator in the X direction of the luminance channel to activate the pixel.

[0042] Preferably, the module M1.5 includes: setting a threshold to activate a pixel if the saturation channel of a pixel is within a hard-coded threshold, or if the scaled Sobel pixel of its luminance channel is within a separate hard-coded threshold.

[0043] The module M6 includes: high-dimensional feature vectors are fed into the value function network and the optimal advantage function network respectively. The network shares and fuses the parameters of the neural network layer that generates the high-dimensional feature vectors, and finally scores the actions in the action set.

[0044] Thirdly, a computer-readable storage medium is provided that stores a computer program, which, when executed by a processor, implements the steps of the bird's-eye view-based unmanned vehicle reinforcement learning unknown environment perception method.

[0045] Fourthly, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the bird's-eye view-based unmanned vehicle reinforcement learning unknown environment perception method.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] 1. This invention is aimed at unmanned vehicle platforms for urban search and rescue or exploration of unknown environments. It can quickly perceive obstacle information in the surrounding environment under conditions of no road and no prior human information, and meet the requirements of real-time operation of unmanned vehicles and the transferability from virtual to reality.

[0048] 2. This invention establishes a multimodal reinforcement learning framework that integrates information from lidar, binocular vision, and IMU inertial sensors to quickly perceive obstacle and terrain information in unknown environments, thereby improving the autonomous exploration and navigation capabilities of unmanned vehicles in unknown environments without roads or prior human information.

[0049] Other beneficial effects of the present invention will be explained in detail through the introduction of specific technical features and technical solutions in specific embodiments. Those skilled in the art should be able to understand the beneficial technical effects brought about by these technical features and technical solutions through the introduction of these technical features and technical solutions. Attached Figure Description

[0050] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0051] Figure 1 shows the reinforcement learning interaction framework;

[0052] Figure 2 shows the process of generating a bird's-eye view. Detailed Implementation

[0053] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0054] This invention provides a reinforcement learning-based method for unmanned vehicles (UAVs) to perceive unknown environments based on bird's-eye view data. It is designed for mobile robot platforms used in urban search and rescue or exploration of unknown environments, specifically for autonomous navigation tasks in the absence of roads and prior human information. By utilizing depth cameras and LiDAR sensors, it rapidly generates a bird's-eye view of obstacles and a point cloud terrain map of the area in front of the UAV. This enables accurate environmental perception in unknown environments without human annotation. The bird's-eye view ensures convenient transfer of obstacle perception between virtual and real environments. Finally, a reinforcement learning framework is used to fuse multimodal information to achieve autonomous navigation for the UAV.

[0055] The autonomous vehicle is equipped with sensors including LiDAR, RGBD binocular cameras, and an IMU inertial sensor. The overall reinforcement learning framework, as shown in Figure 1, is divided into a basic modal perception module and a reinforcement learning policy module. The perception module uses a multimodal information encoding method for environmental perception (steps S1-S5 in this invention), collecting and processing sensor observation data. The policy module uses a reinforcement learning method to select actions to execute based on observations (steps S6-S7 in this invention), provides rewards according to the reward function, adjusts the action value function, and ultimately achieves accurate action output. The specific steps are as follows:

[0056] Step S1: Acquire depth images using a binocular camera and convert them to generate a bird's-eye view image;

[0057] Step S2: Input the bird's-eye view image data into the convolutional and fully connected layers of the neural network to generate a high-dimensional feature vector of the image data;

[0058] Step S3: Local point cloud data around the unmanned vehicle is collected by LiDAR and input into a multilayer perceptron composed of convolutional layers and max pooling layers. First, the point cloud is normalized according to its spatial distribution. Then, the spatial distribution feature information of the point cloud is extracted by convolution and pooling operations to generate the feature vector of the terrain point cloud data.

[0059] Step S4: Input the pose data of the autonomous vehicle into two consecutive fully connected layers of the neural network through the IMU inertial sensor to generate a pose data encoding vector;

[0060] Step S5: Decode the feature vectors of image data, terrain point cloud data, and pose data through a fully connected layer, and fuse them into a high-dimensional feature vector;

[0061] Step S6: Construct an action value network and an optimal advantage function network. The two networks share and fuse the parameters of the neural network layer that generates high-dimensional feature vectors. The action value network updates its parameters in real time, while the optimal advantage function network updates them non-instantly. The high-dimensional feature vectors are fed into both networks respectively, and the current environment is evaluated using these vectors. A set of actions is output, where each action contains a score.

[0062] Step S7: Select the highest-scoring action and output it in the form of angular velocity and linear velocity;

[0063] Step S8: The unmanned vehicle chassis receives the selected action and converts it into control commands for execution;

[0064] Step S9: Start from step 1 again and repeat the above steps until the destination is reached.

[0065] Specifically, referring to Figure 2, step S1, which involves converting and generating a bird's-eye view image, includes:

[0066] Step S1.1: Before the module runs, the camera is calibrated to eliminate distortion caused by the lens. The camera matrix and distortion parameter matrix are obtained using a checkerboard calibration board containing standard quadrilaterals of a specific size. The distortion characteristics are modeled based on the camera matrix. For each pixel position, the corrected coordinates are calculated based on the distortion model and the distortion parameter matrix. The corrected coordinates are then mapped back to the coordinate space of the distorted image using interpolation, thereby eliminating all image distortion.

[0067] Step S1.2: Apply a 3x3 Sobel operator to the depth map in the X direction of the image, multiply the pixel value at the corresponding position of the operator with the pixel in the image, and then sum them. The purpose of this step is to separate the horizontal gradient in the image and give an image that can be used to separate obstacles.

[0068] Step S1.3: Perform a bidirectional Sobel operator operation in the X and Y directions of the image to remove noise from the previous image;

[0069] Step S1.4: In order to accurately extract obstacle information, this step calculates the gradient direction as the y-direction gradient divided by the arctangent of the x-direction gradient, further eliminating other noise;

[0070] Step S1.5: Separate the luminance and saturation channels of the color image by converting the RGB image to the HSI color space. Then, take an absolute Sobel operator in the X direction of the luminance channel and set a threshold. If a pixel's saturation channel is within the hard-coded threshold, or if its luminance channel pixel value is within a separate hard-coded threshold, then activate the pixel.

[0071] The present invention also provides an unmanned vehicle reinforcement learning unknown environment perception system based on bird's-eye view. The unmanned vehicle reinforcement learning unknown environment perception system based on bird's-eye view can be implemented by executing the process steps of the unmanned vehicle reinforcement learning unknown environment perception method based on bird's-eye view. That is, those skilled in the art can understand the unmanned vehicle reinforcement learning unknown environment perception method based on bird's-eye view as a preferred embodiment of the unmanned vehicle reinforcement learning unknown environment perception system based on bird's-eye view.

[0072] This invention provides a method and system for unmanned vehicles to perceive unknown environments using reinforcement learning based on bird's-eye view. Compared with obstacle perception using semantic segmentation images, the advantages of using a bird's-eye view input reinforcement learning framework are:

[0073] (1) Segmentation maps rely on prior annotation information, while bird's-eye view maps rely only on depth maps to filter out obstacles, saving the manual and time costs of sample annotation.

[0074] (2) Bird’s-eye view can represent the information of obstacles in time and space. It is equivalent to a step of information concentration on the basis of depth map, making it easier for the network to focus on the spatial information of obstacles. It saves the network from the long process of learning obstacles from every pixel of depth map. Compared with semantic segmentation image, it can provide more intuitive and richer information, which helps the network to better understand the unknown environment and make more favorable judgments.

[0075] (3) Because bird's-eye view contains concentrated information (only the obstacle part needs to be considered), its resolution can be compressed more when input into the network. In contrast, segmented images need to consider the information of every pixel, which requires more computational space and slows down the inference time. Therefore, using bird's-eye view is more in line with the real-time requirements of autonomous vehicle navigation.

[0076] (4) When facing virtual to real migration deployment or significant changes in the surrounding environment, the bird's-eye view processing module does not need to perform additional operations, while the segmentation map needs to be re-annotated and retrained. Therefore, the bird's-eye view has better mobility and environmental adaptability.

[0077] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0078] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A reinforcement learning-based method for unmanned vehicles to perceive unknown environments based on bird's-eye view, characterized in that, include: Step S1: Acquire depth images and convert them into bird's-eye view images; Step S2: Input the bird's-eye view image data into the convolutional and fully connected layers of the neural network to generate a high-dimensional feature vector of the image data; Step S3: Input the local point cloud data around the unmanned vehicle collected by the LiDAR into a multilayer perceptron composed of convolutional and max-pooling layers. First, normalize the point cloud according to its spatial distribution, and then use convolution and pooling operations to extract the spatial distribution feature information of the point cloud to generate the feature vector of the terrain point cloud data; Step S4: Input the pose data of the unmanned vehicle into two consecutive fully connected layers of the neural network through the IMU inertial sensor to generate a pose data encoding vector; Step S5: Decode the feature vector of the image data, the feature vector of the terrain point cloud data, and the position data through the fully connected layer. Pose data encoding vectors are fused into high-dimensional feature vectors; Step S6: Construct an action value network and an optimal advantage function network. The two networks share the parameters of the neural network layer that generates the high-dimensional feature vectors. The action value network updates its parameters in real time, while the optimal advantage function network does not update in real time; The high-dimensional feature vectors are fed into both networks respectively, and the current environment is evaluated through the high-dimensional feature vectors, outputting an action set, in which each action contains a score; Step S7: Select the action with the highest score and output it in the form of angular velocity and linear velocity; Step S8: The autonomous vehicle chassis receives the selected action and converts it into control commands for execution; Step S9: Execute again from step S1, repeating the above steps until the destination is reached.

2. The method for unmanned vehicle unknown environment perception based on bird's-eye view reinforcement learning according to claim 1, characterized in that, The step S1 of generating the bird's-eye view image includes: Step S1.1: running the pre-calibrated camera to obtain the camera matrix and distance coefficients to eliminate all image distortions from the camera; Step S1.2: applying the Sobel operator to the depth map in the X direction of the image to calculate the absolute value, separating the gradient in the image, and providing an image that can be used to separate obstacles; Step S1.3: performing a bidirectional Sobel operator operation in the X and Y directions of the image to remove noise from the previous image; Step S1.4: calculating the gradient direction as the gradient in the y direction divided by the arctangent of the gradient in the x direction to further eliminate the remaining noise; Step S1.5: separating the brightness and saturation channels of the color image, and then taking an absolute Sobel operator in the X direction of the brightness channel to activate the pixel.

3. The method for unmanned vehicle unknown environment perception based on bird's-eye view according to claim 2, characterized in that, Step S1.5 includes setting a threshold to activate a pixel if its saturation channel is within a hard-coded threshold, or if its luminance channel's scaled Sobel pixel is within a separate hard-coded threshold.

4. The method for unmanned vehicle unknown environment perception based on bird's-eye view according to claim 1, characterized in that, Step S6 includes: the high-dimensional feature vectors will be fed into the value function network and the optimal advantage function network respectively, the network shares and fuses the parameters of the neural network layer that generates the high-dimensional feature vectors, and finally scores the actions in the action set.

5. A reinforcement learning-based unknown environment perception system for unmanned vehicles based on bird's-eye view, characterized in that, The system includes: Module M1: Acquires depth images and converts them into bird's-eye view images; Module M2: Inputs the bird's-eye view image data into the convolutional and fully connected layers of a neural network to generate high-dimensional feature vectors of the image data; Module M3: Acquires local point cloud data around the unmanned vehicle using LiDAR and inputs it into a multilayer perceptron composed of convolutional and max-pooling layers. The data is first normalized based on the spatial distribution of the point cloud, and then convolution and pooling operations are used to extract the spatial distribution features of the point cloud to generate feature vectors of the terrain point cloud data; Module M4: Inputs the pose data of the unmanned vehicle into two consecutive fully connected layers of a neural network using an IMU inertial sensor to generate pose data encoding vectors; Module M5: Decodes the image data feature vectors and terrain feature vectors through fully connected layers. The feature vectors of the shape point cloud data and the pose data encoding vectors are fused into a high-dimensional feature vector; Module M6: Constructs an action value network and an optimal advantage function network. The two networks share the parameters of the neural network layer that generates the high-dimensional feature vector. The action value network updates its parameters in real time, while the optimal advantage function network does not update in real time. The high-dimensional feature vector is fed into both networks respectively. The current environment is evaluated through the high-dimensional feature vector, and an action set is output, in which each action contains a score; Module M7: Selects the action with the highest score and outputs it in the form of angular velocity and linear velocity; Module M8: The autonomous vehicle chassis receives the selected action and converts it into control commands for execution; Module M9: Execution starts again from module M1, and the above modules are looped until the destination is reached.

6. The unmanned vehicle reinforcement learning unknown environment perception system based on bird's-eye view according to claim 5, characterized in that, The conversion and generation of the bird's-eye view image in module M1 includes: Module M1.1: Calibrate the camera before running, obtain the camera matrix and distance coefficients to eliminate all image distortions from the camera; Module M1.2: Apply the Sobel operator to the depth map in the X direction of the image to calculate the absolute value, separate the gradient in the image, and provide an image that can be used to separate obstacles; Module M1.3: Perform bidirectional Sobel operator operation in the X and Y directions of the image to remove noise from the previous image; Module M1.4: Calculate the gradient direction as the gradient in the y direction divided by the arctangent value of the gradient in the x direction, further eliminating the remaining noise; Module M1.5: Separate the brightness and saturation channels of the color image, and then take an absolute Sobel operator in the X direction of the brightness channel to activate the pixel.

7. The unmanned vehicle reinforcement learning unknown environment perception system based on bird's-eye view according to claim 6, characterized in that, The module M1.5 includes: setting a threshold to activate a pixel if its saturation channel is within a hard-coded threshold, or if its luminance channel's scaled Sobel pixel is within a separate hard-coded threshold.

8. The unmanned vehicle reinforcement learning unknown environment perception system based on bird's-eye view according to claim 5, characterized in that, The module M6 includes: high-dimensional feature vectors are fed into the value function network and the optimal advantage function network respectively. The network shares and fuses the parameters of the neural network layer that generates the high-dimensional feature vectors, and finally scores the actions in the action set.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the bird's-eye view-based reinforcement learning method for perceiving unknown environments for unmanned vehicles as described in any one of claims 1 to 4.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed by the processor, it implements the steps of the bird's-eye view-based reinforcement learning method for perceiving unknown environments for unmanned vehicles as described in any one of claims 1 to 4.

Images