Automatic driving 3D operation scene generation method based on aerial view perception

Abstract

The invention provides an automatic driving 3D operation scene generation method based on aerial view perception. The method comprises the following steps: collecting a multi-source data set of the self-driving automobile through a plurality of sensors, wherein the multi-source data set comprises visual information and distance information; generating a three-dimensional aerial view model of the autonomous vehicle according to the visual information and the distance information; and using the three-dimensional aerial view models at all moments to form a 3D operation scene based on aerial view perception of the self-driving automobile. According to the method, the spatial relation in a complex environment is considered, the automatic driving system can recognize surrounding road conditions, personnel flow and potential obstacles by researching the use of the bird's-eye view angle, and it is ensured that the automatic driving vehicle has high adaptive capacity and safety in the driving process.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method for generating a 3D operating scene for autonomous driving based on bird's-eye view perception. Background Art

[0002] Autonomous driving scenario generation is a core element of testing autonomous vehicles. It aims to ensure the functional safety of autonomous vehicles by simulating various traffic elements and traffic conditions that may be encountered in real driving environments. Common scenario generation methods are divided into mechanism modeling methods and data-driven methods. Safety-critical scenario generation pays special attention to constructing extreme situations that may cause serious traffic accidents to test the performance of autonomous vehicles. These safety-critical scenarios include not only complex and changeable road conditions and behavior patterns of traffic participants, but also other unforeseen environmental factors. By simulating scenarios that are difficult to encounter frequently in practice but are critical, it ensures that autonomous vehicles have emergency decision-making capabilities when facing potential fatal risks.

[0003] The mechanism modeling method relies on the researchers' understanding and professional experience of the internal laws of changes in the scenario system, and uses mathematical theories and formulas to describe the physical mechanisms behind the physical prototypes. For example, some scholars have proposed a method for adaptive generation of scenario libraries based on Bayesian optimization. The main idea is to compensate for the differences between the alternative model and the real system through an adaptive testing process, thereby generating a customized scenario library with higher efficiency and robustness. Other scholars have improved the search strategy of the Monte Carlo Tree (MCT) by increasing the rewards for different trajectories, making the strategy biased towards exploration rather than utilization, thus solving the problem of lack of scenario diversity.

[0004] Data-driven methods mine scene information from large amounts of data, and usually require the collection of massive amounts of scene data to generate a diverse scene library, where scene data collection relies on advanced cameras, radars, and other high-precision sensing devices. Some scholars have proposed a method for identifying system boundary scenes using a Gaussian process classification method, which can achieve scene division based on performance boundaries. Other scholars have proposed a method for effectively training scene feature extractors using autoencoders, which can identify boundary scenes in a large number of natural driving data sets.

[0005] The shortcomings of the autonomous driving scene generation methods in the above-mentioned prior art include: the existing scene generation methods usually process data based on the front view or side view, which leads to a relatively single perception of the environment. This limitation makes it difficult for the system to fully understand the spatial relationships in complex environments. For example, in busy city streets, information such as intersection conditions, pedestrian movements, and road obstacles is often ignored. At the same time, in complex and dynamic environments, existing scene generation methods are insufficient in identifying and tracking dynamic objects. Due to information delays, the resulting decisions may lead to increased accident risks. The dynamic recognition capabilities of many existing technologies are limited to the processing of static scenes and are difficult to adapt to rapidly changing environments. Summary of the invention

[0006] An embodiment of the present invention provides a method for generating a 3D operating scene for autonomous driving based on bird's-eye view perception, so as to effectively enhance the adaptability and safety of the autonomous driving vehicle during driving.

[0007] In order to achieve the above object, the present invention adopts the following technical scheme.

[0008] A method for generating a 3D operating scene for autonomous driving based on bird's-eye view perception, comprising:

[0009] Collecting a multi-source dataset of an autonomous vehicle through a variety of sensors, the multi-source dataset includes visual information and distance information;

[0010] generating a three-dimensional bird's-eye view model of the autonomous vehicle based on the visual information and the distance information;

[0011] The 3D bird's-eye view model at all times is used to construct a 3D operating scene of the autonomous driving car based on bird's-eye view perception.

[0012] Preferably, the multi-source data set of the autonomous driving vehicle is collected by multiple sensors, and the multi-source data set includes visual information and distance information, including:

[0013] Six cameras with different viewing angles are used to capture the operating environment of the self-driving car. The cameras are arranged around the self-driving car to cover the front, side and rear of the vehicle. The image data from the six viewing angles collected by the camera are aligned and fused using an image stitching algorithm to form 360-degree visual information.

[0014] The point cloud data of the autonomous driving car is collected by the laser radar installed on the autonomous driving car, and the distance information of the environment around the autonomous driving car is obtained based on the laser radar. The distance data represents the three-dimensional position of objects in the environment around the autonomous driving car.

[0015] Preferably, the generating of a three-dimensional bird's-eye view model of the autonomous driving vehicle according to the visual information and the distance information comprises:

[0016] The multi-source data set is input into the trained CNN convolutional neural network model, and the CNN model is used to extract features of the visual information. The CNN model extracts feature points in the visual information through multi-layer convolution and pooling operations, and maps the extracted feature points to a predefined bird's-eye view coordinate system through a geometric transformation algorithm. The geometric transformation algorithm includes the conversion of perspective and space coordinates; in the bird's-eye view coordinate system, the center position of the vehicle is the coordinate origin, the front of the vehicle is the positive direction of the Y axis, and the right is the positive direction of the X axis;

[0017] Each point in the point cloud data P represents the position of an object relative to the laser emitter. A sub-matching algorithm is used to find the correspondence between the feature points of the bird's-eye view and the point cloud data. The point cloud data and the feature points of the bird's-eye view are matched, corrected and fused, and the transformation matrix T is calculated. The corrected point cloud data is P'=TP, and a three-dimensional scene model is generated. After multiple rounds of iterations and model optimization, a three-dimensional bird's-eye view model of the autonomous driving car is generated. The three-dimensional bird's-eye view model includes the car body, other surrounding vehicles, pedestrians and their relative positions.

[0018] Preferably, the extracted feature points are mapped to a predefined bird's-eye view coordinate system by a geometric transformation algorithm, and the geometric transformation algorithm includes conversion of viewing angle and space coordinates, including:

[0019] Use perspective transformation to transform the image feature points extracted by the CNN model from the image coordinate system to the camera coordinate system. Apply homography matrix transformation according to the internal and external parameters of the camera to calculate the position of the feature points in the camera coordinate system. Use the transformation parameters from the vehicle body coordinate system to the bird's-eye view coordinate system to transform the feature point coordinates from the camera coordinate system to the bird's-eye view coordinate system.

[0020] 2D image coordinates P image =(u,v) T , where (u, v) is the pixel coordinate on the image plane, and the camera intrinsic parameter matrix is ​​K, then for a given pixel point P image , and get the camera coordinate system representation:

[0021]

[0022] The camera external parameters are represented by the transformation matrix T, including the rotation matrix R and the translation vector t. Convert from camera coordinate system to bird's-eye view coordinate system:

[0023]

[0024] Preferably, the three-dimensional bird's-eye view model at all times is used to form a 3D operation scene of the autonomous driving vehicle based on bird's-eye view perception, including:

[0025] Based on the 3D bird's-eye view model at the current time t, the features of the 3D bird's-eye view at time t+1 are aligned. When different obstacles exist at the same position of the two, the two obstacles are marked at the same time to generate the 3D bird's-eye view model at time t+1. The 3D bird's-eye view models at all times are used to construct a 3D operation scene of the autonomous driving car based on bird's-eye perception.

[0026] It can be seen from the technical solutions provided by the above-mentioned embodiments of the present invention that the present invention proposes a method for generating a 3D operating scene for autonomous driving based on bird's-eye view perception. The method takes into account the spatial relationship in a complex environment, effectively reduces the visual blind spots by studying the use of a bird's-eye view, and provides a more comprehensive environmental information analysis approach, so that the autonomous driving system can identify the surrounding road conditions, personnel flow and potential obstacles, ensuring that the autonomous driving vehicle has strong adaptability and safety during driving.

[0027] This invention will provide practical suggestions for dynamic object recognition and tracking. By introducing deep learning technology, the ability to recognize dynamic objects is improved. By using advanced algorithms such as convolutional neural networks, the system can recognize and predict the behavior of dynamic objects in real time, thereby significantly reducing the risk of accidents. It not only enhances the safety of autonomous driving vehicles, but also improves their stability and driving efficiency in complex traffic conditions.

[0028] Additional aspects and advantages of the present invention will be given in part in the following description, which will become obvious from the following description, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0030] Figure 1 A schematic diagram of an implementation method of a method for generating a 3D operating scene for autonomous driving based on bird's-eye view perception provided in an embodiment of the present invention;

[0031] Figure 2 A processing flow chart of a method for generating a 3D operating scene for autonomous driving based on bird's-eye view perception provided by an embodiment of the present invention;

[0032] Figure 3 A schematic diagram of a three-dimensional bird's-eye view model of an autonomous driving vehicle provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0034] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or coupling. The term "and / or" used herein includes any unit and all combinations of one or more associated listed items.

[0035] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.

[0036] To facilitate understanding of the embodiments of the present invention, several specific embodiments will be further explained below with reference to the accompanying drawings, and each embodiment does not constitute a limitation on the embodiments of the present invention.

[0037] Bird's-eye perception: refers to observing the environment from a fixed height above the ground to provide a comprehensive understanding of the spatial layout and object distribution. This perspective is usually presented in the form of a plan view, which can intuitively display the relative position and dynamic behavior of traffic elements (such as roads, vehicles, pedestrians, traffic signs, etc.). The introduction of a bird's-eye view can provide more comprehensive environmental information and reduce visual blind spots.

[0038] This paper proposes a method for generating 3D operation scenes for autonomous driving based on bird's-eye view perception, which improves the environmental perception capability, decision-making accuracy and operational safety of the autonomous driving system. The solution integrates multiple sensor data and uses deep learning technology to build a highly reliable environmental perception model.

[0039] The implementation principle diagram of a method for generating a 3D operation scene for autonomous driving based on bird's-eye view perception provided by an embodiment of the present invention is as follows: Figure 1 As shown, the processing flow of this method is as follows Figure 2 As shown, the processing steps include the following:

[0040] Step S10: Collect a multi-source data set of the autonomous driving vehicle through a variety of sensors, where the multi-source data set includes visual information and distance information.

[0041] In the operation of the autonomous driving system, sensor data is the basis for achieving effective environmental perception. The present invention uses a variety of sensors (such as lidar, camera, etc.) to collect relevant data of the autonomous driving car. The complementary characteristics of various sensors ensure that the system can obtain comprehensive information in a variety of complex scenarios.

[0042] The present invention uses six cameras with different viewing angles to shoot the operating environment of the autonomous vehicle. These cameras are arranged along the periphery of the autonomous vehicle, covering the front, side and rear of the vehicle. The image data of the six viewing angles collected by the cameras are aligned and fused using an image stitching algorithm to ensure seamless connection and minimize overlap and blind spots, forming 360-degree visual information, thereby achieving three-dimensional perception of the operating environment of the autonomous vehicle.

[0043] The laser radar installed on the self-driving car collects high-precision point cloud data of the self-driving car. The laser radar obtains distance information of the environment around the self-driving car by emitting laser beams and receiving reflected signals. The distance data can accurately measure the three-dimensional position of objects in the environment to form a dense three-dimensional point cloud. The point cloud data is processed to remove noise and redundant points to ensure that the generated point cloud accurately reflects the actual characteristics of the environment.

[0044] Step S20: Generate a three-dimensional bird's-eye view model of the autonomous driving vehicle based on the above visual information and distance information.

[0045] The visual information and distance information collected by the above sensors include the relative position relationship between static objects (such as buildings and traffic signs) and dynamic objects (such as pedestrians and vehicles).

[0046] The present invention uses existing classic CNN models such as ResNet-50 and ResNet-101 to extract image features. In training, the existing data sets (such as KITTI, nuScenes, etc.) are first used to pre-train ResNet-50 models, and the model effect is evaluated by defining a loss function (such as cross entropy loss), and then the network weights are continuously updated through back propagation to achieve the expected effect.

[0047] In the generation of bird's-eye view perception scenes, visual information is input into a trained CNN (Convolutional Neural Networks) model, and the CNN model is used to extract features from the visual information. The CNN model is optimized for environmental perception. The CNN model extracts feature points from the visual information through multi-layer convolution and pooling operations, making subsequent coordinate mapping more accurate and efficient. The extracted feature points are mapped to a predefined bird's-eye view coordinate system through a geometric transformation algorithm. The above-mentioned geometric transformation algorithm includes the conversion of perspective and spatial coordinates.

[0048] In the bird's-eye view coordinate system, the center of the vehicle is usually used as the origin of the coordinate system, the front of the vehicle is the positive direction of the Y axis, and the right is the positive direction of the X axis. First, use perspective transformation to transform the image feature points from the image coordinate system to the camera coordinate system. According to the internal and external parameters of the camera, apply Homography transformation to calculate the position of the feature points in the camera coordinate system. Through the known external parameters (transformation from the vehicle body coordinate system to the bird's-eye view coordinate system), the coordinates of the feature points are transformed from the camera coordinate system to the bird's-eye view coordinate system.

[0049] The two-dimensional image coordinates P image =(u,v) T , where (u, v) is the pixel coordinate on the image plane, and the camera intrinsic parameter matrix is ​​K, then for a given pixel point P image , we can get the camera coordinate system representation:

[0050]

[0051] The camera external parameters are represented by the transformation matrix T, including the rotation matrix R and the translation vector t. Convert from camera coordinate system to bird's-eye view coordinate system:

[0052]

[0053] Ensure that objects in the image can be accurately located from a bird's-eye view. The key to this step is to reasonably calculate the relationship between the camera and the bird's-eye view observation point to ensure that the generated bird's-eye view can reflect the actual environment layout.

[0054] After the bird's-eye view is established, an accurate 3D model will be generated under supervision. The supervision here mainly comes from the point cloud data of the lidar obtained earlier. The point cloud data provides accurate depth information of objects in the environment. The point cloud data generated by the lidar is matched, corrected and fused with the feature points extracted from the bird's-eye view to generate a 3D scene model.

[0055] First, use the sub-matching algorithm (such as nearest neighbor search, etc.) to find the correspondence between the feature points of the bird's-eye view and the point cloud features, and calculate the transformation matrix T by matching the feature points. The matched and corrected point cloud data is P'=TP

[0056] Each point in the point cloud data represents the position of an object relative to the laser emitter, which makes the 3D scene model not only valid on the plane, but also accurate in three-dimensional space. After multiple rounds of iterations and model optimization, the method finally outputs an accurate 3D bird's-eye view model of the self-driving car, ensuring that the generated 3D bird's-eye view model is spatially consistent with the actual environment.

[0057] Figure 3 A schematic diagram of a three-dimensional bird's-eye view model of an autonomous driving vehicle provided in an embodiment of the present invention, such as Figure 3 As shown, the three-dimensional bird's-eye view model of the autonomous vehicle includes the vehicle itself, other surrounding vehicles, pedestrians and their relative positions.

[0058] Step S30: Use the three-dimensional bird's-eye view model at all times to construct a 3D operating scene of the autonomous driving vehicle based on bird's-eye view perception.

[0059] The above-mentioned three-dimensional bird's-eye view model can provide comprehensive environmental information for the autonomous driving system and promote subsequent real-time dynamic monitoring and decision support.

[0060] Based on the 3D bird's-eye view model at the current time t, the features of the 3D bird's-eye view at time t+1 are aligned. When different obstacles exist at the same position of the two, the two obstacles are marked at the same time to prevent the occurrence of perspective overlap problems, and the 3D bird's-eye view model at time t+1 is generated.

[0061] In this process, by integrating new data, the method can correct and optimize previous predictions to ensure the accuracy and real-time performance of the 3D bird's-eye view model. The 3D bird's-eye view model at all times is used to construct a 3D operating scene based on bird's-eye perception for the autonomous vehicle.

[0062] The 3D running scene generation method based on bird's-eye view perception of the present invention is particularly suitable for dynamic object recognition and tracking in an autonomous driving system. This process can use the previous recognition results to speed up the generation speed and improve the robustness of the method.

[0063] In summary, the embodiment of the present invention combines the camera and lidar data of six viewing angles to construct an environmental model with a bird's-eye view. This all-round perspective can reduce visual blind spots, achieve global perception of environmental information, and enable the autonomous driving system to have a deeper understanding of complex road conditions and the behavior of traffic participants.

[0064] The present invention proposes real-time recognition and tracking of dynamic objects, using deep learning technology and combining historical trajectory data to perform real-time recognition and motion prediction of dynamic objects based on a bird's-eye view. Through the continuously updated bird's-eye view, the system can not only quickly identify dynamic objects, but also predict their motion trajectories. This capability significantly improves the vehicle's driving safety and emergency decision-making capabilities, and provides a guarantee for avoiding potential accidents.

[0065] This invention will provide practical suggestions for the current autonomous driving technology in complex environment modeling, dynamic object recognition, real-time updating, and bird's-eye view applications, and provide a theoretical basis for improving the environmental perception capabilities of autonomous driving systems, thereby promoting the further development and application of autonomous driving technology.

[0066] The method for generating 3D operating scenes for autonomous driving based on bird's-eye view perception proposed in the present invention integrates camera and lidar data, extracts image features using a convolutional neural network, and maps them into a bird's-eye view through coordinate transformation, thereby effectively reducing blind spots in viewing angles and improving the comprehensiveness of environmental information, enabling the autonomous driving system to have a deeper understanding of complex environmental layouts.

[0067] 2. The method for generating 3D operation scenes for autonomous driving based on bird's-eye view perception proposed in this invention realizes real-time recognition and tracking of dynamic objects. By using time series data dependency, the autonomous driving system can efficiently analyze the surrounding dynamic environment and ensure the correct prediction of the movement of pedestrians and other vehicles, thereby reducing the risk of accidents.

[0068] 3. The present invention can provide real-time path planning and decision support for the autonomous driving system. The system can flexibly adjust the driving route in complex environments, effectively avoid conflicts with dynamic objects, and ensure driving safety and efficiency.

[0069] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0070] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention or certain parts of the embodiments.

[0071] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0072] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for generating a 3D operating scene for autonomous driving based on bird's-eye view perception, characterized in that: include: Collecting a multi-source dataset of an autonomous vehicle through a variety of sensors, the multi-source dataset includes visual information and distance information; generating a three-dimensional bird's-eye view model of the autonomous vehicle based on the visual information and the distance information; The 3D bird's-eye view model at all times is used to construct a 3D operating scene of the autonomous driving car based on bird's-eye view perception.

2. The method according to claim 1, characterized in that The multi-source data set of the autonomous driving vehicle is collected by multiple sensors, and the multi-source data set includes visual information and distance information, including: Six cameras with different viewing angles are used to capture the operating environment of the self-driving car. The cameras are arranged around the self-driving car to cover the front, side and rear of the vehicle. The image data from the six viewing angles collected by the camera are aligned and fused using an image stitching algorithm to form 360-degree visual information. The point cloud data of the autonomous driving car is collected by the laser radar installed on the autonomous driving car, and the distance information of the environment around the autonomous driving car is obtained based on the laser radar. The distance data represents the three-dimensional position of objects in the environment around the autonomous driving car.

3. The method according to claim 2, characterized in that The method of generating a three-dimensional bird's-eye view model of the autonomous driving vehicle according to the visual information and the distance information includes: The multi-source data set is input into the trained CNN convolutional neural network model, and the CNN model is used to extract features of the visual information. The CNN model extracts feature points in the visual information through multi-layer convolution and pooling operations, and maps the extracted feature points to a predefined bird's-eye view coordinate system through a geometric transformation algorithm. The geometric transformation algorithm includes the conversion of perspective and space coordinates; in the bird's-eye view coordinate system, the center position of the vehicle is the coordinate origin, the front of the vehicle is the positive direction of the Y axis, and the right is the positive direction of the X axis; Each point in the point cloud data P represents the position of an object relative to the laser emitter. A sub-matching algorithm is used to find the correspondence between the feature points of the bird's-eye view and the point cloud data. The point cloud data and the feature points of the bird's-eye view are matched, corrected and fused, and the transformation matrix T is calculated. The corrected point cloud data is P'=TP, and a three-dimensional scene model is generated. After multiple rounds of iterations and model optimization, a three-dimensional bird's-eye view model of the autonomous driving car is generated. The three-dimensional bird's-eye view model includes the car body, other surrounding vehicles, pedestrians and their relative positions.

4. The method according to claim 3, characterized in that The extracted feature points are mapped to a predefined bird's-eye view coordinate system through a geometric transformation algorithm, and the geometric transformation algorithm includes the conversion of viewing angle and space coordinates, including: Use perspective transformation to transform the image feature points extracted by the CNN model from the image coordinate system to the camera coordinate system. Apply homography matrix transformation according to the internal and external parameters of the camera to calculate the position of the feature points in the camera coordinate system. Use the transformation parameters from the vehicle body coordinate system to the bird's-eye view coordinate system to transform the feature point coordinates from the camera coordinate system to the bird's-eye view coordinate system. 2D image coordinates P image =(u,v) T , where (u, v) is the pixel coordinate on the image plane, and the camera intrinsic parameter matrix is ​​K, then for a given pixel point P image , and get the camera coordinate system representation: The camera external parameters are represented by the transformation matrix T, including the rotation matrix R and the translation vector t. Convert from camera coordinate system to bird's-eye view coordinate system:

5. The method according to claim 2, characterized in that: The 3D bird's-eye view model at all times is used to form a 3D operation scene of the autonomous driving vehicle based on bird's-eye view perception, including: Based on the 3D bird's-eye view model at the current time t, the features of the 3D bird's-eye view at time t+1 are aligned. When different obstacles exist at the same position of the two, the two obstacles are marked at the same time to generate the 3D bird's-eye view model at time t+1. The 3D bird's-eye view models at all times are used to construct a 3D operation scene of the autonomous driving car based on bird's-eye perception.

Images