Vehicle driving control method, vehicle and computing platform

Abstract

The invention relates to a vehicle driving control method, a vehicle and a computing platform, and the method comprises the steps: carrying out the three-dimensional point cloud data collection and all-round view image data collection of an environment where the vehicle is located, so as to obtain a three-dimensional point cloud frame and an aerial view image feature; rasterizing point clouds in the three-dimensional point cloud frame to obtain grid point clouds corresponding to a plurality of grids; determining a target deep learning model according to the distance between the grid and the vehicle; performing feature extraction on the grid point clouds in the grids by adopting a target deep learning model to obtain point cloud features of the grid point clouds; projecting the aerial view image features into a plurality of grids; fusing the aerial view image features and the point cloud features in the same grid to obtain fused features; identifying the fusion feature to obtain a target detection result of the three-dimensional point cloud frame and a classification detection result of the plurality of grids; and controlling the vehicle to run according to the target detection result of the three-dimensional point cloud frame and the classification detection result of the plurality of grids. And the accuracy of target detection is improved.

Description

Technical Field

[0001] This disclosure relates to the field of vehicle engineering technology, and more specifically, to a vehicle driving control method, a vehicle, and a computing platform. Background Technology

[0002] Vehicles employ machine vision systems for environmental perception, using sensors to collect information about their surroundings and processing the data through machine learning algorithms to achieve functions such as autonomous driving, localization, and parking. This allows them to better adapt to complex delivery environments and improve delivery efficiency. However, when performing full-scene perception and recognition, directly processing point cloud data is not only computationally intensive but also makes it difficult to extract effective features, resulting in high computational demands and low recognition accuracy, thus lowering the safety of autonomous driving. Summary of the Invention

[0003] The purpose of this disclosure is to provide a vehicle driving control method, a vehicle, and a computing platform to improve the perception effect in all scenarios and enhance the accuracy of target detection, thereby improving the safety of autonomous driving.

[0004] To achieve the above objectives, the first aspect of this disclosure provides a method for controlling the driving of a vehicle, comprising: The environment in which the vehicle is located is subjected to the acquisition of three-dimensional point cloud data and surround view image data to obtain three-dimensional point cloud frames and bird's-eye view image features, respectively. The point cloud in the three-dimensional point cloud frame is rasterized to obtain raster point clouds corresponding to multiple raster grids; The target deep learning model is determined based on the distance between the grid and the vehicle; The target deep learning model is used to extract features from the grid point cloud in the grid to obtain the point cloud features of the grid point cloud; The bird's-eye view features are projected onto the plurality of grids; The bird's-eye view features located in the same grid are fused with the point cloud features to obtain fused features; The fused features are identified to obtain the target detection results of the three-dimensional point cloud frame and the classification detection results of the multiple grids; The vehicle's movement is controlled based on the target detection results of the three-dimensional point cloud frame and the classification detection results of the multiple grids.

[0005] Optionally, determining the target deep learning model based on the distance between the grid and the vehicle includes: Determine whether the distance between the grid and the vehicle exceeds a preset distance threshold; If the distance does not exceed the preset distance threshold, a first deep learning model is determined as the target deep learning model; if the distance exceeds the preset distance threshold, a second deep learning model is determined as the target deep learning model; wherein, the first deep learning model is used for feature extraction of a regular convolutional neural network, and the second deep learning model is used for feature extraction of a sparse convolutional neural network.

[0006] Optionally, there are multiple 3D point cloud frames, and the step of rasterizing the point clouds in the 3D point cloud frames to obtain multiple raster point clouds corresponding to the raster grids includes: Based on the initial transformation relationship of multiple 3D point cloud frames, determine the rotation and translation matrix between the multiple 3D point cloud frames; The point clouds in the plurality of three-dimensional point cloud frames are aligned according to the rotation and translation matrix; The aligned point cloud is rasterized to obtain the raster point cloud corresponding to the multiple raster grids.

[0007] Optionally, controlling the vehicle's movement based on the target detection results of the three-dimensional point cloud frame and the classification detection results of the multiple grids includes: Determine the preceding frame from the plurality of three-dimensional point cloud frames; The target detection results based on the three-dimensional point cloud frame and the classification detection results of the multiple grids are converted to polar coordinates. The target detection results of the three-dimensional point cloud frame in the polar coordinate system and the classification detection results of the multiple grids are projected onto the preceding frame to temporally align the target detection results of the three-dimensional point cloud frame and the classification detection results of the multiple grids with the preceding frame. Multiple time-aligned target detection results and classification detection results are temporally fused to obtain a fused detection result and an occupancy detection result indicating whether the grid is occupied. The vehicle's movement is controlled based on the fusion detection results and the occupancy detection results.

[0008] Optionally, the target detection result is used to characterize the target object present in the three-dimensional point cloud frame, and the classification detection result is used to characterize the target object to which the raster belongs.

[0009] Optionally, surround view image data of the environment in which the vehicle is located is acquired to obtain bird's-eye view image features, including: Based on the intrinsic parameter matrix of the image acquisition device using the panoramic image data, the image features of the panoramic image data in image coordinates are converted into three-dimensional points in camera coordinates; Based on the extrinsic parameter matrix of the image acquisition device, the three-dimensional points are rotated and translated to convert the three-dimensional points in the camera coordinates into three-dimensional points in the world coordinates. Based on the position information of the image acquisition device relative to the vehicle, the three-dimensional points in the world coordinates are mapped to the two-dimensional image plane to obtain the bird's-eye view image features.

[0010] Optionally, the step of mapping three-dimensional points in world coordinates to a two-dimensional image plane based on the position information of the image acquisition device relative to the vehicle to obtain the bird's-eye view image features includes: The acquisition direction of the image acquisition device is determined based on its position relative to the vehicle. Using the vehicle's position as a reference point, and based on the acquisition direction and a preset perspective transformation matrix, the three-dimensional points in the world coordinates are mapped onto the two-dimensional image plane to obtain the bird's-eye view image features.

[0011] Optionally, the factors for setting the preset distance threshold include the vehicle's perception capabilities, safety requirements, or mission requirements.

[0012] A second aspect of this disclosure provides a vehicle, comprising: A three-dimensional point cloud data acquisition device is used to acquire three-dimensional point cloud data of the environment in which the vehicle is located; A surround view image data acquisition device is used to acquire surround view image data of the environment in which the vehicle is located; a memory having a computer program stored thereon; A processor for executing the computer program in the memory to implement the steps of the method of any one of the first aspects.

[0013] A third aspect of this disclosure provides a computing platform, comprising: A three-dimensional point cloud data receiving device is used to receive three-dimensional point cloud data of the environment in which the vehicle is located; A surround view image data receiving device, used to receive surround view image data of the environment in which the vehicle is located; a memory having a computer program stored thereon; A processor for executing the computer program in the memory to implement the steps of the method of any one of the first aspects.

[0014] The above technical solution can achieve at least the following beneficial effects: The system acquires 3D point cloud data and surround view image data of the vehicle's environment to obtain 3D point cloud frames and bird's-eye view features, respectively. The point clouds in the 3D point cloud frames are rasterized to obtain raster point clouds corresponding to multiple raster cells. A target deep learning model is determined based on the distance between the raster cells and the vehicle. Using the target deep learning model, features are extracted from the raster point clouds to obtain point cloud features. Bird's-eye view features are projected onto multiple raster cells. Bird's-eye view features and point cloud features located in the same raster cell are fused to obtain fused features. The fused features are then recognized to obtain target detection results for the 3D point cloud frames and classification detection results for multiple raster cells. Vehicle movement is controlled based on the target detection results for the 3D point cloud frames and the classification detection results for multiple raster cells. Different deep learning models are used to extract the most effective features for objects at different distances, thereby improving the efficiency and accuracy of point cloud feature extraction. Furthermore, information from different sources (such as image features and point cloud features) is fused to improve the accuracy and robustness of recognition. This improves the perception effect across all scenarios and enhances the accuracy of target detection, thereby improving the safety of autonomous driving.

[0015] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating a vehicle driving control method according to an exemplary embodiment.

[0017] Figure 2 This is an implementation illustrated according to an exemplary embodiment. Figure 1 The flowchart of S2 in the middle.

[0018] Figure 3 This is an implementation illustrated according to an exemplary embodiment. Figure 1 The flowchart of S8 in the middle.

[0019] Figure 4 This is a flowchart illustrating another method for controlling the driving of a vehicle according to an exemplary embodiment.

[0020] Figure 5 This is an implementation illustrated according to an exemplary embodiment. Figure 1 The flowchart of S1.

[0021] Figure 6 This is an implementation illustrated according to an exemplary embodiment. Figure 5 The flowchart of S13.

[0022] Figure 7 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0023] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0024] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with authorization from the owner of the relevant device.

[0025] Before introducing the vehicle driving control method, vehicle, and computing platform provided in the embodiments of this disclosure, the technical means in the relevant scenario are first introduced. In the relevant scenario, voxel features and column features are extracted from the point cloud data collected by the LiDAR, and a target point cloud feature map is generated by combining the voxel features and column features. Based on the target point cloud feature map, target detection information is generated based on a preset neural network model. This model only considers the point cloud data collected by the LiDAR, without combining camera data, and only considers the current frame, without considering multi-frame fusion, and without considering issues such as vegetation, road edges, guardrails, and potholes, resulting in low accuracy. In addition, the sparsity of the point cloud is not considered, and the computational load increases significantly when the detection range is expanded.

[0026] In view of this, the present disclosure provides a vehicle driving control method, which aims to address the full-scene characteristics required for autonomous driving perception tasks. Taking into account objects of different distances, scales, and categories, it utilizes both LiDAR and surround-view cameras to achieve multimodal complementarity. Furthermore, it outputs detection results for various types of target objects, thereby improving the full-scene perception effect and the accuracy of target detection, ultimately enhancing the safety of autonomous driving.

[0027] Furthermore, taking into full account the different sparsity of point cloud data collected by LiDAR in near and long distance scenarios, different deep learning models are adopted for point cloud data collected at near and long distances, thereby saving the computational load of target detection when point cloud data is collected at long distances.

[0028] Figure 1 This is a flowchart illustrating a vehicle driving control method according to an exemplary embodiment. See also... Figure 1 As shown, it includes the following steps: In S1, three-dimensional point cloud data and surround view image data of the environment in which the vehicle is located are collected to obtain three-dimensional point cloud frames and bird's-eye view image features, respectively.

[0029] Here, a point cloud is a collection of points in three-dimensional space, which can be obtained by scanning with a lidar system configured on the vehicle. Each point contains spatial coordinate information (X, Y, Z). The vehicle in this embodiment can be an unmanned delivery vehicle for order delivery, or a pure electric, range-extended, or fuel-powered vehicle for passenger use.

[0030] The surround-view image acquisition device can be multiple cameras installed around the vehicle to capture panoramic images of the vehicle's surroundings. These cameras can acquire images from different angles and positions, thus constructing a complete view of the vehicle's surroundings.

[0031] Among them, bird's-eye view features can be obtained by converting the image features acquired by the panoramic image acquisition device to a bird's-eye view (or top-down view). This conversion helps to unify image features from different perspectives into the same coordinate system, which facilitates subsequent feature fusion and recognition.

[0032] In this embodiment of the disclosure, multi-view images can be acquired from a panoramic image acquisition device. Features, such as edges and textures, are extracted from these images. Transforming these image features from the original viewpoint to a bird's-eye view typically involves perspective transformation or projection operations. Finally, the transformed bird's-eye image features are projected onto a predefined grid, where each grid may contain multiple image features.

[0033] In S2, the point cloud in the three-dimensional point cloud frame is rasterized to obtain raster point clouds corresponding to multiple raster grids.

[0034] A grid can divide space into multiple small, regular cells for storing and processing spatial data.

[0035] In S3, the target deep learning model is determined based on the distance between the grid and the vehicle.

[0036] In this embodiment, the distance between the grid and the vehicle varies, therefore the distance between the grid point cloud falling within the grid and the vehicle also varies. Selecting target grids with different grid point cloud features for feature extraction avoids performing feature extraction on grids that do not have the corresponding grid point cloud features, thereby reducing the computational load and improving the efficiency of point cloud feature extraction.

[0037] In this embodiment of the disclosure, the grid point cloud features may include at least one of the following: grid point cloud quantity, grid point cloud density, grid point cloud variance, and grid point cloud intensity. The grid point cloud intensity may be the reflection intensity of the laser emitted by the lidar from the point cloud. Point cloud features may include various information such as the shape, structure, size, position, pose, orientation, surface attributes, and texture of the target object.

[0038] In this embodiment of the disclosure, the point cloud density, distribution, and complexity may differ in grids at different distances. Targeted feature extraction can improve the efficiency of point cloud feature extraction. The feature extraction methods may include statistical features (such as point cloud density, height distribution, etc.), geometric features (such as shape, orientation, etc.), or deep learning features (features learned through neural networks).

[0039] In this process, different deep learning models select target graticules with varying numbers of point clouds to extract point cloud features. Based on the distance between the graticule and the vehicle, different deep learning models are chosen for feature extraction, which can more efficiently process point cloud data at different distances, improving the vehicle's perception of its surroundings and decision-making accuracy.

[0040] In this embodiment of the disclosure, a target deep learning model corresponding to different distances can be used to extract features from the raster point cloud within the target raster within that distance. This process typically involves inputting point cloud data into the model, processing it layer by layer, and finally outputting the extracted point cloud features.

[0041] For example, when vehicles are traveling on a road, closer grid cells may contain more details and complex features (such as pedestrians and vehicles), while farther grid cells may contain less detail and simpler features (such as distant buildings). Therefore, for closer grid cells, more complex feature extraction methods may be used to capture more detailed information; for farther grid cells, simpler feature extraction methods may be used to save computational resources.

[0042] In S4, the target deep learning model is used to extract features from the grid point cloud in the grid to obtain the point cloud features of the grid point cloud.

[0043] Different deep learning models may select different numbers of target rasters for feature extraction. This is because different models may have different processing capabilities and requirements for point cloud data. Some models may need more raster data to capture global information about the environment, while others may focus more on local details and require less raster data.

[0044] Deep learning models, based on neural networks (such as convolutional neural networks), can learn and automatically extract features from the point cloud of the grid, which can then be used for target object recognition. In this embodiment, different deep learning models can be trained using different samples depending on the distance between the grid and the vehicle. Therefore, multiple deep learning models can be pre-trained, each optimized for different distance ranges. These models may have different network structures, parameter settings, or training datasets to adapt to the characteristics of point cloud data at different distances. During real-time processing, the best-matching model is selected from the pre-trained models for feature extraction based on the distance between the grid and the vehicle.

[0045] For example, for close-range grids (such as those within 5 meters), which contain a large amount of detailed information and high-density point cloud data—meaning the LiDAR points are relatively dense—a deep learning model optimized for close-range applications can be selected. This model may have a more complex network structure, capable of capturing more subtle feature changes, thereby more accurately identifying obstacles such as pedestrians and vehicles.

[0046] For example, for grids at a slightly longer distance (medium distance) (such as grids more than 5 meters away but less than 50 meters away), the density of the features contained is lower compared to grids within 5 meters. A deep learning model optimized for medium distance can be selected. This model selects a more macroscopic grid to perform point cloud feature extraction compared to the model corresponding to close distance.

[0047] For example, for distant grids (such as those more than 50 meters away), since the point cloud data is relatively sparse and lacks detail, i.e., the LiDAR points are relatively sparse, a deep learning model optimized for long distances can be selected. This model may focus more on extracting global information and be able to identify key information such as distant buildings and traffic signs by processing less grid data.

[0048] In S5, the bird's-eye view features are projected onto the plurality of grids.

[0049] Bird's-eye view features are image features observed from a vertical perspective above a scene, and can be applied to autonomous driving and navigation of vehicles. For example, with the vehicle as the center, the horizontal space can be divided into H*W grids, that is, the horizontal space can be divided into H*W BEV (Bird's Eye View) spaces.

[0050] In S6, the bird's-eye view image features located in the same grid are fused with the point cloud features to obtain fused features.

[0051] The point cloud features located in the same grid under the bird's-eye view are fused with the bird's-eye view image features to obtain the target detection result for the three-dimensional point cloud frame and the classification detection result for each point cloud.

[0052] This method involves fusing information from different sources (such as image features and point cloud features) to improve the accuracy and robustness of recognition. In autonomous driving, fusion recognition is commonly used for tasks such as target detection and classification. In this embodiment, fusion can be achieved by stitching together or adding point cloud features with the bird's-eye view image features, or by dynamically adjusting the weights between different features based on attention or gating mechanisms to achieve more effective fusion. The fused features are then used for target detection and classification. For example, this can be achieved through machine learning or deep learning algorithms, such as convolutional neural networks (CNNs).

[0053] In this embodiment, the point cloud obtained by LiDAR scanning is transformed from the original three-dimensional coordinate system to a vehicle-centered coordinate system. Then, a projection algorithm maps the three-dimensional point cloud onto a two-dimensional plane to form a bird's-eye view. Finally, according to a preset grid size, the bird's-eye view is divided into multiple grids, each grid containing a certain amount of point cloud data. Therefore, a grid may correspond to a small area on the road, containing all the point cloud data within that area. Alternatively, a grid may correspond to a small area of ​​roadside railings and roadside vegetation, containing all the point cloud data within that area. Or, a grid may correspond to a small area of ​​a puddle, containing all the point cloud data within that area.

[0054] The identification process can be the classification and recognition of target objects. Classification can divide target objects into different categories or types. For example, target object classification and recognition can identify objects on the road as pedestrians, motor vehicles, non-motor vehicles, tricycles, etc. The classification and detection results for each point cloud can be the classification to which the point cloud on the road belongs. For example, does the point cloud belong to the road surface, obstacles, potholes, noise, vegetation, etc., or does it belong to pedestrians, motor vehicles, non-motor vehicles, tricycles, etc.

[0055] It should be noted that due to occlusion or missing data, for example, when a pedestrian is standing on the sidewalk, the pedestrian may be identified as the target object due to the occlusion of the trees on the sidewalk. However, when the identification is performed globally, part of the point cloud belongs to the pedestrian and part belongs to the trees. This disclosure can not only identify the target object, but also identify vegetation, curbs, railings, potholes, etc. in the scene. Therefore, the classification and detection results for each point cloud can more accurately reflect the accuracy of target detection in autonomous driving across the entire scene.

[0056] In this embodiment of the disclosure, machine learning or deep learning algorithms (such as neural networks) can be used for recognition and classification. The recognition algorithm can detect targets (such as vehicles, pedestrians, etc.) in a 3D point cloud frame and determine their positions; the classification algorithm classifies the point cloud in each grid cell to determine which category it belongs to (such as road, obstacle, etc.). For example, point cloud features are extracted from the grid, and these features can then be used to identify vehicles and pedestrians on the road. By comparing the degree of matching between the features and the target model, it can be determined whether a vehicle or pedestrian exists in the 3D point cloud frame and their positions estimated. Simultaneously, the point cloud in each grid cell can be classified to determine whether it belongs to an obstacle or other important element.

[0057] In this embodiment, point cloud features such as height, density, and reflection intensity can be extracted from each grid cell. For example, height variations can be used to identify obstacles, and reflection intensity can be used to distinguish different materials. Bird's-eye view image features of the corresponding grid cell can also be extracted. These features may include color, texture, and edges. For example, color distribution can be used to identify traffic signs, and texture information can be used to identify road surface types. A fusion algorithm (such as feature stitching or weighted averaging) is used to combine these two types of features to form the fused features of each grid cell.

[0058] In S7, the fused features are identified to obtain the target detection results of the three-dimensional point cloud frame and the classification detection results of the multiple grids.

[0059] In this embodiment of the disclosure, the fused features of each grid are input into an object detection or classification algorithm (e.g., a neural network). The algorithm identifies the target object (such as a pedestrian, vehicle, etc.) based on the fused features and provides its position in the 3D point cloud frame. For point cloud data, each point or set of points is also classified according to the fused features to determine its target object category.

[0060] For example, point cloud features within the same grid in a bird's-eye view can be fused with features from the bird's-eye view image to obtain fused features. For instance, when identifying pedestrians, combining the height information provided by the point cloud with the texture information provided by the image can more accurately distinguish pedestrians from other obstacles.

[0061] In S8, the vehicle is controlled to move based on the target detection results of the three-dimensional point cloud frame and the classification detection results of the multiple grids.

[0062] In this embodiment, the vehicle control system formulates a corresponding driving strategy based on the target detection results of the three-dimensional point cloud frame and the classification detection results of the point cloud. For example, if an obstacle or pedestrian is detected ahead, the control system will trigger a deceleration or avoidance mechanism; if the road is clear, the control system will adjust the vehicle speed to improve driving efficiency. Furthermore, the control system can also adjust the vehicle's trajectory or route based on the classification results.

[0063] The above technical solution involves acquiring 3D point cloud data and surround view image data of the vehicle's environment to obtain 3D point cloud frames and bird's-eye view image features, respectively. The point clouds in the 3D point cloud frames are rasterized to obtain raster point clouds corresponding to multiple raster cells. A target deep learning model is determined based on the distance between the raster cells and the vehicle. This target deep learning model is then used to extract features from the raster point clouds to obtain point cloud features. Bird's-eye view image features are projected onto multiple raster cells. Bird's-eye view image features and point cloud features located in the same raster cell are fused to obtain fused features. These fused features are then recognized to obtain target detection results for the 3D point cloud frames and classification detection results for multiple raster cells. Based on the target detection results for the 3D point cloud frames and the classification detection results for multiple raster cells, vehicle movement is controlled. Different deep learning models are used to extract the most effective features for objects at different distances, thereby improving the efficiency and accuracy of point cloud feature extraction. Furthermore, information from different sources (such as image features and point cloud features) is fused to improve the accuracy and robustness of recognition. This improves the perception effect across all scenarios and enhances the accuracy of target detection, thereby improving the safety of autonomous driving.

[0064] Optionally, in S3, determining the target deep learning model based on the distance between the grid and the vehicle includes: Determine whether the distance between the grid and the vehicle exceeds a preset distance threshold; If the distance does not exceed the preset distance threshold, a first deep learning model is determined as the target deep learning model; if the distance exceeds the preset distance threshold, a second deep learning model is determined as the target deep learning model; wherein, the first deep learning model is used for feature extraction of a regular convolutional neural network, and the second deep learning model is used for feature extraction of a sparse convolutional neural network.

[0065] Deep learning models, based on neural networks (such as convolutional neural networks), can learn and automatically extract features from the point cloud of the grid, which can then be used for target object recognition. In this embodiment, different deep learning models can be trained using different samples depending on the distance between the grid and the vehicle. Therefore, multiple deep learning models can be pre-trained, each optimized for different distance ranges. These models may have different network structures, parameter settings, or training datasets to adapt to the characteristics of point cloud data at different distances. During real-time processing, the best-matching model is selected from the pre-trained models for feature extraction based on the distance between the grid and the vehicle.

[0066] For example, for close-range grids (such as those within 5 meters), which contain a large amount of detailed information and high-density point cloud data—meaning the LiDAR points are relatively dense—a deep learning model optimized for close-range applications can be selected. This model may have a more complex network structure, capable of capturing more subtle feature changes, thereby more accurately identifying obstacles such as pedestrians and vehicles.

[0067] For example, for grids at a slightly longer distance (medium distance) (such as grids more than 5 meters away but less than 50 meters away), the density of the features contained is lower compared to grids within 5 meters. A deep learning model optimized for medium distance can be selected. This model selects a more macroscopic grid to perform point cloud feature extraction compared to the model corresponding to close distance.

[0068] For example, for distant grids (such as those more than 50 meters away), since the point cloud data is relatively sparse and lacks detail, i.e., the LiDAR points are relatively sparse, a deep learning model optimized for long distances can be selected. This model may focus more on extracting global information and be able to identify key information such as distant buildings and traffic signs by processing less grid data.

[0069] In this embodiment of the disclosure, the vehicle divides the environment into multiple grids based on its current location and perceived point cloud data. Each grid represents a space within a certain range and contains point cloud data within that space. Next, the vehicle calculates the distance to each grid. For example, a preset distance threshold is set, such as 10 meters. The vehicle checks whether the distance to each grid exceeds this threshold. This threshold is set based on the vehicle's perception capabilities, safety requirements, and mission requirements.

[0070] If the distance to the grid is less than or equal to the preset distance threshold, a first deep learning model corresponding to the distance less than or equal to the preset distance threshold is selected from a plurality of pre-trained deep learning models as the target deep learning model. The first deep learning model selects all grids whose distance is less than or equal to the preset distance threshold as the target grids to extract point cloud features.

[0071] In this embodiment of the disclosure, for grids with a distance of 10 meters or less (referred to as "near-range grids"), the vehicle selects a pre-trained deep learning model corresponding to the near range, i.e., a first deep learning model (e.g., a regular convolutional neural network). This model may be specifically designed for processing near-range, high-density point cloud data, and thus can extract features within these grids more accurately.

[0072] The first deep learning model selects all grid cells with a distance of 10 meters or less as target grid cells and extracts features from the point cloud data within these grid cells. Since the point cloud data within nearby grid cells is usually richer, the model can capture more detailed information, which is very helpful for tasks such as obstacle detection and road understanding.

[0073] If the distance to the grid is greater than the preset distance threshold, a second deep learning model corresponding to the distance greater than the preset distance threshold is selected from the pre-trained deep learning models as the target deep learning model. The second deep learning model selects the grid with the distance greater than the preset distance threshold and the grid point cloud as the target grid to extract point cloud features.

[0074] In this embodiment of the disclosure, for grids with a distance greater than 10 meters (referred to as "long-distance grids"), the vehicle selects a deep learning model corresponding to the long distance, i.e., a second deep learning model (e.g., a sparse convolutional neural network). This model may focus more on global information extraction and sparse point cloud processing, thus enabling it to effectively extract features within the grid even at long distances.

[0075] The second deep learning model does not select all distant graticles as target graticles, but only those graticles that actually contain point cloud data. This is because the sparsity of point cloud data increases with distance, and many distant graticles may not contain valid point cloud data. By selecting only graticles with point cloud data as target graticles, unnecessary computation can be reduced and the efficiency of feature extraction can be improved.

[0076] In the second deep learning model, when extracting features from point cloud data within a distant target grid, it may focus more on extracting features that are important for long-distance perception, such as the outline and position of objects. These features can help vehicles perceive key environmental information at long distances, such as distant traffic signs and road directions.

[0077] The above technical solution selects an appropriate deep learning model for feature extraction based on the distance between the grid and the vehicle. This allows the vehicle to effectively perceive the environment and make decisions at different distances, thereby improving the accuracy and safety of its autonomous navigation and delivery.

[0078] Optionally, there are multiple 3D point cloud frames, see [link to relevant documentation]. Figure 2 As shown, in S2, the step of rasterizing the point cloud in the 3D point cloud frame to obtain multiple raster point clouds corresponding to each raster includes: In S21, the rotation and translation matrix between the multiple three-dimensional point cloud frames is determined based on the initial transformation relationship of the multiple three-dimensional point cloud frames.

[0079] Among them, multiple 3D point cloud frames can be multiple single images captured in time sequence by sensors (such as LiDAR), containing environmental information corresponding to consecutive time points.

[0080] In this embodiment of the disclosure, due to the motion of the sensor itself or changes in the external environment, the point cloud data in each three-dimensional point cloud frame may not be in the same coordinate system. Therefore, by calculating the initial transformation relationship between the three-dimensional point cloud frames, the rotation and translation matrix between them is determined. This matrix contains rotation angles and translation vectors, which are used to align the point cloud data in different three-dimensional point cloud frames to the same coordinate system.

[0081] For example, multiple 3D point cloud frames are continuously captured using LiDAR. Due to vehicle movement and bumps, the point cloud data in each 3D point cloud frame may have slight deviations. To integrate this data for subsequent processing, the vehicle first calculates the rotation and translation matrices between adjacent 3D point cloud frames. These matrices describe the vehicle's attitude changes at different times and the resulting transformations in the point cloud data.

[0082] In S22, the point clouds in the three-dimensional space of multiple three-dimensional point cloud frames are aligned according to the rotation and translation matrix.

[0083] In this embodiment, a rotation and translation matrix can be used to align point cloud data in multiple 3D point cloud frames. By applying rotation and translation transformations to each point, they can be transformed into a unified coordinate system, thereby eliminating deviations caused by sensor motion or environmental changes. For example, during vehicle operation, the calculated rotation and translation matrix can be used to align point cloud data in continuously captured 3D point cloud frames. After alignment, the originally biased point cloud data is corrected, enabling them to accurately reflect the 3D structure of the vehicle's surrounding environment.

[0084] In S23, the aligned point cloud is rasterized to obtain the raster point cloud corresponding to the multiple raster grids.

[0085] In this embodiment, the aligned point cloud data undergoes coordinate transformation and rasterization. First, based on the requirements of the bird's-eye view, a suitable projection plane and projection method are determined to map points in three-dimensional space onto a two-dimensional plane. Then, the two-dimensional plane is divided into multiple grids, and points are assigned to the corresponding grids according to their positions.

[0086] In the scenario of the autonomous vehicle disclosed herein, in order to better understand and process information about the surrounding environment, the vehicle converts the aligned point cloud data into a grid. In this way, the point cloud data, which was originally scattered in three-dimensional space, is organized into a two-dimensional grid, with each grid containing point cloud information within a specific area.

[0087] Optionally, see Figure 3 As shown, in S8, controlling the vehicle's movement based on the target detection results of the three-dimensional point cloud frame and the classification detection results of the multiple grids includes: In S81, the preceding frame is determined from the plurality of three-dimensional point cloud frames.

[0088] In this context, the preceding frame refers to a 3D point cloud frame captured before the current processing frame, typically used for temporal alignment or comparison with the target frame. The preceding frame can be a single 3D point cloud frame or multiple 3D point cloud frames.

[0089] In this embodiment of the disclosure, three 3D point cloud frames can be arbitrarily selected from the first four acquired 3D point cloud frames as preceding frames. For example, the frames in the buffer are sorted according to the timestamp corresponding to each 3D point cloud frame (indicating the time it was captured) to ensure that they are processed in the correct order. Therefore, three 3D point cloud frames can be arbitrarily selected from the first four acquired 3D point cloud frames as preceding frames.

[0090] In S82, the target detection results based on the three-dimensional point cloud frame and the classification detection results of the multiple grids are converted to polar coordinates.

[0091] In this context, the polar coordinate system is a coordinate system in which the position of each point is determined by the distance (radius) from a fixed point (polar point) to that point and the angle from the positive direction to that point. It is understandable that the target detection results (such as pedestrian positions) and the point cloud classification detection results (such as road edges) in the 3D point cloud frame are transformed from the Cartesian coordinate system to the vehicle-centered polar coordinate system in order to better reflect the environmental information around the vehicle.

[0092] In this embodiment of the disclosure, the three-dimensional point cloud frames acquired by the lidar are all in a Cartesian coordinate system, which is a two-dimensional or three-dimensional spatial representation based on rectangular coordinates. The position of each point is determined by its coordinate values ​​on the X-axis and Y-axis (or Z-axis). In the polar coordinate system, the position of a point is determined by the distance (radius) from the pole (usually the origin or a specific point) to that point and the angle from the positive direction (such as the positive X-axis) to that point.

[0093] For autonomous vehicles, a polar coordinate system centered on the vehicle can use the vehicle's position as the pole, while objects in the surrounding environment are located based on their distance and angle relative to the vehicle. This allows the vehicle to more easily perceive and understand the distribution and dynamic changes of objects in its surrounding environment.

[0094] During the conversion process, target detection results (such as pedestrian positions) in the 3D point cloud frame can be extracted from the Cartesian coordinate system. This typically involves processing the image using a target detection algorithm to identify target objects such as pedestrians and vehicles, and obtaining their 3D spatial coordinates in the 3D point cloud frame. Then, based on the relative positional relationship between the LiDAR and the vehicle, these 3D spatial coordinates are converted into Cartesian coordinates in the real world.

[0095] Next, these Cartesian coordinates are converted to polar coordinates. This typically involves calculating the distance (i.e., radius) of each target object to the vehicle and their angles relative to the direction directly in front of the vehicle. The distance can be calculated using the Euclidean distance between the target object's X and Y (or Z) coordinates in the Cartesian coordinate system and the vehicle's position. The angle can be calculated using the arctangent function or a similar method, based on the relationship between the target object's coordinates in the Cartesian coordinate system and the direction directly in front of the vehicle.

[0096] Similarly, a similar coordinate transformation process is needed for point cloud classification and detection results (such as road edges). Point cloud data is typically acquired by sensors such as LiDAR and represented directly as a set of points in three-dimensional space. First, the point cloud data is classified to identify different objects or regions (such as roads, buildings, obstacles, etc.). Then, based on the coordinates of each point in the Cartesian coordinate system, their distance and angle relative to the vehicle are calculated, thus converting it to a polar coordinate system representation.

[0097] By transforming object detection results and point cloud classification results from a Cartesian coordinate system to a vehicle-centric polar coordinate system, we can better reflect the environmental information surrounding the vehicle. This transformation enables the vehicle to more intuitively perceive and understand the distribution and dynamic changes of objects around it, thereby making more accurate driving decisions and obstacle avoidance maneuvers. At the same time, the polar coordinate system representation simplifies subsequent data processing and fusion, improving the system's processing speed and responsiveness.

[0098] In S83, the target detection results of the three-dimensional point cloud frame in the polar coordinate system and the classification detection results of the multiple grids are projected onto the preceding frame to temporally align the target detection results of the three-dimensional point cloud frame and the classification detection results of the multiple grids with the preceding frame.

[0099] Temporal alignment involves aligning multiple 3D point cloud frames to the same timeline or reference frame for comparison or fusion. This ensures that target detection results and point cloud classification detection results from different time points remain consistent over time.

[0100] In this embodiment, the correspondence between each target or point cloud data point in the target frame and its corresponding target or point cloud data point in the preceding frame can first be determined. This is typically based on the unique identifier of the target or the spatial location of the point cloud data point. Next, the expected position of the target or point cloud data point in the preceding frame can be calculated based on the time difference between the target frame and the preceding frame and the vehicle's motion information (such as speed, direction, etc.). This involves appropriately transforming the polar coordinates to reflect the vehicle's motion and the passage of time. Finally, the target detection results and point cloud classification detection results in the target frame can be projected onto the corresponding positions in the preceding frame. This can be achieved through interpolation, nearest neighbor search, or other appropriate algorithms to ensure the accuracy and consistency of the projection results.

[0101] In S84, multiple time-aligned target detection results and classification detection results are time-fused to obtain a fused detection result and an occupation detection result indicating whether the grid is occupied.

[0102] Temporal fusion involves merging multiple temporally aligned 3D point cloud frames to obtain more comprehensive and accurate information. Occupation detection results are obtained by analyzing point cloud data or other sensor data to determine whether each grid cell is occupied by an object (i.e., whether there is an obstacle).

[0103] In this embodiment, multiple time-aligned 3D point cloud frames are fused to combine target detection results and point cloud classification detection results from different frames. Simultaneously, by analyzing the point cloud data, it is determined whether each grid cell is occupied by an obstacle, as some 3D point cloud frames may have unoccupied grid cells while others may have occupied ones. This fused information provides crucial information for subsequent driving control.

[0104] In S85, the vehicle is controlled to move based on the fusion detection result and the occupancy detection result.

[0105] In this embodiment of the disclosure, based on the fused detection results, information such as road conditions and obstacle locations can be determined, thereby making appropriate driving decisions. For example, the fused detection results may indicate that a pedestrian is crossing the road ahead, and the occupancy detection results may show that the relevant grid is occupied. Based on this information, the system decides to slow down or stop to avoid the pedestrian, ensuring the safety and stability of vehicle driving.

[0106] The target detection result is used to characterize the target object present in the 3D point cloud frame, and the classification detection result is used to characterize the target object to which the raster belongs.

[0107] See Figure 4 As shown, firstly, in S501, the point cloud in three-dimensional space can be acquired by the lidar configured on the vehicle. In S502, the acquired point cloud is rasterized. Then, for the near-range point cloud, S503 ordinary convolutional neural network is executed to extract the near-range point cloud features. For the far-range point cloud, S504 sparse convolutional neural network is executed to extract the far-range point cloud features. Then, the extracted point cloud features are converted into point cloud features under the BEV view.

[0108] Furthermore, in S506, images acquired by the image acquisition device of the vehicle's surround-view configuration can be obtained. In S507, image features are extracted using a convolutional neural network. Then, in S508, the features are projected according to camera parameters. Finally, in S509, the image features from the image acquisition device are converted into bird's-eye view features under the BEV view. Further, in S510, point cloud features and bird's-eye view features within the same grid are fused. Then, in S511, features corresponding to multiple 3D point cloud frames are fused temporally to obtain the target detection results for the 3D point cloud frames in S512 and the occupancy detection results for each grid in S513.

[0109] The above technical solution fuses point cloud features within the same grid with aerial image features for identification, resulting in target detection results for a 3D point cloud frame and classification detection results for each point cloud element. This fully utilizes information from both image and point cloud data, improving the accuracy and reliability of the identification process.

[0110] Optionally, see Figure 5 As shown, in S1, surround view image data of the environment in which the vehicle is located is acquired to obtain bird's-eye view image features, including: In S11, based on the intrinsic parameter matrix of the image acquisition device for the panoramic image data, the image features of the panoramic image data in image coordinates are converted into three-dimensional points in camera coordinates.

[0111] The intrinsic parameter matrix describes the internal properties of the image acquisition device (such as a camera), including focal length and principal point coordinates, and is used to convert pixel coordinates into three-dimensional points in camera coordinates. Using the intrinsic parameter matrix, image features are transformed from pixel coordinates to camera coordinates through a formula transformation. This transformation takes into account the camera's internal parameters such as focal length and principal point coordinates.

[0112] Pixel coordinates represent the position coordinates of each pixel in the image, usually denoted as (u, v), where u is the column number (horizontal coordinate) and v is the row number (vertical coordinate). Camera coordinates are three-dimensional coordinates in a coordinate system with the camera optical center as the origin and the camera optical axis as the Z-axis.

[0113] In S12, based on the extrinsic parameter matrix of the image acquisition device, the three-dimensional point is rotated and translated to convert the three-dimensional point in the camera coordinates into a three-dimensional point in the world coordinates.

[0114] The extrinsic parameter matrix describes the position and orientation of the image acquisition device (such as a camera) in the world coordinate system, and is used to convert 3D points in camera coordinates to 3D points in world coordinates. World coordinates are used to describe the position and orientation of objects in the real world.

[0115] In this embodiment, an extrinsic parameter matrix is ​​used to rotate and translate a 3D point in camera coordinates, transforming it to world coordinates. This transformation reflects the camera's position and orientation in the world.

[0116] In S13, based on the position information of the image acquisition device relative to the vehicle, the three-dimensional points in the world coordinates are mapped to the two-dimensional image plane to obtain the bird's-eye view image features.

[0117] Among them, the location information is used to describe the orientation and positional relationship of an object relative to a vehicle.

[0118] In this embodiment of the disclosure, the surround-view image acquisition device typically includes multiple cameras mounted at different locations on the vehicle to acquire an all-around view of the vehicle's surroundings. Each camera has its unique mounting location and orientation, and the precise position and orientation of each camera relative to the vehicle can be determined through calibration.

[0119] Furthermore, perspective projection is used to transform 3D points in camera coordinates to image coordinates. This is done based on internal parameters such as the camera's focal length, principal point coordinates, and the direction of light projection. Perspective projection converts 3D points in camera coordinates into 2D points in image coordinates. Finally, the boundary range of each grid cell in the image coordinate system is determined, and image coordinate points within this range are assigned to the corresponding grid cells.

[0120] Optionally, see Figure 6 As shown, in S13, the step of mapping three-dimensional points in world coordinates to a two-dimensional image plane based on the position information of the image acquisition device relative to the vehicle to obtain the bird's-eye view image features includes: In S131, the acquisition direction of the image acquisition device is determined based on the position of the image acquisition device relative to the vehicle.

[0121] The orientation of the image acquisition device can be the position and direction of the image acquisition equipment (such as a camera) installed around the vehicle relative to the center of the vehicle or a fixed reference point. The acquisition direction can be a vector or angle pointing from the image acquisition device toward its shooting direction, reflecting the field of view seen by the camera.

[0122] In this embodiment of the disclosure, the acquisition direction of the image acquisition device can be calibrated by looking around the vehicle and determining its position relative to the vehicle. During the calibration process, the direction and position of each camera relative to the vehicle coordinate system are recorded.

[0123] In S132, using the position of the vehicle as a reference point, and based on the acquisition direction and a preset perspective transformation matrix, the three-dimensional point in the world coordinates is mapped onto the two-dimensional image plane to obtain the bird's-eye view image features.

[0124] In this embodiment, using the vehicle's position as a reference point, and combining the viewing direction of the image acquisition device with a preset perspective transformation matrix, three-dimensional points in world coordinates can be transformed onto a two-dimensional image plane. The perspective transformation matrix considers the position and orientation of objects in three-dimensional space, as well as their projection onto the two-dimensional image plane, including changes in object size due to perspective effects. The points mapped onto the two-dimensional image plane are further projected onto a grid. This is typically achieved by calculating the relationship between the coordinates of the points and the grid boundaries to determine the grid to which each point belongs.

[0125] In this embodiment of the disclosure, the factors for setting the preset distance threshold include the vehicle's perception capabilities, safety requirements, or mission requirements.

[0126] This disclosure also provides a vehicle, including: A three-dimensional point cloud data acquisition device is used to acquire three-dimensional point cloud data of the environment in which the vehicle is located; A surround view image data acquisition device is used to acquire surround view image data of the environment in which the vehicle is located; a memory having a computer program stored thereon; A processor is configured to execute the computer program in the memory to implement the steps of any of the methods described in the foregoing embodiments.

[0127] This disclosure also provides a computing platform, including: A three-dimensional point cloud data receiving device is used to receive three-dimensional point cloud data of the environment in which the vehicle is located; A surround view image data receiving device, used to receive surround view image data of the environment in which the vehicle is located; a memory having a computer program stored thereon; A processor for executing the computer program in the memory to implement the steps of any of the methods described in the foregoing embodiments.

[0128] Figure 7 This is a block diagram illustrating an electronic device 700 according to an exemplary embodiment. Figure 7 As shown, the electronic device 700 may include a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.

[0129] The processor 701 controls the overall operation of the electronic device 700 to complete all or part of the steps in the aforementioned vehicle driving control method. The memory 702 stores various types of data to support the operation of the electronic device 700. This data may include, for example, instructions for any application or method operating on the electronic device 700, and application-related data. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 703 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 702 or transmitted via communication component 705. The audio component also includes at least one speaker for outputting audio signals. I / O interface 704 provides an interface between processor 701 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IoT, eMTC, or other 5G technologies, or combinations thereof, is not limited here. Therefore, the corresponding communication component 705 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0130] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described vehicle driving control method.

[0131] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the vehicle driving control method described above. For example, the computer-readable storage medium may be the memory 702 including program instructions, which may be executed by the processor 701 of the electronic device 700 to complete the vehicle driving control method described above.

[0132] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0133] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0134] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A method for controlling the movement of a vehicle, characterized in that, include: The environment in which the vehicle is located is subjected to the acquisition of three-dimensional point cloud data and surround view image data to obtain three-dimensional point cloud frames and bird's-eye view image features, respectively. The point cloud in the three-dimensional point cloud frame is rasterized to obtain raster point clouds corresponding to multiple raster grids; The target deep learning model is determined based on the distance between the grid and the vehicle; The target deep learning model is used to extract features from the grid point cloud in the grid to obtain the point cloud features of the grid point cloud; The bird's-eye view features are projected onto a plurality of the grids; The bird's-eye view features located in the same grid are fused with the point cloud features to obtain fused features; The fused features are identified to obtain the target detection results of the three-dimensional point cloud frame and the classification detection results of the multiple grids; The vehicle's movement is controlled based on the target detection results of the three-dimensional point cloud frame and the classification detection results of the multiple grids.

2. The method according to claim 1, characterized in that, Determining the target deep learning model based on the distance between the grid and the vehicle includes: Determine whether the distance between the grid and the vehicle exceeds a preset distance threshold; If the distance does not exceed the preset distance threshold, a first deep learning model is determined as the target deep learning model; if the distance exceeds the preset distance threshold, a second deep learning model is determined as the target deep learning model; wherein, the first deep learning model is used for feature extraction of a regular convolutional neural network, and the second deep learning model is used for feature extraction of a sparse convolutional neural network.

3. The method according to claim 1, characterized in that, There are multiple 3D point cloud frames. The step of rasterizing the point clouds in the 3D point cloud frames to obtain multiple raster point clouds corresponding to different raster grids includes: Based on the initial transformation relationship of multiple 3D point cloud frames, determine the rotation and translation matrix between the multiple 3D point cloud frames; The point clouds in the plurality of three-dimensional point cloud frames are aligned according to the rotation and translation matrix; The aligned point cloud is rasterized to obtain the raster point cloud corresponding to the multiple raster grids.

4. The method according to claim 3, characterized in that, The step of controlling the vehicle's movement based on the target detection results of the 3D point cloud frame and the classification detection results of the multiple grids includes: Determine the preceding frame from the plurality of three-dimensional point cloud frames; The target detection results based on the three-dimensional point cloud frame and the classification detection results of the multiple grids are converted to polar coordinates. The target detection results of the three-dimensional point cloud frame in the polar coordinate system and the classification detection results of the multiple grids are projected onto the preceding frame to temporally align the target detection results of the three-dimensional point cloud frame and the classification detection results of the multiple grids with the preceding frame. Multiple time-aligned target detection results and classification detection results are temporally fused to obtain a fused detection result and an occupancy detection result indicating whether the grid is occupied. The vehicle's movement is controlled based on the fusion detection results and the occupancy detection results.

5. The method according to any one of claims 1-4, characterized in that, The target detection result is used to characterize the target objects present in the three-dimensional point cloud frame, and the classification detection result is used to characterize the target object to which the raster belongs.

6. The method according to any one of claims 1-4, characterized in that, The environment in which the vehicle is located is acquired through surround view image data to obtain bird's-eye view image features, including: Based on the intrinsic parameter matrix of the image acquisition device using the panoramic image data, the image features of the panoramic image data in image coordinates are converted into three-dimensional points in camera coordinates; Based on the extrinsic parameter matrix of the image acquisition device, the three-dimensional points are rotated and translated to convert the three-dimensional points in the camera coordinates into three-dimensional points in the world coordinates. Based on the position information of the image acquisition device relative to the vehicle, the three-dimensional points in the world coordinates are mapped to the two-dimensional image plane to obtain the bird's-eye view image features.

7. The method according to claim 6, characterized in that, The step of mapping three-dimensional points in world coordinates to a two-dimensional image plane based on the position information of the image acquisition device relative to the vehicle to obtain the bird's-eye view image features includes: The acquisition direction of the image acquisition device is determined based on its position relative to the vehicle. Using the vehicle's position as a reference point, and based on the acquisition direction and a preset perspective transformation matrix, the three-dimensional points in the world coordinates are mapped onto the two-dimensional image plane to obtain the bird's-eye view image features.

8. The method according to claim 2, characterized in that, The factors for setting the preset distance threshold include the vehicle's perception capabilities, safety requirements, or mission requirements.

9. A vehicle, characterized in that, include: A three-dimensional point cloud data acquisition device is used to acquire three-dimensional point cloud data of the environment in which the vehicle is located; A surround view image data acquisition device is used to acquire surround view image data of the environment in which the vehicle is located; A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-8.

10. A computing platform, characterized in that, include: A three-dimensional point cloud data receiving device is used to receive three-dimensional point cloud data of the environment in which the vehicle is located; A surround view image data receiving device, used to receive surround view image data of the environment in which the vehicle is located; A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-8.

Images