A road surface preview method, system, device and medium based on point cloud input

Abstract

The present application discloses a road surface preview method, system, device and medium based on point cloud input, which relates to the field of intelligent vehicle driving control technology. The method includes: acquiring real-time point cloud data of a target area in a sensor coordinate system; segmenting the real-time point cloud data in the sensor coordinate system to obtain a ground area point cloud and a non-ground area point cloud; updating map grid information based on the ground area point cloud and the non-ground area point cloud to obtain a local grid map; based on odometer information, fusing the local grid map with the global grid map at the previous moment to generate a global grid map at the current moment; predicting the wheel trajectory of the intelligent vehicle based on the vehicle trajectory and geometric parameters of the intelligent vehicle; mapping the wheel trajectory to the global grid map at the current moment, and calculating the elevation information of the wheel trajectory. The present application improves the accuracy and efficiency of feature reconstruction of terrains of different scales in an environment.

Description

Technical Field

[0001] The present application relates to the field of intelligent vehicle driving control technology, and in particular to a road surface preview method, system, device and medium based on point cloud input. Background Art

[0002] The road surface is the core element of a vehicle's interaction during driving, and its characteristics directly impact vehicle safety and comfort. Smart vehicles' onboard sensors (including but not limited to lidar and binocular cameras) analyze road conditions ahead, providing feedforward input for path planning, motion control, and body control, effectively improving vehicle ride comfort and safety.

[0003] Currently, there are road surface information preview methods based on monocular cameras, binocular cameras, and lidar. However, due to the lack of depth information, monocular cameras cannot directly provide accurate geometric features and distance information. Deep learning-based methods can only detect categories included in the training set, so the algorithm has poor generalization. Road surface information preview methods based on binocular cameras are susceptible to noise interference in complex scenes, and the depth estimation accuracy is insufficient. In addition, the camera itself has weak anti-interference capabilities, making it difficult to achieve accurate modeling and detection. Road surface information preview methods based on lidar are difficult to meet the accuracy requirements. Summary of the Invention

[0004] The purpose of this application is to provide a road preview method, system, device and medium based on point cloud input, which can improve the accuracy and efficiency of feature reconstruction of terrain of different scales in the environment.

[0005] To achieve the above objectives, this application provides the following solutions.

[0006] In a first aspect, the present application provides a road surface preview method based on point cloud input, comprising the following steps.

[0007] Obtain real-time point cloud data of the target area in the sensor coordinate system.

[0008] The real-time point cloud data in the sensor coordinate system is segmented to obtain the ground area point cloud and the non-ground area point cloud.

[0009] Based on the ground area point cloud and the non-ground area point cloud, the grid information of the local map model of the target area of ​​interest is updated respectively to obtain a local grid map; the grid information includes the elevation information, confidence level and grid type of the grid; the grid type includes the ground area and the non-ground area.

[0010] Based on the odometer information, the local grid map is fused with the global grid map at the previous moment to generate the global grid map at the current moment.

[0011] The wheel trajectory of the intelligent vehicle is predicted based on the vehicle's own trajectory and geometric parameters.

[0012] The wheel trajectory is mapped to the global grid map at the current moment, and the elevation information of the wheel trajectory is calculated.

[0013] Optionally, the real-time point cloud data in the sensor coordinate system is segmented to obtain the ground area point cloud and the non-ground area point cloud, specifically including: preprocessing the real-time point cloud data in the sensor coordinate system to obtain the point cloud data in the area of ​​interest; using the point cloud segmentation algorithm to segment the point cloud data in the area of ​​interest to obtain the ground area point cloud and the non-ground area point cloud.

[0014] Optionally, the real-time point cloud data in the sensor coordinate system is preprocessed to obtain point cloud data in the area of ​​interest, specifically including: performing posture transformation on the real-time point cloud data in the sensor coordinate system to obtain point cloud data in the map coordinate system; extracting the point cloud data in the map coordinate system according to a preset local grid map size to obtain point cloud data in the area of ​​interest.

[0015] Optionally, based on the ground area point cloud and the non-ground area point cloud, the raster information of the local map model of the target area of ​​interest is updated respectively to obtain a local raster map, specifically including: matching the ground area point cloud with the raster in the local map model based on the position coordinates of the ground area point cloud; for each grid, updating the raster information of the grid based on the position coordinates of the ground area point cloud matched with the grid to obtain updated ground raster information; sorting the non-ground area point clouds according to their heights, traversing the non-ground area point clouds in ascending order, updating the raster information in the digital elevation map to obtain updated non-ground raster information; all updated ground raster information and updated non-ground raster information constitute an updated raster map; determining the resolution of the map area corresponding to the grid according to the confidence of the grid in the updated raster map to achieve resolution adaptive optimization to obtain a local raster map.

[0016] Optionally, based on the odometer information, the local grid map is fused with the global grid map at the previous moment to generate the global grid map at the current moment, specifically including: based on the odometer information, converting the global grid map at the previous moment to the map coordinate system at the current moment to obtain the global grid map at the previous moment in the map coordinate system; based on the grid type of the grid, fusing the local grid map with the global grid map at the previous moment in the map coordinate system to generate the global grid map at the current moment.

[0017] Optionally, the wheel driving trajectory of the intelligent vehicle is predicted based on the ego vehicle trajectory and geometric parameters of the intelligent vehicle, specifically including: constructing a kinematic model of the intelligent vehicle, predicting the vehicle driving trajectory based on the vehicle's real-time speed information and front wheel turning angle information to obtain the ego vehicle trajectory; predicting the wheel driving trajectory of the intelligent vehicle based on the geometric parameters of the intelligent vehicle and the ego vehicle trajectory.

[0018] Optionally, the wheel trajectory is mapped to the global grid map at the current moment, and the elevation information of the wheel trajectory is calculated, specifically including: determining the target grid according to the tire width of the intelligent vehicle and the resolution of the global grid map at the current moment; the target grid is the grid occupied by the predicted wheel trajectory at the same moment; and performing weighted summation on the elevation information of all the target grids to obtain the elevation information of the wheel trajectory.

[0019] In a second aspect, the present application provides a road surface preview system based on point cloud input, comprising the following modules.

[0020] The real-time point cloud data acquisition module is used to obtain real-time point cloud data of the target area in the sensor coordinate system.

[0021] The point cloud data segmentation module is used to segment the real-time point cloud data in the sensor coordinate system to obtain the ground area point cloud and the non-ground area point cloud.

[0022] The grid information update module is used to update the grid information of the local map model of the target area of ​​interest based on the ground area point cloud and the non-ground area point cloud, respectively, to obtain a local grid map; the grid information includes the elevation information, confidence level and grid type of the grid; the grid type includes ground area and non-ground area.

[0023] The global grid map generation module is used to fuse the local grid map with the global grid map at the previous moment based on the odometer information to generate the global grid map at the current moment.

[0024] The wheel driving trajectory prediction module is used to predict the wheel driving trajectory of the intelligent vehicle based on the vehicle trajectory and geometric parameters of the intelligent vehicle.

[0025] The wheel track elevation information calculation module is used to map the wheel track to the global grid map at the current moment and calculate the wheel track elevation information.

[0026] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned road surface preview method based on point cloud input.

[0027] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned road surface preview method based on point cloud input.

[0028] According to the specific embodiments provided in this application, this application discloses the following technical effects: This application provides a road preview method, system, equipment and medium based on point cloud input, which realizes the distinction between ground areas and non-ground areas (non-ground areas are obstacles) in the environment through point cloud segmentation, updates the grid information based on the ground area point cloud and the non-ground area point cloud to obtain a local grid map, and merges it with the global grid map at the previous moment to obtain the global grid map at the current moment, realizes digital reconstruction of the vehicle's surrounding environment through modeling and analysis of the environmental point cloud, predicts the elevation information of the wheel trajectory, improves the feature reconstruction accuracy and efficiency of terrain of different scales in the environment, and provides accurate road information for autonomous driving vehicles and advanced assisted driving functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 This is a diagram of the application environment of a road preview method based on point cloud input in one embodiment of the present application.

[0031] Figure 2 A schematic flow chart of a road surface preview method based on point cloud input provided in one embodiment of the present application.

[0032] Figure 3 A schematic diagram of the specific process of a road surface preview method based on point cloud input provided in one embodiment of the present application.

[0033] Figure 4 for Figure 2 Detailed flowchart of steps 202 to 206.

[0034] Figure 5 A schematic diagram of a coordinate system definition provided in an embodiment of the present application.

[0035] Figure 6 A schematic diagram of a vehicle kinematic model provided in one embodiment of the present application.

[0036] Figure 7 A schematic diagram of the functional modules of a road preview system based on point cloud input provided in one embodiment of the present application.

[0037] Figure 8 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0039] In typical driving scenarios, if a vehicle faces small obstacles (such as piers or cones) ahead, the onboard sensors can identify these obstacles through high-precision point cloud data, providing accurate environmental information to the path planning algorithm, helping the vehicle to execute effective evasive maneuvers or prompting the driver to take appropriate measures to prevent a collision. Alternatively, if the road ahead has significant unevenness (such as speed bumps or localized bumps), the point cloud data can be analyzed to construct a digital reconstruction model of the vehicle's driving scenario. Combined with the vehicle's motion model, the wheel trajectory can be predicted, allowing for a preview analysis of the vehicle's trajectory elevation. This provides feedforward input to intelligent vehicle systems, particularly the active suspension system, significantly improving the vehicle's ride smoothness.

[0040] Monocular cameras are widely used in road surface information recognition tasks due to their high resolution and ease of integration. A dynamic chassis preview control method based on road surface state recognition, using camera image data combined with advanced deep learning algorithms, can segment the drivable area of ​​the road and effectively identify typical road features (such as potholes and cracks). However, due to the lack of depth information, monocular cameras cannot directly provide accurate geometric features and distance information. Deep learning-based methods can only detect categories included in the training set, resulting in poor generalization.

[0041] Road surface preview methods based on binocular cameras use stereo matching algorithms to generate depth maps, thereby acquiring three-dimensional information about the environment. However, the quality of depth maps is highly dependent on the performance of the stereo matching algorithm, and existing algorithms are susceptible to noise in complex scenes, resulting in insufficient depth estimation accuracy. Furthermore, cameras themselves have weak anti-interference capabilities and are easily affected by factors such as lighting changes, shadows, and inclement weather, resulting in degraded image quality. Furthermore, existing methods often struggle to accurately model and detect road surface defects that are small in size and have less pronounced geometric features. This, to a certain extent, limits the effectiveness of cameras in complex road surface scenarios.

[0042] LiDAR-based road surface information preview methods construct a map of the area in front of the vehicle by processing LiDAR point cloud data. However, during the map construction process, these methods only calculate the elevation information of the corresponding location and fail to consider the map's attribute information, such as distinguishing between ground and non-ground areas. If the elevation information of non-ground areas is used as the feedforward input of the active suspension, it may cause the active suspension actuator to respond incorrectly. In addition, the map resolution has a direct impact on the accuracy of terrain reconstruction: larger resolutions make it difficult to accurately depict terrain changes, while smaller resolutions, while improving accuracy, will significantly increase the amount of calculation.

[0043] To address the aforementioned issues and achieve accurate preview of the road ahead, this application proposes a point cloud-based road preview method. This method constructs a digital elevation map of the vehicle's driving environment using environmental point cloud data captured by sensors, providing precise environmental information support for advanced driver assistance features in intelligent vehicles. First, the collected real-time point cloud data is processed and segmented into ground and non-ground area point clouds. This allows for the extraction of prior information about the drivable area and ensures accurate recognition of road features. The spatial distribution and attribute information of the point cloud (ground and non-ground area point clouds) are then used to dynamically update the local digital elevation map model. A quadtree-based optimization algorithm is introduced to adaptively optimize the grid resolution based on the discreteness of the local grid point cloud, enabling accurate recognition of road features at different scales. Second, the updated local digital elevation map is fused with historical maps using the odometer's position information to generate a global elevation map, providing the vehicle with comprehensive environmental awareness. Finally, based on the intelligent vehicle's current speed and front wheel angle information, the vehicle's future wheel trajectory is predicted and mapped onto a global grid map, thereby achieving accurate preview of the road surface roughness characteristics ahead and providing high-quality data input for vehicle smoothness control and path planning.

[0044] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0045] The road preview method based on point cloud input provided in the embodiment of the present application can be applied to Figure 1In the application environment shown, the terminal 102 communicates with the server 104 via a network. The data storage system can store data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send real-time point cloud data to the server 104. After receiving the real-time point cloud data, the server 104 segments the real-time point cloud data in the sensor coordinate system to obtain a ground area point cloud and a non-ground area point cloud. Based on the ground area point cloud and the non-ground area point cloud, the map grid information is updated to obtain a local grid map. Based on the odometry information, the local grid map is fused with the global grid map at the previous moment to generate a global grid map at the current moment. Based on the vehicle trajectory and geometric parameters of the intelligent vehicle, the wheel trajectory of the intelligent vehicle is predicted, the wheel trajectory is mapped to the global grid map at the current moment, and the elevation information of the wheel trajectory is calculated. The server 104 can feedback the obtained wheel trajectory elevation information to the terminal 102. In addition, in some embodiments, the road surface preview method based on point cloud input can also be implemented independently by the server 104 or the terminal 102. For example, the terminal 102 can directly perform road surface preview processing on the real-time point cloud data, or the server 104 can obtain the real-time point cloud data from the data storage system and perform road surface preview processing on the real-time point cloud data.

[0046] The terminal 102 may be, but is not limited to, various desktop computers and laptop computers. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0047] In an exemplary embodiment, Figure 2 、 Figure 3 and Figure 4 As shown, a road preview method based on point cloud input is provided. The method is executed by a computer device, specifically a computer device such as a terminal or a server, or a terminal and a server. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used for explanation, including the following steps 201 to 206.

[0048] Step 201: Acquire real-time point cloud data of the target area in the sensor coordinate system.

[0049] Step 202: Segment the real-time point cloud data in the sensor coordinate system to obtain ground area point clouds and non-ground area point clouds.

[0050] Step 203: Based on the ground area point cloud and the non-ground area point cloud, the grid information of the local map model of the target area of ​​interest is updated respectively to obtain a local grid map; the grid information includes the elevation information, confidence level and grid type of the grid; the grid type includes ground area and non-ground area.

[0051] Step 204: Based on the odometer information, the local grid map is merged with the global grid map at the previous moment to generate the global grid map at the current moment.

[0052] Step 205: Predict the wheel trajectory of the intelligent vehicle based on the vehicle trajectory and geometric parameters of the intelligent vehicle.

[0053] Step 206: Map the wheel trajectory to the global grid map at the current moment, and calculate the elevation information of the wheel trajectory.

[0054] By implementing steps 201 to 206 above, the ground area and non-ground area (i.e., obstacles) in the environment are distinguished through point cloud segmentation. The grid information is updated based on the ground area point cloud and the non-ground area point cloud to obtain a local grid map. This is then fused with the global grid map at the previous moment to obtain the global grid map at the current moment. Through modeling and analysis of the environmental point cloud, the vehicle's surrounding environment is digitally reconstructed, and the elevation information of the wheel track is predicted. This improves the accuracy and efficiency of feature reconstruction of terrain at different scales in the environment, providing accurate road surface information for autonomous vehicles and advanced driver assistance functions.

[0055] The road surface preview method based on point cloud input provided in this application includes four parts: sensor joint calibration, point cloud segmentation, resolution adaptive mapping and vehicle trajectory elevation prediction.

[0056] The sensor joint calibration part includes the sensor vehicle platform joint calibration to determine the transformation relationship between the sensor coordinate system S and the vehicle body coordinate system B.

[0057] In order to accurately describe the workflow of the entire system, such as Figure 5 As shown, this application introduces four coordinate systems, namely the vehicle body coordinate system B, the sensor coordinate system S, the world coordinate system W, and the map coordinate system M. Vehicle body coordinate system B: The vehicle body coordinate system is fixedly connected to the vehicle platform, with the origin O vehicle The x-axis is defined as the forward direction of the vehicle platform, the y-axis is defined as the left direction of the vehicle platform, and the z-axis is defined as the vertical upward direction. sensorThe world coordinate system W is the same as the vehicle initial coordinate system, and the coordinate axis direction is the same as the vehicle body coordinate system direction. The map coordinate system M is fixed to the vehicle platform, with the origin O map It is consistent with the ground projection point of the origin of the vehicle body coordinate system, and the direction of the coordinate axis is the same as that of the vehicle body coordinate system.

[0058] Through the previous offline calibration, the coordinate transformation matrix between the vehicle body coordinate system B and the sensor coordinate system S is known.

[0059] (1).

[0060] Where, represents the coordinate transformation matrix between the body coordinate system B and the sensor coordinate system S, Represents the rotation matrix from the body coordinate system B to the sensor coordinate system S, Represents the translation matrix from the body coordinate system B to the sensor coordinate system S.

[0061] The coordinate transformation matrix between the vehicle coordinate system B and the world coordinate system W is estimated online by the odometer, and the position and attitude changes of the vehicle coordinate system B relative to the world coordinate system W are output. The attitude change can be expressed as three angles between the vehicle body and the world coordinate system W, namely, the pitch angle , yaw angle and roll angle The odometer can use algorithms such as laser odometer and real-time kinematic (RTK) measurement technology.

[0062] (2).

[0063] Where, Represents the position and posture change matrix of the vehicle body coordinate system B relative to the world coordinate system W.

[0064] The pose transformation of the vehicle body coordinate system B relative to the world coordinate system W can be decoupled into the form shown in the following equation.

[0065] (3).

[0066] Where, represents the first matrix, Represents the second matrix.

[0067] In another exemplary embodiment of the present application, the above step 202 may include the following steps 301 and 302.

[0068] Step 301: Preprocess the real-time point cloud data in the sensor coordinate system S to obtain point cloud data within the region of interest (ROI).

[0069] Among them, step 301 specifically includes: performing posture transformation on the real-time point cloud data in the sensor coordinate system S to obtain point cloud data in the map coordinate system M; extracting the point cloud data in the map coordinate system M according to the preset local grid map size to obtain point cloud data in the area of ​​interest.

[0070] Step 302: Segment the point cloud data within the region of interest using a point cloud segmentation algorithm to obtain a ground area point cloud and a non-ground area point cloud.

[0071] The point cloud segmentation process includes point cloud preprocessing and segmentation. Based on the pre-set preview distance, the ROI region of interest (ROI) is set and the point cloud within the ROI is extracted. The point cloud segmentation algorithm is used to segment the ROI region into ground area point clouds and non-ground area point clouds.

[0072] In the point cloud preprocessing process, the real-time point cloud data obtained in the sensor coordinate system S is first Transform to map coordinate system M through pose transformation matrix , in order to eliminate the influence of vehicle dynamic response caused by road roughness excitation, vehicle acceleration and deceleration, etc. on point cloud data, and ensure the consistency and accuracy of point cloud data in the coordinate system at different times.

[0073] (4).

[0074] Where, 、 and They represent the first The x-coordinate, y-coordinate, and z-coordinate of each point cloud; Represents the pose transformation matrix from the map coordinate system M to the sensor coordinate system S; 、 、 Respectively represent the first The x-coordinate, y-coordinate, and z-coordinate of a point cloud.

[0075] According to the preset local grid map size, the point cloud within the region of interest is extracted from the converted data, and the point cloud data is further analyzed and processed within the specific area to reduce the interference of point clouds in non-ROI areas and improve the calculation speed.

[0076] During the acquisition of the environmental point cloud, sensors scan the road environment in front of the vehicle, generating point cloud data covering both ground and non-ground areas. These sensors include, but are not limited to, LiDAR and binocular cameras. Non-ground point clouds are often caused by other road users (such as pedestrians and vehicles) or objects (such as trees and obstacles) in the environment, which can affect accurate ground area reconstruction. Therefore, prior to mapping, the environmental point cloud must be classified to effectively distinguish between ground and non-ground point clouds, ensuring the accuracy and reliability of the subsequent map construction process. However, in real-world road scenarios, road slopes, uneven surfaces, and road discontinuities can lead to over- or under-segmentation when directly performing segmentation on the global environmental point cloud. To address complex road scenarios, the environmental point cloud is partitioned into equal-sized grids based on the assumption that local road areas are flat. A plane fitting method is then used to segment the point cloud within each grid. This local segmentation approach helps reduce the risk of mis-segmentation and improves the accuracy and reliability of the ground area point cloud.

[0077] The plane fitting method can be a method based on random sampling consistency or a principal component analysis method, etc.

[0078] The adaptive resolution mapping component includes point cloud updates, adaptive resolution optimization, and global map fusion. A two-layer raster map structure is constructed, storing the attributes (i.e., raster type) and elevation information of the corresponding location. The attribute information indicates whether the location is ground or non-ground, and the elevation information indicates the height of the location in the map coordinate system M. The segmented point cloud data is used to update the information in the initially constructed low-resolution map based on its attributes and XY plane position information, while the point cloud data for each grid is also saved. Discrete features are then calculated for the point cloud information within each grid. A quadtree structure is used to subdivide grids that do not meet the discrete feature threshold requirements until they meet the requirements or reach the highest resolution setting, achieving adaptive optimization for resolving terrain changes at different locations. Finally, the constructed local raster map is fused with the global raster map from the previous moment based on odometry information to generate the current global raster map.

[0079] In another exemplary embodiment of the present application, the above step 203 can be replaced by the following steps 401 to 404.

[0080] Step 401: Based on the position coordinates of the ground area point cloud, match the ground area point cloud with the grid in the local map model.

[0081] Step 402: For each grid, based on the position coordinates of the ground area point cloud matched with the grid, the grid information of the grid is updated to obtain updated ground grid information.

[0082] Step 403: Sort the non-ground area point clouds according to their heights, traverse the non-ground area point clouds in ascending order, update the raster information in the digital elevation map, and obtain updated non-ground raster information; all updated ground raster information and updated non-ground raster information constitute an updated raster map.

[0083] Step 404: Determine the resolution of the map area corresponding to the grid according to the confidence level of the grid in the updated grid map, so as to achieve adaptive resolution optimization and obtain a local grid map.

[0084] Based on the ground area point cloud and non-ground area point cloud segmented previously, the elevation data, confidence level, and grid type of the local map model are updated. First, the ground area point cloud is traversed and assigned to the grid at the corresponding position according to the position of the point cloud. The elevation information, confidence level, and grid type of the grid are then updated. The calculation formula is shown below.

[0085] (5).

[0086] in, Indicates the Z-axis coordinate value of the point cloud; and Respectively represent the elevation data and confidence data of the corresponding grid of the point cloud before updating; Indicates the variance of the point cloud during processing and the distance from the point cloud to the center of the sensor coordinate system and threshold related; and They respectively represent the elevation data and confidence data of the corresponding grid of the point cloud after the update; after the ground area point cloud is updated, the grid attribute where the ground area point cloud is located is set to the ground area.

[0087] The digital elevation map is updated by traversing the non-ground area point cloud data. The non-ground area point cloud contains obstacles connected to the ground, such as pedestrians, vehicles, and obstacles, as well as overhanging obstacles such as branches and signs. If the same update method based on Kalman filtering is used, the calculated grid elevation data will be lower than the actual obstacle height. At the same time, the height of the overhanging obstacle will cover the point cloud data of the ground area within the corresponding grid, causing the elevation map to be inconsistent with the actual situation. Therefore, before updating the ground area point cloud, the non-ground area point clouds are first sorted according to their height, and the non-ground area point cloud data is traversed starting from the point cloud with the lowest height. The elevation information, confidence level, and grid type in the digital elevation map are updated. The update formula corresponding to the non-ground area is shown below.

[0088] (6).

[0089] Where, Indicates that the difference between the Z-axis coordinate value of the point cloud and the elevation data of the corresponding grid before the point cloud is updated is not greater than the set threshold. When the difference is not greater than the set threshold, the Z-axis coordinate value of the point cloud is used as the elevation data of the corresponding grid after the point cloud is updated. When the difference is greater than the set threshold, the elevation data before the corresponding grid is updated is used as the elevation data of the corresponding grid after the point cloud is updated.

[0090] In the process of building a raster map, a larger grid resolution leads to low mapping accuracy and makes it difficult to accurately describe terrain changes. A smaller resolution will increase the computing power consumption of the elevation map construction process and increase the number of empty grids. Therefore, resolution adaptive optimization is introduced in the mapping process. The resolution of the map is adaptively determined according to the confidence level of the corresponding position. A larger resolution is used in areas with relatively flat roads to save computing power consumption in the overall map construction process. A smaller resolution is used in areas with large changes in road elevation to more accurately describe ground changes.

[0091] The above step 204 specifically includes the following steps 501 and 502.

[0092] Step 501: Based on the odometer information, the global grid map at the previous moment is converted to the map coordinate system at the current moment to obtain the global grid map at the previous moment in the map coordinate system.

[0093] Step 502: Based on the grid type of the grid, the local grid map is merged with the global grid map of the previous moment in the map coordinate system to generate the global grid map of the current moment.

[0094] In order to improve the accuracy and precision of mapping, the local grid map data is fused with the global grid map, and the two connected frames of elevation map are solved based on the odometry (including but not limited to: laser odometry, real-time kinematic measurement technology (RTK) etc.) and The pose transformation relationship between , first convert the global map of the previous moment to the map coordinate system M of the current moment. The calculation formula is shown below.

[0095] (7).

[0096] in, Indicates t- 1st moment in the global grid map Rank The position of the column grid relative to the map coordinate system M; They are t- 1st moment in the global grid map Row, No. The x-, y-, and z-coordinates of the column grid relative to the map coordinate system M; Indicates the maximum value of the global grid map in the vertical direction, Indicates the minimum value of the global grid map in the horizontal direction; resolution Indicates the resolution of the global grid map, Indicates the time t-1 Rank Column raster elevation information, Indicates t The global grid map at the moment Rank The position of the column grid relative to the map coordinate system M; They are t The global grid map at the moment Row, No. The x-, y-, and z-coordinates of the column grid relative to the map coordinate system M.

[0097] Will t The local grid map constructed at each moment t- The global grid map at time 1 is fused. First, calculate At the grid position at time t ( m , n ), the calculation formula is shown below.

[0098] (8).

[0099] During the data fusion process, when the grid attributes are inconsistent, the local grid map constructed at the current moment shall prevail.

[0100] (9).

[0101] in, and They represent the elevation information and confidence level of the corresponding grid position of the local grid map at the current moment, and Respectively represent the elevation information and confidence of the corresponding grid position of the global grid map at the previous moment, and Respectively represent the elevation information and confidence of the corresponding grid position in the fused global grid map, is the grid type of the corresponding grid position in the local grid map at the current moment, Indicates the grid type of the corresponding grid position in the global grid map at the current moment. Represents the grid type of the corresponding grid position in the local grid map at the previous moment.

[0102] The vehicle trajectory elevation prediction component includes wheel trajectory prediction and extraction. A kinematic model of the vehicle platform is constructed, and the vehicle trajectory is predicted based on real-time vehicle speed and front wheel angle information. Wheel trajectory prediction is achieved using calibrated vehicle geometry. The wheel trajectory is mapped onto a constructed global grid. Because the width of a wheel exceeds the resolution of the global map, a weighted average method is used to calculate the wheel trajectory elevation information.

[0103] In another exemplary embodiment of the present application, the above step 205 specifically includes the following steps 601 to 602.

[0104] Step 601: Construct a kinematic model of the intelligent vehicle, predict the vehicle's trajectory based on the vehicle's real-time speed information and front wheel angle information, and obtain the vehicle's trajectory.

[0105] Step 602: Predict the wheel trajectory of the intelligent vehicle based on the geometric parameters of the intelligent vehicle and the vehicle trajectory.

[0106] Construct the kinematic model of the vehicle itself. In order to simplify the calculation, only the movement of the vehicle in the XOY plane coordinate system is considered, such as Figure 6 shown.

[0107] In real driving scenarios, the driver controls the vehicle's speed through the accelerator and brakes. , control the vehicle's front wheel angle through the steering wheel angle The state of the vehicle is the position x in the X-axis direction, the position Y in the Y-axis direction and the heading angle of the vehicle. .

[0108] By decomposing the front and rear wheel speeds along the axial direction, the speed relationship between the front and rear wheels can be obtained as shown in the following formula.

[0109] (10).

[0110] Where, Indicates the speed of the rear wheel in the longitudinal direction, represents the speed of the rear wheel, Indicates the speed of the front wheel in the longitudinal direction, Indicates the front wheel speed.

[0111] The front and rear wheel speeds are decomposed in the lateral direction as shown in the following equation.

[0112] (11).

[0113] Where, Indicates the speed of the rear wheel in the horizontal direction, Indicates the speed of the front wheel in the horizontal direction.

[0114] The lateral speed will cause the vehicle to move in a circular motion with the rear wheel center as the center, so the angular velocity of the vehicle swing is Calculated by the following formula.

[0115] (12).

[0116] Where, Indicates the distance between the front and rear wheels.

[0117] Based on the above analysis, the state space equation of the Ackerman steering vehicle kinematic model can be obtained, as shown below.

[0118] (13).

[0119] In another exemplary embodiment of the present application, the above step 206 specifically includes the following steps 701 to 702.

[0120] Step 701: Determine a target grid based on the tire width of the intelligent vehicle and the resolution of the global grid map at the current moment; the target grid is the grid occupied by the predicted wheel driving trajectory at the same moment.

[0121] Step 702: performing weighted summation on the elevation information of all target grids to obtain the elevation information of the wheel track.

[0122] Based on the predicted vehicle trajectory and combined with the vehicle's geometric parameters, the wheel trajectory is predicted and mapped to the constructed global grid map at the current moment. Since the wheel width is greater than the map resolution, the weighted average method is used to fuse the elevation data occupied by the wheel width. The calculation formula is shown below.

[0123] (14).

[0124] Where, Indicates the number of target grids, which is determined by the tire width and map resolution; Indicates the width of the wheel. Elevation information of each target grid; Indicates the weight of the corresponding elevation; Represents the fused elevation, that is, the elevation information of the wheel track.

[0125] The vehicle's surroundings grid map constructed using this method not only displays elevation information but also expresses its attributes, providing multi-dimensional environmental data support for advanced driver assistance features in intelligent vehicles. By employing a resolution-adaptive optimization algorithm, a balance is achieved between mapping accuracy and time consumption. This approach accurately reconstructs terrain features at varying scales while avoiding a significant increase in mapping time.

[0126] This application also provides an application scenario that utilizes the aforementioned road surface preview method based on point cloud input. Specifically, the road surface preview method based on point cloud input provided in this embodiment can be applied in a road surface preview scenario. The road surface preview scenario includes an information collection process and a road surface preview field processing link; real-time point cloud data enters the road surface preview field processing link from the information collection process to obtain the corresponding wheel track elevation information. The road surface preview method based on point cloud input provided in this embodiment belongs to the road surface preview field processing link. Specifically, in the road surface preview field processing link for intelligent vehicles, the real-time point cloud data in the sensor coordinate system can be segmented to obtain ground area point clouds and non-ground area point clouds. Based on the ground area point clouds and non-ground area point clouds, the map grid information is updated to obtain a local grid map. Based on the odometry information, the local grid map is fused with the global grid map at the previous moment to generate a global grid map at the current moment. The wheel trajectory of the intelligent vehicle is predicted based on the vehicle trajectory and geometric parameters, and the wheel trajectory is mapped to the global grid map at the current moment to calculate the elevation information of the wheel trajectory.

[0127] Based on the same inventive concept, embodiments of the present application also provide a point cloud input-based road preview system for implementing the aforementioned point cloud input-based road preview method. The solution provided by this system is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the point cloud input-based road preview system provided below can be found in the above-mentioned limitations of the point cloud input-based road preview method and will not be further elaborated here.

[0128] In an exemplary embodiment, Figure 7 As shown, a road preview system based on point cloud input is provided, which includes the following modules.

[0129] The real-time point cloud data acquisition module T1 is used to obtain the real-time point cloud data of the target area in the sensor coordinate system.

[0130] The point cloud data segmentation module T2 is used to segment the real-time point cloud data in the sensor coordinate system to obtain the ground area point cloud and the non-ground area point cloud.

[0131] The grid information updating module T3 is used to update the grid information of the local map model of the target area of ​​interest based on the ground area point cloud and the non-ground area point cloud, respectively, to obtain a local grid map; the grid information includes the elevation information, confidence level and grid type of the grid; the grid type includes ground area and non-ground area.

[0132] The global grid map generation module T4 is used to fuse the local grid map with the global grid map at the previous moment based on the odometer information to generate the global grid map at the current moment.

[0133] The wheel driving trajectory prediction module T5 is used to predict the wheel driving trajectory of the intelligent vehicle based on the vehicle trajectory and geometric parameters of the intelligent vehicle.

[0134] The wheel track elevation information calculation module T6 is used to map the wheel track to the global grid map at the current moment and calculate the elevation information of the wheel track.

[0135] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store road surface preview processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a road surface preview method based on point cloud input is implemented.

[0136] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0137] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0138] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0139] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0140] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0141] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0142] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0143] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A road preview method based on point cloud input, characterized in that: The road surface preview method based on point cloud input includes: Obtain real-time point cloud data of the target area in the sensor coordinate system; Segment the real-time point cloud data in the sensor coordinate system to obtain the ground area point cloud and the non-ground area point cloud; Based on the ground area point cloud and the non-ground area point cloud, the grid information of the local map model of the target area of ​​interest is updated respectively to obtain a local grid map; the grid information includes the elevation information, confidence level and grid type of the grid; the grid type includes ground area and non-ground area; Based on the odometer information, the local grid map is merged with the global grid map at the previous moment to generate the global grid map at the current moment; Predict the wheel trajectory of the intelligent vehicle based on the vehicle's own trajectory and geometric parameters; The wheel trajectory is mapped to the global grid map at the current moment, and the elevation information of the wheel trajectory is calculated.

2. The road preview method based on point cloud input according to claim 1, characterized in that: Segment the real-time point cloud data in the sensor coordinate system to obtain the ground area point cloud and the non-ground area point cloud, including: Preprocess the real-time point cloud data in the sensor coordinate system to obtain the point cloud data within the area of ​​interest; The point cloud segmentation algorithm is used to segment the point cloud data in the area of ​​interest to obtain the ground area point cloud and the non-ground area point cloud.

3. The road preview method based on point cloud input according to claim 2, characterized in that: Preprocess the real-time point cloud data in the sensor coordinate system to obtain the point cloud data within the area of ​​interest, including: Perform pose transformation on the real-time point cloud data in the sensor coordinate system to obtain point cloud data in the map coordinate system; According to the preset local grid map size, the point cloud data in the map coordinate system is extracted to obtain the point cloud data in the area of ​​interest.

4. The road preview method based on point cloud input according to claim 1, characterized in that: Based on the ground area point cloud and the non-ground area point cloud, the grid information of the local map model of the target area of ​​interest is updated respectively to obtain a local grid map, specifically including: Based on the position coordinates of the ground area point cloud, the ground area point cloud is matched with the grid in the local map model; For each grid, based on the position coordinates of the ground area point cloud matched with the grid, the grid information of the grid is updated to obtain updated ground grid information; Sort the non-ground area point clouds according to their heights, traverse the non-ground area point clouds in ascending order, update the raster information in the digital elevation map, and obtain updated non-ground raster information; all updated ground raster information and updated non-ground raster information constitute an updated raster map; The resolution of the map area corresponding to the grid is determined according to the confidence of the grid in the updated grid map to achieve adaptive resolution optimization and obtain a local grid map.

5. The road preview method based on point cloud input according to claim 1, characterized in that: Based on the odometer information, the local grid map is merged with the global grid map at the previous moment to generate the global grid map at the current moment, specifically including: Based on the odometer information, the global grid map at the previous moment is converted to the map coordinate system at the current moment to obtain the global grid map at the previous moment in the map coordinate system; Based on the grid type of the grid, the local grid map is fused with the global grid map of the previous moment in the map coordinate system to generate the global grid map of the current moment.

6. The road surface preview method based on point cloud input according to claim 1, characterized in that: Predict the wheel trajectory of the intelligent vehicle based on its own trajectory and geometric parameters, specifically including: Build a kinematic model of the intelligent vehicle, predict the vehicle's trajectory based on the vehicle's real-time speed information and front wheel angle information, and obtain the vehicle's trajectory; According to the geometric parameters of the intelligent vehicle and the vehicle trajectory, the wheel trajectory of the intelligent vehicle is predicted.

7. The road preview method based on point cloud input according to claim 1, characterized in that: Map the wheel trajectory to the global grid map at the current moment and calculate the elevation information of the wheel trajectory, including: Determine the target grid based on the tire width of the intelligent vehicle and the resolution of the global grid map at the current moment; the target grid is the grid occupied by the predicted wheel trajectory at the same moment; The elevation information of all the target grids is weightedly summed to obtain the elevation information of the wheel track.

8. A road preview system based on point cloud input based on the road preview method based on point cloud input according to any one of claims 1 to 7, characterized in that: The road surface preview system based on point cloud input includes: Real-time point cloud data acquisition module, used to obtain real-time point cloud data of the target area in the sensor coordinate system; Point cloud data segmentation module, used to segment the real-time point cloud data in the sensor coordinate system to obtain ground area point cloud and non-ground area point cloud; A grid information update module is used to update the grid information of the local map model of the target area of ​​interest based on the ground area point cloud and the non-ground area point cloud, respectively, to obtain a local grid map; the grid information includes the grid elevation information, confidence level and grid type; the grid type includes ground area and non-ground area; A global grid map generation module is used to fuse the local grid map with the global grid map at the previous moment based on the odometer information to generate a global grid map at the current moment; A wheel driving trajectory prediction module is used to predict the wheel driving trajectory of the intelligent vehicle based on the vehicle trajectory and geometric parameters of the intelligent vehicle; The wheel track elevation information calculation module is used to map the wheel track to the global grid map at the current moment and calculate the wheel track elevation information.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the road surface preview method based on point cloud input according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the road surface preview method based on point cloud input according to any one of claims 1 to 7 is implemented.

Images