Vehicle and obstacle detection method

Abstract

The invention discloses a vehicle and an obstacle detection method. The method comprises the following steps: acquiring a plurality of pose data of a vehicle and point cloud data corresponding to the pose data; splicing the plurality of point cloud data according to the plurality of pose data to obtain a point cloud map; performing downsampling on the point cloud map to obtain a grid map, wherein the grid map comprises a plurality of grids; height difference values between the grids and neighborhood grids are obtained, a target layer is constructed according to the multiple height difference values, the target layer comprises multiple pixel points, the pixel values of the pixel points are used for representing the height difference between a target position corresponding to the pixel points and the surrounding terrain of the target position, and the target position is located in the surrounding environment of the vehicle; and according to the pixel value of each pixel point in the target layer, detecting an obstacle in the surrounding environment of the vehicle. According to the embodiment of the invention, the problems of rare features and difficult or insufficient feature extraction of the point cloud map can be avoided, and the obstacles around the vehicle can be accurately detected.

Description

Technical Field

[0001] The present application belongs to the technical field of obstacle detection, and in particular relates to a vehicle and obstacle detection method. Background Art

[0002] When a vehicle is driving, it often encounters obstacles, such as low obstacles. These obstacles are more difficult to identify than most obstacles on the road, and therefore affect the vehicle's ability to avoid low obstacles.

[0003] In related technologies, road obstacle detection is mainly achieved by collecting sparse point clouds around the vehicle. However, if obstacle detection is performed directly in the sparse point clouds, missed detection or false detection may easily occur. Summary of the Invention

[0004] The embodiments of the present application provide a vehicle and obstacle detection method that can, at least to a certain extent, avoid the problems of sparse features in point cloud maps and difficult or insufficient feature extraction, and can accurately detect obstacles around the vehicle.

[0005] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by practice of the present application.

[0006] A first aspect of an embodiment of the present application provides an obstacle detection method, comprising:

[0007] Acquire multiple posture data of a vehicle and point cloud data corresponding to the posture data, wherein the point cloud data is point cloud data of the surrounding environment of the vehicle having the posture data, and at least two of the posture data are different;

[0008] splicing the plurality of point cloud data according to the plurality of posture data to obtain a point cloud map;

[0009] Downsampling the point cloud map to obtain a grid map, wherein the grid map includes a plurality of grids arrayed along a first direction and a second direction, and the first direction and the second direction intersect;

[0010] Obtaining a height difference between the grid and a neighboring grid, and constructing a target layer based on the plurality of height difference values, wherein the height difference value is a height difference value in a third direction, the third direction intersecting the first direction and the second direction, the target layer including a plurality of pixel points, the pixel values ​​of the pixel points being used to represent a height difference between a target position corresponding to the pixel point and a surrounding terrain of the target position, the target position being located in a surrounding environment of the vehicle;

[0011] Obstacles in the surrounding environment of the vehicle are detected according to the pixel value of each pixel point in the target layer.

[0012] Optionally, the step of splicing the plurality of point cloud data according to the plurality of pose data to obtain a point cloud map includes:

[0013] For each point cloud data, converting the point cloud data into a target coordinate system according to the pose data corresponding to the point cloud data, to obtain a sub-point cloud map of the point cloud data in the target coordinate system;

[0014] Each of the sub-point cloud maps is spliced ​​and fused to obtain the point cloud data.

[0015] Optionally, when obtaining point cloud data corresponding to the pose data, the method further includes:

[0016] The point cloud data in the surrounding environment of the vehicle is updated based on a dynamic window, where the range of the dynamic window is smaller than or equal to the range of the surrounding environment.

[0017] Optionally, updating the point cloud data in the surrounding environment of the vehicle based on a dynamic window includes:

[0018] Acquire real-time point cloud data in real time within the dynamic window range;

[0019] If the difference between the real-time point cloud data and the point cloud data in the local map is greater than or equal to a preset difference threshold, the local map is updated based on the real-time point cloud data.

[0020] Optionally, the range of the dynamic window is positively correlated with the speed of the vehicle.

[0021] Optionally, downsampling the point cloud map to obtain a raster map includes:

[0022] Downsampling the point cloud map at a first resolution along the first direction and at a second resolution along the second direction to obtain the grid map;

[0023] Each of the grids in the grid map carries grid information, and the grid information includes coordinate information of the grid in the grid map and height information of the grid, and the height information includes one or more of the maximum height, minimum height and average height of the point cloud set within the grid.

[0024] Optionally, detecting obstacles in the surrounding environment of the vehicle according to the pixel value of each pixel point in the target layer includes:

[0025] If the pixel value of the pixel point is greater than the pixel threshold, and the pixel value of the pixel point is greater than the pixel values ​​of each of the surrounding pixels, then there is a raised obstacle at the target position;

[0026] If the pixel value of the pixel point is less than the pixel threshold, and the pixel value of the pixel point is less than the pixel values ​​of each of the surrounding pixel points, then there is a pothole obstacle at the target position.

[0027] Optionally, before detecting obstacles in the surrounding environment of the vehicle according to the pixel values ​​of each pixel point in the target layer, the method further includes:

[0028] Delete a target pixel point in the target layer, where the pixel value of the target pixel point is equal to the pixel threshold.

[0029] Optionally, the acquiring of a plurality of posture data of the vehicle and point cloud data corresponding to the posture data includes:

[0030] Acquire multiple initial posture data of the vehicle and the point cloud data corresponding to the initial posture data;

[0031] Matching the point cloud data with the local map, and obtaining point-surface observation residuals according to the matching results;

[0032] Based on the point-surface observation residuals and the IMU prediction residuals, the initial posture is corrected to obtain a plurality of posture data.

[0033] A second aspect of an embodiment of the present application provides an obstacle detection device, comprising:

[0034] an acquisition unit, configured to acquire a plurality of posture data of a vehicle and point cloud data corresponding to the posture data, wherein the point cloud data is point cloud data of a surrounding environment of the vehicle having the posture data, and at least two of the posture data are different;

[0035] a splicing unit, configured to splice the plurality of point cloud data according to the plurality of posture data to obtain a point cloud map;

[0036] a downsampling unit, configured to downsample the point cloud map to obtain a grid map, wherein the grid map comprises a plurality of grids arrayed along a first direction and a second direction, the first direction and the second direction intersecting;

[0037] a construction unit, configured to obtain a height difference between the grid and a neighboring grid, and construct a target layer based on the plurality of height difference values, wherein the height difference value is a height difference value in a third direction, the third direction intersecting the first direction and the second direction, the target layer including a plurality of pixel points, the pixel values ​​of the pixel points being used to represent a height difference between a target position corresponding to the pixel point and a surrounding terrain of the target position, the target position being located in a surrounding environment of the vehicle;

[0038] The detection unit is used to detect obstacles in the surrounding environment of the vehicle based on the pixel value of each pixel point in the target layer.

[0039] According to a third aspect of an embodiment of the present application, a vehicle is provided, comprising one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to implement the operations performed by any of the methods described in the first aspect.

[0040] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one computer program instruction, and the at least one computer program instruction is loaded and executed by a processor to implement the operations performed by any method described in the first aspect.

[0041] The one or more technical solutions provided by the embodiments of the present invention achieve at least the following technical effects or advantages:

[0042] The present application obtains multiple posture data of a vehicle and point cloud data corresponding to the posture data, wherein the point cloud data is point cloud data of the surrounding environment of the vehicle with posture data, and at least two posture data are different; multiple point cloud data are spliced ​​according to the multiple posture data to obtain a point cloud map; the point cloud map is downsampled to obtain a grid map, wherein the grid map includes multiple grids arranged in an array along a first direction and a second direction, and the first direction and the second direction intersect; the height difference between the grid and the neighboring grid is obtained, and a target layer is constructed according to the multiple height difference values, wherein the height difference value is the height difference value in a third direction, and the third direction intersects with the first direction and the second direction, and the target layer includes multiple pixel points, and the pixel value of the pixel point is used to represent the height difference between the target position corresponding to the pixel point and the surrounding terrain of the target position, and the target position is located in the surrounding environment of the vehicle; obstacles in the surrounding environment of the vehicle are detected according to the pixel value of each pixel point in the target layer. Therefore, the embodiment of the present application can avoid the problem of sparse features of the point cloud map, difficult or insufficient feature extraction, and can accurately detect obstacles around the vehicle.

[0043] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings are incorporated into and constitute a part of the specification, illustrating embodiments consistent with the present application and, together with the specification, explaining the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can derive other drawings based on these drawings without inventive effort. In the drawings:

[0045] Figure 1 A flowchart of an obstacle detection method according to an embodiment of the present application is shown;

[0046] Figure 2 A structural diagram of an obstacle detection device according to an embodiment of the present application is shown;

[0047] Figure 3 A schematic diagram of the structure of a vehicle computer system suitable for implementing an embodiment of the present application is shown. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0049] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner.In the following description, many specific details are provided so as to provide a full understanding of the embodiments of the present application. However, it will be appreciated by those skilled in the art that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps etc. can be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0050] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. In other words, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different models and / or processor devices and / or microcontroller devices.

[0051] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0052] It should also be noted that the terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that shown or described.

[0053] When a vehicle is driving, it often encounters obstacles, such as low obstacles. These obstacles are more difficult to identify than most obstacles on the road, and therefore affect the vehicle's ability to avoid low obstacles.

[0054] In related technologies, road obstacle detection is mainly achieved by collecting sparse point clouds around the vehicle. However, if obstacle detection is performed directly in the sparse point clouds, missed detection or false detection may easily occur.

[0055] In view of this, the first aspect of the embodiment of the present application provides an obstacle detection method, which can avoid

[0056] In a first aspect, an embodiment of the present application provides an obstacle detection method that can be executed on a vehicle controller, including but not limited to steps S10 to S50:

[0057] Step S10. Acquire multiple posture data of the vehicle and point cloud data corresponding to the posture data, wherein the point cloud data is point cloud data of the surrounding environment of the vehicle having the posture data, and at least two of the posture data are different;

[0058] It is understandable that the posture data may refer to the data obtained by integrating and predicting the acceleration data and angular velocity data measured by the IMU (Inertial Measurement Unit), or the posture data obtained by fusing the IMU measurement data and the point cloud data, which is not limited here. Among them, the IMU is used to measure the motion state and posture of the vehicle, and includes multiple sensors, such as accelerometers, gyroscopes and magnetometers. The accelerometer is used to measure the acceleration of the vehicle in space, and the gyroscope is used to measure the angular velocity of the wheel. The output data of the IMU sensor includes acceleration and angular velocity values ​​in three axes. The three axes can represent the front x, left y, and top z directions of the IMU installation position. The angular velocity can be the speed of rotation around the x-axis, around the y-axis, and around the z-axis respectively.

[0059] It can be understood that point cloud data can be collected through sensors such as lidar, RGB-D (RGB-Depth, visual sensing that combines color images (RGB) and depth information (Depth)) cameras, stereo cameras, etc. The point cloud data is point cloud data in the vehicle's surrounding environment, and the surrounding environment includes the road surface where the vehicle is located.

[0060] It should be noted that point cloud data is used to represent the three-dimensional spatial information in the vehicle's surrounding environment. Each point cloud data contains multiple spatial points, and each spatial point has its corresponding spatial coordinates (x, y, z). Set the transformation between the map coordinate system and the world coordinate system to R (rotation matrix) and t (translation vector), then the point cloud data can be unified to the world coordinate system according to the above transformation to form a local map. Among them, the local map is the three-dimensional structure of the current vehicle's surrounding environment, for example, the environment within a radius of 10 meters to 50 meters with the vehicle as the center, which is usually generated by accumulating historical point cloud data or extracting features.

[0061] It can be understood that the posture data can be used to describe the position, posture and speed of the vehicle, etc. The point cloud data corresponding to the posture data refers to the point cloud data of the vehicle's surrounding environment collected by the point cloud sensor when the vehicle is in this posture.

[0062] At least two of the posture data are different, which may mean that each of the posture data is different.

[0063] Therefore, the point cloud data collected by the embodiment of the present application are different point cloud data collected by the vehicle in multiple postures, and the environments corresponding to each point cloud data may overlap or may not overlap.

[0064] In some embodiments, the acquiring of a plurality of posture data of the vehicle and point cloud data corresponding to the posture data includes:

[0065] Step S11. Acquire multiple initial posture data of the vehicle and the point cloud data corresponding to the initial posture data;

[0066] It can be understood that the initial posture data is obtained by integrating the acceleration and angular velocity collected by the IMU. Before performing the IMU integration, the state of the IMU can be initialized. The state of the IMU includes the initial position, velocity and posture of the sensor (represented by the rotation matrix or quaternion). During the initialization process, the initial state can be determined by auxiliary information from the GPS (Global Positioning System) or other external positioning systems, or a known static state can be used as the starting point.

[0067] It can be understood that during the driving process of the vehicle, the posture data of the vehicle is in a continuous changing process. Then, for the posture data at the current moment, it can be obtained by integrating the posture data at the previous moment and the acceleration and angular velocity collected by the IMU at the current moment. Exemplary speed integration: Integrate the acceleration data to obtain the speed change of the vehicle, and obtain the speed increment of the vehicle within the time step by continuously integrating the acceleration. Position integration: By integrating the speed data, the displacement of the vehicle within the time step is obtained, and the obtained displacement information is used to update the position of the vehicle. Attitude integration: The angular velocity data measured by the gyroscope is integrated using quaternions or rotation matrices to obtain the attitude change of the vehicle, and the obtained attitude information is used to update the orientation of the vehicle.

[0068] It can be understood that at the current moment of obtaining the initial posture data, point cloud data around the vehicle is also obtained through sensors such as laser radar, and at this time the point cloud data corresponds to the initial posture data.

[0069] Step S12: matching the point cloud data with the local map, and obtaining point-surface observation residuals according to the matching results;

[0070] It is understandable that during the IMU integration process, sensor noise, drift, and other error sources may cause cumulative errors in position, velocity, and attitude. In view of this, an update mechanism based on error correction is adopted to avoid the impact of errors on the IMU state estimate.

[0071] In three-dimensional space, the correspondence between points and surfaces can be measured by the distance from the point to a plane in the local map. For a planar model in the local map, each point in the point cloud is matched to the nearest plane, and the distance from the point to the plane is calculated. Based on the matching relationship between points and planes, the point-to-plane observation residual can be constructed.

[0072] It should be noted that the point-to-plane observation residual represents the difference between the distance from each point to the plane and the theoretical value. This residual is often used to optimize the matching accuracy between point cloud data and the plane model in the local map. If there is no error, the distance between the point and the plane is the theoretical value of 0. The point-to-plane distance represents the measured value of the point-to-plane distance corresponding to the predicted pose. Therefore, by changing the pose, the measured distance from the point to the plane can be changed, bringing the measured value closer to the theoretical value of 0. Therefore, the pose can be optimized using the point-to-plane observation residual.

[0073] Where, for each point cloud: r point-plane,j =|p j n+d|, where p j is the position of the point, n is the normal vector of the plane, and d is the constant term of the plane equation.

[0074] By minimizing the point-surface observation residuals, the matching accuracy between point cloud data and plane models is optimized, and it is used to adjust the parameters of point cloud data or plane models to achieve the best match.

[0075] Step S13. Based on the point-surface observation residuals and the IMU prediction residuals, the initial posture is corrected to obtain a plurality of posture data.

[0076] It should be noted that the IMU prediction residual is obtained by comparing the predicted IMU state with the pose obtained from the local map. That is, the IMU prediction residual represents the difference between the predicted state and the actual observed state. The IMU prediction residual can include position residual, velocity residual, and attitude residual. Among them, the position residual represents the difference between the predicted position and the actual position, the velocity residual represents the difference between the predicted velocity and the actual velocity, and the attitude residual represents the difference between the predicted attitude (usually represented by quaternions) and the actual attitude.

[0077] Therefore, the IMU provides the acceleration and angular velocity information of the vehicle, and the point cloud provides the three-dimensional geometric structure of the environment. By fusing the information of these two sensors, a more accurate and reliable posture estimation of the vehicle can be obtained.

[0078] For example, a comprehensive residual function is constructed by using the IMU prediction residual and the point-surface observation residual. The comprehensive residual function can be expressed as:

[0079]

[0080] Among them, r imu,i is the IMU prediction residual, n represents multiple pose data, such as speed, position and attitude, i represents one of the pose data, r point-plane,j is the residual error of point-surface observation, m represents multiple point cloud data, and j represents one of the point cloud data. The pose is optimized by minimizing the comprehensive residual function.

[0081] For example, the comprehensive residual function is optimized using a nonlinear least squares method to obtain the precise pose of the vehicle at the current moment. The optimized pose includes position, velocity, and attitude information. During the optimization process, the matching parameters of the IMU state and point cloud data are adjusted to minimize the residual between the IMU prediction and the point cloud observation, thereby improving the accuracy of the pose estimation. The optimized pose is then used to update the system state. The updated pose serves as the basis for the IMU integration and point cloud observation at the next moment, forming a continuous optimization process. That is, pose calculation is a continuous process. The result of this calculation is an increment superimposed on the previous result. Therefore, the pose prediction at the next moment is an increment superimposed on the result of this calculation. Since the result of this calculation is optimized by the sum residual of the IMU and point cloud observations, the pose obtained by this calculation can be considered accurate and can provide a good initial value for the pose calculation at the next moment, enabling the residual calculation at the next moment to converge as quickly as possible.

[0082] In some embodiments, when acquiring point cloud data corresponding to the pose data, the method further includes:

[0083] The point cloud data in the surrounding environment of the vehicle is updated based on a dynamic window, where the range of the dynamic window is smaller than or equal to the range of the surrounding environment.

[0084] Therefore, the calculation of the dynamic window only focuses on the area around the vehicle, without globally updating the entire point cloud map. This local update method effectively reduces the amount of calculation required for each update, thereby reducing the use of computing resources.

[0085] In some embodiments, updating the point cloud data in the surrounding environment of the vehicle based on a dynamic window includes:

[0086] Step S101. Acquire real-time point cloud data within the dynamic window range in real time;

[0087] Step S102: If the difference between the real-time point cloud data and the point cloud data in the local map is greater than or equal to a preset difference threshold, the local map is updated based on the real-time point cloud data.

[0088] That is to say, by calculating the difference between each point cloud data point and the point cloud data in the local map, the point cloud data in the area with large differences or the newly added point cloud data in the area is selectively updated. Then, for stable areas, the update frequency of the point cloud data is relatively low, which can effectively reduce unnecessary calculations.

[0089] In some embodiments, the range of the dynamic window is positively correlated with the speed of the vehicle.

[0090] In other words, the dynamic window dynamically adjusts its range based on the vehicle's real-time motion. For example, at high speeds, the dynamic window can expand to accommodate a larger update area; at low speeds or when the vehicle is stationary, the dynamic window can decrease, reducing unnecessary calculations. This real-time adjustment of the dynamic window ensures that only the areas most in need of updates are calculated, reducing system resource usage.

[0091] In some embodiments, for the point cloud data within the dynamic window, voxel filtering is used to reduce the resolution of the point cloud, thereby compressing and simplifying the data, thereby reducing computing resource usage while maintaining sufficient environmental accuracy.

[0092] Furthermore, a hierarchical computing strategy can be employed based on the computational requirements of different areas of the vehicle's surroundings. For example, lower computational complexity can be applied to areas with fewer obstacles, while higher accuracy requirements can be applied to high-priority areas with obstacles.

[0093] Therefore, during the dynamic window update process, the updated point cloud data obtained will be integrated into the existing local map, and the point cloud that exceeds the current vehicle's perception range can be deleted, thereby ensuring that the local map can follow the vehicle's position in real time.

[0094] Step S20: splicing the plurality of point cloud data according to the plurality of pose data to obtain a point cloud map;

[0095] In some embodiments, the step of splicing the plurality of point cloud data according to the plurality of pose data to obtain a point cloud map includes:

[0096] Step S21. For each point cloud data, convert the point cloud data into a target coordinate system according to the pose data corresponding to the point cloud data, to obtain a sub-point cloud map of the point cloud data in the target coordinate system;

[0097] Step S22: stitching and fusing the sub-point cloud maps to obtain the point cloud data.

[0098] Exemplary stitching methods include those based on rigid transformation, in which the point cloud data is converted to a unified coordinate system and fused by applying corresponding pose data to each point cloud data point. This merges point cloud data from multiple observation locations into a single point cloud map, and fuses sparse point cloud data, making the stitched point cloud map denser and more feature-rich, facilitating subsequent feature extraction. This point cloud map can more accurately reflect the environmental structure surrounding the vehicle and provide spatial perception information for subsequent obstacle extraction. Furthermore, by combining IMU sensor data with lidar sensor data, data missing due to obstacles blocking the radar ranging signal can be supplemented, avoiding misidentification of potholes.

[0099] Among them, point cloud data is a collection of clusters of points. Since a cluster of point clouds only corresponds to a unique rigid transformation, each point in a cluster of point clouds uses the same rigid transformation; and the point cloud map is composed of multiple clusters (frames) of point clouds, so the rigid transformation corresponding to each cluster of point clouds can transform each cluster of point clouds into a unified coordinate system for representation.

[0100] Step S30. Downsample the point cloud map to obtain a grid map, wherein the grid map includes a plurality of grids arrayed along a first direction and a second direction, and the first direction and the second direction intersect, for example, the first direction and the second direction are perpendicular.

[0101] For example, the first direction and the second direction may refer to directions on the road where the vehicle is located.

[0102] In some embodiments, downsampling the point cloud map to obtain a raster map includes:

[0103] Downsampling the point cloud map at a first resolution along the first direction and at a second resolution along the second direction to obtain the grid map;

[0104] Each of the grids in the grid map carries grid information, and the grid information includes coordinate information of the grid in the grid map and height information of the grid, and the height information includes one or more of the maximum height, minimum height and average height of the point cloud set within the grid.

[0105] For example: obtain the point cloud set in each grid with the first resolution and the second resolution in the first direction x and the second direction y respectively, and store the maximum height, minimum height, and average height in the point cloud set in the following format (x, y, hmax, hmin, haver); wherein the two-dimensional coordinates of each grid are (x, y), representing the position of the grid in the point cloud map coordinate system; each grid is also provided with three fields (hmax, hmin, haver).

[0106] Step S40. Obtaining a height difference between the grid and a neighboring grid, and constructing a target layer based on the plurality of height difference values, wherein the height difference value is a height difference value in a third direction, the third direction intersecting the first direction and the second direction, the target layer including a plurality of pixel points, the pixel values ​​of the pixel points being used to represent a height difference between a target location corresponding to the pixel point and a surrounding terrain of the target location, the target location being located in the surrounding environment of the vehicle;

[0107] It should be noted that the neighborhood grid of a grid refers to multiple grids adjacent to the grid, such as the 8-neighborhood grid. After the point cloud is discretized into a grid map, except for the boundary points, the grid corresponding to each other point has eight adjacent grids around it, namely, up, down, left, right, upper left, upper right, lower left, and lower right, which are called 8-neighborhoods; the corresponding height value (h value) in each grid is the maximum, minimum, and average Z value of the point cloud in the corresponding area; then the sum of the differences between the height value corresponding to the grid and the corresponding height values ​​of the surrounding 8 grids is the value of the pixel point corresponding to the grid in the target layer.

[0108] When there are multiple neighboring grids, the height difference value is actually the sum of the height differences of each grid, for example, the sum of the height differences with 8 neighboring grids.

[0109] It can be understood that, with the road surface as a reference, the third direction can refer to the height direction of the road surface. Then, the pixel value of the pixel point corresponding to the grid is used to represent the height difference between the target position corresponding to the pixel point and the surrounding terrain of the target position, which can actually reflect whether there is an obstacle on the road surface at the target position.

[0110] Step S50: Detect obstacles in the surrounding environment of the vehicle according to the pixel value of each pixel point in the target layer.

[0111] In some embodiments, detecting obstacles in the surrounding environment of the vehicle according to the pixel value of each pixel point in the target layer includes:

[0112] Step S51: If the pixel value of the pixel point is greater than the pixel threshold, and the pixel value of the pixel point is greater than the pixel values ​​of each of the surrounding pixels, then there is a raised obstacle at the target location;

[0113] For example, assuming the road surface is used as the reference surface, and the road surface height is the pixel threshold, which can be zero, for example. Then, if the pixel value of a pixel is greater than 0 and greater than the pixel values ​​of the surrounding pixels, it indicates that the pixel value of the pixel is a local maximum, indicating that there is a raised obstacle at the target location corresponding to the pixel, and the maximum value represents the average height of the obstacle at that location.

[0114] Step S52: If the pixel value of the pixel point is less than the pixel threshold, and the pixel value of the pixel point is less than the pixel values ​​of each of the surrounding pixels, then there is a pothole obstacle at the target location.

[0115] It can be understood that when the pixel value of a pixel point is less than 0 and is smaller than or greater than the pixel value of any surrounding point, it means that the point is a local minimum, indicating that the obstacle here is a pothole obstacle, and the minimum value represents the average height of the obstacle here.

[0116] In some embodiments, before detecting obstacles in the surrounding environment of the vehicle based on the pixel values ​​of each pixel point in the target layer, the method further includes:

[0117] Delete the target pixel point in the target layer, the pixel value of the target pixel point is equal to the pixel threshold, so that the pixel points other than the target pixel point constitute the obstacle layer.

[0118] In an embodiment of the present application, by detecting obstacles in the vehicle's surrounding environment, data support can be provided for vehicle trajectory prediction, trajectory clipping, etc.

[0119] Based on the above-disclosed content, the embodiments of the present application can complete environmental perception during the vehicle's movement, and can extract obstacle information around the vehicle body regardless of the vehicle speed; by constructing a local map, multiple point cloud data are superimposed on a unified coordinate system to obtain a local map with dense point clouds, and feature extraction is performed in the local map, which can avoid the problems of sparse features, difficult and insufficient feature extraction directly in sparse point clouds; the embodiments of the present application can detect low obstacles or potholes, and can also extract obstacles on sloped roads, and has a wide range of applications; the present application is based on a dynamic window to achieve feature extraction only in the generated local map, which can reduce the use of computing resources, thereby providing more redundant time for the upper-level decision-making and planning module, and improving driving safety.

[0120] Figure 2A schematic structural diagram of an obstacle detection device according to an embodiment of the present application is shown.

[0121] A second aspect of an embodiment of the present application provides an obstacle detection device 200, comprising:

[0122] An acquisition unit 201 is configured to acquire a plurality of posture data of a vehicle and point cloud data corresponding to the posture data, wherein the point cloud data is point cloud data of the surrounding environment of the vehicle having the posture data, and at least two of the posture data are different;

[0123] A stitching unit 202 is configured to stitch the plurality of point cloud data together according to the plurality of posture data to obtain a point cloud map;

[0124] a downsampling unit 203 configured to downsample the point cloud map to obtain a grid map, wherein the grid map includes a plurality of grids arrayed along a first direction and a second direction, the first direction and the second direction intersecting;

[0125] A construction unit 204 is configured to obtain a height difference between the grid and a neighboring grid, and construct a target layer based on the plurality of height difference values, wherein the height difference value is a height difference value in a third direction, the third direction intersecting the first direction and the second direction, the target layer including a plurality of pixels, the pixel values ​​of the pixels being used to represent a height difference between a target location corresponding to the pixel point and a surrounding terrain of the target location, the target location being located in an environment surrounding the vehicle;

[0126] The detection unit 205 is used to detect obstacles in the surrounding environment of the vehicle according to the pixel value of each pixel point in the target layer.

[0127] According to a third aspect of an embodiment of the present application, a vehicle is provided, comprising one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to implement the operations performed by any of the methods described in the first aspect.

[0128] like Figure 3 As shown, vehicle 400 is represented as a general-purpose computing device. Components of vehicle 400 may include, but are not limited to, the aforementioned at least one processing unit 410, the aforementioned at least one storage unit 420, and a bus 430 connecting various system components (including storage unit 420 and processing unit 410).

[0129] The storage unit stores program code, which can be executed by the processing unit 410, so that the processing unit 410 executes the steps described in the above "Example Method" section of this specification according to various exemplary embodiments of the present application.

[0130] The storage unit 420 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 421 and / or a cache 422 , and may further include a read-only memory unit (ROM) 423 .

[0131] The storage unit 420 may also include a program / utility 424 having a set (at least one) of program modules 425, such program modules 425 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0132] Bus 430 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0133] Vehicle 400 can also communicate with one or more external devices 500 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with vehicle 400, and / or any device that enables vehicle 400 to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can occur via I / O (input / output) interface 450, which can also be connected to display unit 440 for displaying the communications. Furthermore, vehicle 400 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via network adapter 460. As shown, network adapter 460 communicates with other modules of vehicle 400 via bus 430. It should be understood that, although not shown, other hardware and / or software modules can be used in conjunction with vehicle 400, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0134] The functions described herein may be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions may be stored as one or more instructions or codes on or transmitted via a computer-readable medium. Other examples and implementations are within the scope and spirit of the present invention and the appended claims. For example, due to the nature of software, the functions described above may be implemented using software executed by a processor, hardware, firmware, hardwiring, or a combination of any of these. Furthermore, each functional unit may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit.

[0135] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one computer program instruction, and the at least one computer program instruction is loaded and executed by a processor to implement the operations performed by any method described in the first aspect.

[0136] The computer-readable storage medium may be a portable compact disc read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the computer-readable storage medium of the present application is not limited thereto. In the present application, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0137] The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0138] The program code for performing the operations of the present application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and the like, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0139] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0140] The units described as separate components may or may not be physically separate, and the components of the control device may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0141] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc., which can store program code.

[0142] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of the claims of the present application.

Claims

1. An obstacle detection method, characterized in that: include: Acquire multiple posture data of a vehicle and point cloud data corresponding to the posture data, wherein the point cloud data is point cloud data of the surrounding environment of the vehicle having the posture data, and at least two of the posture data are different; splicing the plurality of point cloud data according to the plurality of posture data to obtain a point cloud map; Downsampling the point cloud map to obtain a grid map, wherein the grid map includes a plurality of grids arrayed along a first direction and a second direction, and the first direction and the second direction intersect; Obtaining a height difference between the grid and a neighboring grid, and constructing a target layer based on the plurality of height difference values, wherein the height difference value is a height difference value in a third direction, the third direction intersecting the first direction and the second direction, the target layer including a plurality of pixel points, the pixel values ​​of the pixel points being used to represent a height difference between a target position corresponding to the pixel point and a surrounding terrain of the target position, the target position being located in a surrounding environment of the vehicle; Obstacles in the surrounding environment of the vehicle are detected according to the pixel value of each pixel point in the target layer.

2. The method according to claim 1, characterized in that The step of splicing the plurality of point cloud data according to the plurality of posture data to obtain a point cloud map includes: For each point cloud data, converting the point cloud data into a target coordinate system according to the pose data corresponding to the point cloud data, to obtain a sub-point cloud map of the point cloud data in the target coordinate system; Each of the sub-point cloud maps is spliced ​​and fused to obtain the point cloud data.

3. The method according to claim 1, characterized in that When acquiring point cloud data corresponding to the pose data, the method further includes: The point cloud data in the surrounding environment of the vehicle is updated based on a dynamic window, where the range of the dynamic window is smaller than or equal to the range of the surrounding environment.

4. The method according to claim 3, characterized in that The updating of the point cloud data in the surrounding environment of the vehicle based on the dynamic window includes: Acquire real-time point cloud data in real time within the dynamic window range; If the difference between the real-time point cloud data and the point cloud data in the local map is greater than or equal to a preset difference threshold, the local map is updated based on the real-time point cloud data.

5. The method according to claim 3 or 4, characterized in that The range of the dynamic window is positively correlated with the speed of the vehicle.

6. The method according to claim 1, characterized in that The downsampling of the point cloud map to obtain a raster map includes: Downsampling the point cloud map at a first resolution along the first direction and at a second resolution along the second direction to obtain the grid map; Each of the grids in the grid map carries grid information, and the grid information includes coordinate information of the grid in the grid map and height information of the grid, and the height information includes one or more of the maximum height, minimum height and average height of the point cloud set within the grid.

7. The method according to claim 1, characterized in that Detecting obstacles in the surrounding environment of the vehicle according to the pixel value of each pixel point in the target layer includes: If the pixel value of the pixel point is greater than the pixel threshold, and the pixel value of the pixel point is greater than the pixel values ​​of each of the surrounding pixels, then there is a raised obstacle at the target position; If the pixel value of the pixel point is less than the pixel threshold, and the pixel value of the pixel point is less than the pixel values ​​of each of the surrounding pixel points, then there is a pothole obstacle at the target position.

8. The method according to claim 7, characterized in that Before detecting obstacles in the surrounding environment of the vehicle according to the pixel values ​​of each pixel point in the target layer, the method further includes: Delete a target pixel point in the target layer, where the pixel value of the target pixel point is equal to the pixel threshold.

9. The method according to claim 1, characterized in that The acquiring of a plurality of posture data of the vehicle and point cloud data corresponding to the posture data includes: Acquire multiple initial posture data of the vehicle and the point cloud data corresponding to the initial posture data; Matching the point cloud data with the local map, and obtaining point-surface observation residuals according to the matching results; Based on the point-surface observation residuals and the IMU prediction residuals, the initial posture is corrected to obtain a plurality of posture data.

10. A vehicle, characterized in that: The method comprises one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to implement the operations performed by the method according to any one of claims 1 to 9.

Images