A method, device, and vehicle-mounted terminal for determining dynamic obstacles based on point clouds.

Abstract

This application discloses a method, device, and vehicle-mounted terminal for determining dynamic obstacles based on point clouds. The method includes the following steps: acquiring vehicle-mounted point cloud data of the current frame and vehicle-mounted point cloud data of several key frames; the vehicle-mounted point cloud data of the key frames are vehicle-mounted point cloud data within a preset distance traveled by the vehicle before the current frame; determining dynamic points in the vehicle-mounted point cloud data of the current frame based on the vehicle-mounted point cloud data of the current frame and the vehicle-mounted point cloud data of several key frames; and clustering the dynamic points in the vehicle-mounted point cloud data of the current frame to determine dynamic obstacles. This application determines dynamic points in the vehicle-mounted point cloud data of the current frame based on the vehicle-mounted point cloud data of key frames, greatly reducing the number of point clouds in the vehicle-mounted point cloud data of the current frame. Then, it clusters the dynamic points in the vehicle-mounted point cloud data of the current frame to determine dynamic obstacles, which not only reduces the amount of computation but also has higher accuracy.

Description

Technical Field

[0001] This application relates to the field of positioning technology for autonomous driving, specifically to a method, device, and vehicle-mounted terminal for determining dynamic obstacles based on point clouds. Background Technology

[0002] In automated parking mapping and localization systems, dynamic obstacles significantly impact the accuracy of mapping and localization. Real-time filtering of dynamic obstacles during the mapping process is a crucial technology. In recent years, several dynamic obstacle filtering schemes have emerged, primarily employing a combination of clustering and filtering methods.

[0003] In existing technologies, point clouds in the current frame are clustered, and the characteristics of the point sets are analyzed to determine whether they are dynamic obstacles. Specifically, all point clouds are clustered, and the clustered points include all static and dynamic point sets. Then, dynamic objects are identified. This process is not only computationally intensive but also has a relatively high probability of misclassification. Summary of the Invention

[0004] This application provides a method, device, and vehicle-mounted terminal for determining dynamic obstacles based on point clouds, in order to solve the technical problems of large computational load and high probability of misjudgment when determining dynamic objects in related technologies.

[0005] The first aspect of this application provides a method for determining dynamic obstacles based on point clouds, comprising the following steps:

[0006] The vehicle point cloud data and pose of the current frame, as well as the vehicle point cloud data and pose of several key frames, are obtained; wherein, the vehicle point cloud data of the key frames are the vehicle point cloud data within a preset distance of vehicle movement before the current frame.

[0007] Based on the vehicle point cloud data and pose of the current frame and the vehicle point cloud data and pose of several key frames, determine the dynamic points in the vehicle point cloud data of the current frame.

[0008] Clustering is performed on the dynamic points in the vehicle point cloud data of the current frame to identify dynamic obstacles in the vehicle point cloud data of the current frame.

[0009] Based on the aforementioned technical methods, this application does not cluster all points in the vehicle point cloud data of the current frame, but only clusters dynamic points in the vehicle point cloud data of the current frame, greatly reducing the number of points in the clustered data. By clustering dynamic points, dynamic obstacles are identified, which not only greatly reduces the amount of computation but also improves accuracy, thus reducing false positives.

[0010] Optionally, the vehicle point cloud data of the key frame has at least two frames; determining the dynamic points in the vehicle point cloud data of the current frame based on the vehicle point cloud data and pose of the current frame and the vehicle point cloud data and pose of the key frame includes:

[0011] Based on the vehicle point cloud data and pose of the current frame and the vehicle point cloud data and pose of the key frame, determine the potential dynamic points of the vehicle point cloud data of the current frame.

[0012] Based on the vehicle point cloud data of the remaining key frames, the potential dynamic points in the vehicle point cloud data of the current frame are voted on to obtain the dynamic points in the vehicle point cloud data of the current frame.

[0013] Based on the above technical means, potential dynamic points are first determined based on the vehicle point cloud data and pose of the current frame and the vehicle point cloud data and pose of the key frame. Then, dynamic points are determined from the potential dynamic points, which can yield more accurate dynamic points.

[0014] Optionally, the acquisition time of the vehicle point cloud data of the remaining key frames is between the acquisition time of the vehicle point cloud data of the current frame and the acquisition time of the vehicle point cloud data of the key frames.

[0015] In the voting, the voting score is determined based on the nearest neighbor distance between the potential dynamic point and the vehicle point cloud data of the remaining key frames, and the potential dynamic points with a voting score greater than a preset score are taken as dynamic points in the vehicle point cloud data of the current frame.

[0016] Based on the above technical means, potential dynamic points are determined by comparing the vehicle point cloud data of the key frame with the vehicle point cloud data of the current frame, since the points corresponding to dynamic obstacles in these two vehicle point cloud data differ significantly, resulting in a larger number of potential dynamic points. This approach aims to encompass as many dynamic obstacles as possible and improve the accuracy of identifying dynamic obstacles.

[0017] Optionally, clustering the dynamic points in the vehicle point cloud data of the current frame to determine the dynamic obstacles in the vehicle point cloud data of the current frame includes:

[0018] Cluster the dynamic points in the vehicle point cloud data of the current frame to obtain several clustered point clouds;

[0019] Determine the features of the clustered point cloud; wherein the features of the clustered point cloud include at least one of the following: the number of points in the clustered point cloud, the coordinates of the center point of the clustered point cloud, the length of the bounding rectangle of the clustered point cloud, and the width of the bounding rectangle of the clustered point cloud.

[0020] Based on the characteristics of the clustered point cloud, dynamic obstacles in the vehicle point cloud data of the current frame are determined.

[0021] Based on the aforementioned technical means, further screening of the clustered point clouds can determine their characteristics. Then, based on these characteristics, the clustered point clouds corresponding to dynamic obstacles can be identified from a number of clustered point clouds, which can further improve the accuracy of identifying dynamic obstacles.

[0022] Optionally, determining the dynamic obstacles in the vehicle point cloud data of the current frame based on the features of the clustered point cloud includes:

[0023] When the height of the center point coordinates of the clustered point cloud is within a preset height range, the category of the clustered point cloud is configured as a dynamic obstacle;

[0024] When the number of points in the clustered point cloud exceeds the preset number of points, the category of the clustered point cloud is configured as a dynamic obstacle.

[0025] When the ratio of the length of the bounding rectangle of the clustered point cloud to the width of the bounding rectangle of the clustered point cloud is less than a preset ratio, the category of the clustered point cloud is configured as a dynamic obstacle.

[0026] Based on the aforementioned technical means, by analyzing the height of the center point coordinates of the clustered point cloud, the number of points in the clustered point cloud, and the aspect ratio of the bounding rectangle of the clustered point cloud, moving people, animals, other vehicles, and other obstacles can be screened as dynamic obstacles, thereby further improving the accuracy of identifying dynamic obstacles.

[0027] Optionally, acquiring the vehicle point cloud data of the current frame and the vehicle point cloud data of several key frames includes:

[0028] Acquire several frames of raw point cloud data and IMU data, and preprocess the raw point cloud data of the current frame in the raw point cloud data to obtain the vehicle point cloud data of the current frame; wherein, the preprocessing includes at least one of the following: voxel filtering the vehicle point cloud data of the current frame, deleting noise in the vehicle point cloud data of the current frame, and deleting ground points in the vehicle point cloud data of the current frame.

[0029] Based on the raw point cloud data of the current frame, vehicle point cloud data of several key frames are determined from the raw point cloud data of several frames.

[0030] Determine the pose of each keyframe;

[0031] The pose of the current frame is determined based on the pose of the previous keyframe and the IMU data of the current frame.

[0032] Based on the above technical means, removing noise and ground points from the vehicle point cloud data of the current frame and performing voxel filtering can greatly reduce the number of points in the vehicle point cloud data of the current frame, reduce the amount of computation, and improve the accuracy of determining dynamic obstacles.

[0033] Optionally, after clustering the dynamic points in the vehicle point cloud data of the current frame to determine the dynamic obstacles in the vehicle point cloud data of the current frame, the determination method further includes:

[0034] Delete the points corresponding to dynamic obstacles in the vehicle point cloud data of the current frame.

[0035] Based on the above technical means, during the mapping and localization process of automatic parking, the points corresponding to dynamic obstacles in the vehicle point cloud data of the current frame are deleted, thereby facilitating the mapping and localization of automatic parking and avoiding the impact of dynamic obstacles on mapping and localization accuracy.

[0036] A second aspect of this application provides a device for determining dynamic obstacles, comprising:

[0037] The acquisition module is used to acquire the vehicle point cloud data and pose of the current frame and the vehicle point cloud data and pose of several key frames; wherein, the vehicle point cloud data of the key frames is the vehicle point cloud data within a preset distance of vehicle movement before the current frame.

[0038] The dynamic point module is used to determine the dynamic points in the vehicle point cloud data of the current frame based on the vehicle point cloud data and pose of the current frame and the vehicle point cloud data and pose of the key frame.

[0039] The determination module is used to cluster dynamic points in the vehicle point cloud data of the current frame to determine dynamic obstacles in the vehicle point cloud data of the current frame.

[0040] Optionally, the vehicle point cloud data of the key frame has at least two frames; the dynamic point module is specifically used to determine the potential dynamic points of the vehicle point cloud data of the current frame based on the vehicle point cloud data and pose of the current frame and the vehicle point cloud data and pose of the key frame; and to vote on the potential dynamic points in the vehicle point cloud data of the current frame based on the vehicle point cloud data of the remaining key frames to obtain the dynamic points in the vehicle point cloud data of the current frame.

[0041] Optionally, the acquisition time of the vehicle point cloud data of the remaining key frames is between the acquisition time of the vehicle point cloud data of the current frame and the acquisition time of the vehicle point cloud data of the key frames; in the voting, the voting score is determined based on the nearest neighbor distance between the potential dynamic point and the vehicle point cloud data of the remaining key frames, and the potential dynamic point with a voting score greater than a preset score is taken as the dynamic point in the vehicle point cloud data of the current frame.

[0042] Optionally, the determining module is specifically used to cluster the dynamic points in the vehicle point cloud data of the current frame to obtain several clustered point clouds; determine the features of the clustered point clouds; wherein the features of the clustered point clouds include at least one of the following: the number of points in the clustered point clouds, the coordinates of the center point of the clustered point clouds, the length of the bounding rectangle of the clustered point clouds, and the width of the bounding rectangle of the clustered point clouds; and determine the dynamic obstacles in the vehicle point cloud data of the current frame based on the features of the clustered point clouds.

[0043] Optionally, the determining module is further configured to: classify the clustered point cloud as a dynamic obstacle when the height of the center point coordinates of the clustered point cloud is within a preset height range; classify the clustered point cloud as a dynamic obstacle when the number of points in the clustered point cloud is greater than a preset number of points; and classify the clustered point cloud as a dynamic obstacle when the ratio of the length of the bounding rectangle of the clustered point cloud to the width of the bounding rectangle of the clustered point cloud is less than a preset ratio.

[0044] Optionally, the acquisition module is specifically used to acquire several frames of raw point cloud data and IMU data, and preprocess the raw point cloud data of the current frame in the raw point cloud data to obtain the vehicle-mounted point cloud data of the current frame; wherein, the preprocessing includes at least one of the following: performing voxel filtering on the vehicle-mounted point cloud data of the current frame, deleting noise from the vehicle-mounted point cloud data of the current frame, and deleting ground points from the vehicle-mounted point cloud data of the current frame; determining vehicle-mounted point cloud data of several key frames from the raw point cloud data of several frames based on the raw point cloud data of the current frame; determining the pose of each key frame; and determining the pose of the current frame based on the pose of the previous key frame and the IMU data of the current frame.

[0045] Optionally, the device for determining dynamic obstacles further includes:

[0046] The deletion module is used to delete points corresponding to dynamic obstacles in the vehicle point cloud data of the current frame.

[0047] A third aspect of this application provides a vehicle-mounted terminal, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the program to implement the point cloud-based dynamic obstacle determination method as described in the above embodiments.

[0048] A fourth aspect of this application provides a vehicle, including: a dynamic obstacle determination device as described in the above embodiments or an in-vehicle terminal as described in the above embodiments.

[0049] The beneficial effects of this application are as follows: Based on the vehicle point cloud data of the key frame, this application determines the dynamic points in the vehicle point cloud data of the current frame, which greatly reduces the number of points in the vehicle point cloud data of the current frame. Then, the dynamic points in the vehicle point cloud data of the current frame are clustered to determine the dynamic obstacles. This not only reduces the amount of computation but also increases the accuracy. Attached Figure Description

[0050] Figure 1 This is a flowchart of a method for determining dynamic obstacles based on point clouds according to an embodiment of this application;

[0051] Figure 2 This is a schematic diagram of the vehicle point cloud data of the current frame and the vehicle point cloud data of the key frame in a specific embodiment of this application;

[0052] Figure 3 This is a schematic diagram of scanning by a laser sensor according to a specific embodiment of this application.

[0053] Wherein, 1-laser sensor; 2-point; 3-point; α-angle. Detailed Implementation

[0054] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0055] The following describes a point cloud-based method for determining dynamic obstacles according to embodiments of this application, with reference to the accompanying drawings. In the related technologies mentioned in the background section, when mapping and locating automatic parking, all point clouds are clustered. The clustered points include all static and dynamic point sets, and then dynamic objects are identified. This results in high computational complexity and a relatively high probability of misjudgment. This application provides a point cloud-based method for determining dynamic obstacles. Based on the vehicle-mounted point cloud data of keyframes, it can determine the dynamic points in the current frame's vehicle-mounted point cloud data, significantly reducing the number of point clouds in the current frame. Then, it clusters the dynamic points in the current frame's vehicle-mounted point cloud data to determine dynamic obstacles. This method not only reduces computational complexity but also improves accuracy. Therefore, it solves the technical problems of high computational complexity and a high probability of misjudgment when determining dynamic objects in related technologies.

[0056] Please also refer to Figures 1-3 This application provides some embodiments of a method for determining dynamic obstacles based on point clouds.

[0057] This application presents a point cloud-based method for determining dynamic obstacles, applied to vehicles equipped with several laser sensors, such as LiDAR, to collect raw point cloud data. These laser sensors can be 3D laser sensors. The vehicle can be in motion or parked. The environment in which the vehicle operates contains both static and dynamic obstacles. Static obstacles are objects that are stationary and that the vehicle must avoid, while dynamic obstacles are objects that are moving and that the vehicle must avoid. Therefore, objects that may move, such as people, animals, and other vehicles, can be considered either dynamic or static obstacles. Fixed buildings and equipment, which are not moving, are typically considered static obstacles. Of course, road surfaces, such as the ground or bridge surfaces, are for vehicle traffic and do not require avoidance; therefore, they are not considered static or dynamic obstacles but are classified as non-obstacles. Objects in the sky (or ceiling) and objects on the sky (or ceiling), which the vehicle does not need to avoid, are also considered non-obstacles. In short, the raw point cloud data contains at least some dynamic obstacles. This application aims to identify these dynamic obstacles in order to delete the corresponding point cloud data, thereby improving the accuracy of mapping and localization. The raw point cloud data may also contain static obstacles and non-obstacles.

[0058] like Figure 1 As shown, the method for determining dynamic obstacles based on point clouds in this application includes the following steps:

[0059] Step S100: Obtain the vehicle point cloud data and pose of the current frame, as well as the vehicle point cloud data and pose of several key frames; wherein, the vehicle point cloud data of the key frames is the vehicle point cloud data within the preset distance of the vehicle movement before the current frame.

[0060] Step S200: Based on the vehicle point cloud data and pose of the current frame and the vehicle point cloud data and pose of several key frames, determine the dynamic points in the vehicle point cloud data of the current frame.

[0061] Step S300: Cluster the dynamic points in the vehicle point cloud data of the current frame to determine the dynamic obstacles in the vehicle point cloud data of the current frame.

[0062] Specifically, vehicle-mounted point cloud data refers to point cloud data collected by vehicle-mounted laser sensors. Based on different collection timestamps, it is divided into several frames. The vehicle-mounted point cloud data collected at the current time is used as the current frame's point cloud data. Keyframes are determined from the vehicle-mounted point cloud data collected before the current time. Not all vehicle-mounted point cloud data collected before the current time can necessarily be used as keyframes. A portion of the vehicle-mounted point cloud data collected before the current time can be selected as keyframes. For example, keyframes can be determined based on the vehicle's travel distance, specifically from point cloud data collected within a preset travel distance. The interval between two adjacent keyframes can be either a certain vehicle travel distance or a certain time interval. For example, with a preset distance of 0.5m, several frames of vehicle point cloud data can be collected within 0.5m of the vehicle's movement. The vehicle point cloud data collected every 10cm of vehicle movement is used as the vehicle point cloud data for keyframes. Figure 2 As shown, the dashed rectangle represents a sliding window, which stores the vehicle point cloud data for all keyframes. There are 5 keyframes of vehicle point cloud data. The vehicle point cloud data for keyframes is determined according to the distance traveled. The vehicle can maintain its movement, saving time. Furthermore, the moving vehicle can alert the driver to nearby static obstacles that can be moved but have not yet moved. These static obstacles may move and transform into dynamic obstacles, facilitating vehicle movement and parking.

[0063] Based on the vehicle-mounted point cloud data of the current frame and the keyframes, dynamic points in the current frame's vehicle-mounted point cloud data can be identified. Dynamic points are those points in the vehicle-mounted point cloud data whose positions have changed. The system uses points from other vehicle-mounted point cloud data to determine if a point in the current frame has changed position; a point with a significant positional change is considered a dynamic point. Then, clustering is performed on the dynamic points in the current frame's vehicle-mounted point cloud data. This means that instead of clustering all points in the current frame's vehicle-mounted point cloud data, only the dynamic points are clustered, significantly reducing the number of points in the cluster. By clustering the dynamic points, dynamic obstacles are identified, greatly reducing computational load, improving accuracy, and reducing false positives.

[0064] Step S100 specifically includes:

[0065] Step S110: Acquire several frames of raw point cloud data and IMU data, and preprocess the raw point cloud data of the current frame in the raw point cloud data to obtain the vehicle point cloud data of the current frame; wherein, the preprocessing includes at least one of the following: voxel filtering of the vehicle point cloud data of the current frame, deleting noise in the vehicle point cloud data of the current frame, and deleting ground points in the vehicle point cloud data of the current frame.

[0066] Step S120: Based on the original point cloud data of the current frame, determine the vehicle-mounted point cloud data of several key frames from the original point cloud data of several frames.

[0067] Step S130: Determine the pose of each keyframe.

[0068] Step S140: Determine the pose of the current frame based on the pose of the previous keyframe and the IMU data of the current frame.

[0069] Specifically, raw point cloud data refers to the unprocessed point cloud data collected by the vehicle-mounted laser sensor. Raw point cloud data contains a large number of points and requires preprocessing to reduce the number of points. Preprocessing can involve voxel filtering of the current frame's vehicle-mounted point cloud data, using voxelization to reduce the number of points. For example, points near the center of a voxel grid can be used to replace all points within that voxel grid. After voxel filtering, the number of points in the current frame's vehicle-mounted point cloud data is significantly reduced, but the shape characteristics of the points remain unchanged. Another preprocessing step is to remove noise from the current frame's vehicle-mounted point cloud data. For each point in the current frame's vehicle-mounted point cloud data, the average distance between that point and several surrounding points is determined. If this average distance is greater than a preset average distance, the point is removed as noise; if the average distance is less than or equal to the preset average distance, the point is retained. Since this application collects vehicle-mounted point cloud data, primarily focusing on point cloud data collection of objects on the ground where the vehicle travels, the vehicle-mounted point cloud data contains ground points—points on the ground where the vehicle travels. As the ground is a non-obstacle, these ground points can be deleted from the vehicle-mounted point cloud data of the current frame. Specifically, the following steps are used to delete ground points from the vehicle-mounted point cloud data of the current frame:

[0070] A100: Map the vehicle point cloud data of the current frame into a depth image.

[0071] A200, determine the first pixel and the second pixel of the depth image; wherein, the first pixel and the second pixel are both located below the horizon of the depth image, and the first pixel and the second pixel are the pixels at the corresponding positions of two adjacent scans of the vehicle-mounted laser sensor.

[0072] A300: When the pixel value of the first pixel and the pixel value of the second pixel meet the preset conditions, the point corresponding to the first pixel and the point corresponding to the second pixel in the vehicle point cloud data of the current frame are both taken as undetermined ground points, and the height of the undetermined ground points is determined.

[0073] A400. Determine the ground height based on the heights corresponding to all undetermined ground points.

[0074] A500: Based on ground height, determine ground points from the undetermined ground points and delete ground points.

[0075] Specifically, the vehicle point cloud data of the current frame is mapped to a depth image. Different pixel values ​​in the depth image correspond to different distances between points in the vehicle point cloud data of the current frame. For example, in the horizontal direction, the farther a point is from the vehicle's laser sensor, the smaller the pixel value of the corresponding pixel in the depth image. Alternatively, the farther a point is from the vehicle's laser sensor, the larger the pixel value of the corresponding pixel in the depth image. Figure 3As shown, during the scanning process, laser sensor 1 first scans horizontally to obtain one row of points 2 (specifically represented by three-dimensional coordinates), completing one scan. Then, it performs the next scan to obtain the next row of points 3. After several scans, several rows of points are obtained, forming a matrix of points, i.e., vehicle-mounted point cloud data. Alternatively, multi-line scanning can be used, scanning each row simultaneously to obtain several rows of points, forming a matrix of points, i.e., vehicle-mounted point cloud data. For example, if the vehicle-mounted point cloud data has m×n points, forming an m×n matrix, where m represents the number of scans (rows) and n represents the number of points in each scan (columns). The first pixel and the second pixel belong to two adjacent scans, and their positions correspond, i.e., they are located in the same column of the matrix. If the vehicle-mounted laser sensor scans the ground, the pixel values ​​of the corresponding pixels in the same column of the matrix should gradually change (specifically, they can change linearly). If other objects exist on the ground, the pixel values ​​of the corresponding pixels in the same column of the matrix will not change significantly. Therefore, based on the first pixel and the second pixel, it can be determined whether the point corresponding to the first pixel and the point corresponding to the second pixel are located on the same plane. If the point corresponding to the first pixel and the point corresponding to the second pixel are located on the same plane (i.e., the angle α between the line connecting point 2 and point 3 corresponding to the first pixel and the ground is small), then these two points can be considered as undetermined ground points. Of course, in addition to points on the ground being located on the same plane, some points on the surface of objects are also located on the same surface, so it is also necessary to determine the height of the undetermined ground points. Although some points on the surface of objects will also be considered as undetermined ground points, the height of these points on the surface of objects is relatively small. Here, height refers to the height of the point from the vehicle-mounted laser sensor in the vertical direction, and the number of point clouds on the surface of objects is also relatively small. Therefore, the heights of all undetermined ground points are counted, and the height with the largest proportion is taken as the ground height. Then, based on the ground height, if the difference between the height of the undetermined ground point and the ground height is less than a preset height threshold, then the undetermined ground point is deleted as a ground point; if the difference between the height of the undetermined ground point and the ground height is greater than the preset height threshold, then the undetermined ground point is retained. Removing ground points from the vehicle point cloud data of the current frame can further reduce the number of points in the vehicle point cloud data of the current frame, reduce the amount of computation, and eliminate the ground as a non-obstacle, thereby improving the accuracy of identifying dynamic obstacles.

[0076] To determine whether the pixel values ​​of the first and second pixels meet preset conditions, the angle α between the line connecting point 2 (corresponding to the first pixel) and point 3 (corresponding to the second pixel) and the ground can be calculated based on the pixel values ​​of the first and second pixels, the location of the first pixel, and the location of the second pixel. When the angle α is less than a preset angle, both points are considered ground points. In the m×n matrix formed by the vehicle-mounted point cloud data, adjacent points in each column may not be perfectly aligned. Therefore, the vehicle-mounted point cloud data is mapped to a depth image, where adjacent pixels in each column are aligned, to more conveniently and accurately determine ground points.

[0077] The poses of each keyframe are directly obtained from the SLAM system or positioning system. IMU data refers to data detected by the Inertial Measurement Unit (IMU). IMU data is collected at a high frequency. The IMU integrated pose is obtained by integrating the IMU data over the time interval between the timestamps of two frames (the previous keyframe and the current frame). The pose of the current frame is then derived from the pose of the previous keyframe and the IMU integrated pose. The pose of the next frame is:

[0078] T i+1 =T i *T i,i+1

[0079] Among them, T i+1 T represents the pose of the current frame. i T represents the pose of the previous keyframe of the current frame. i,i+1 This represents the IMU integral pose. The IMU integral pose is obtained by integrating the IMU data between the current frame and the previous keyframe. The pose of the current frame is determined based on the pose of the previous keyframe and the IMU integral pose. Therefore, the poses of each keyframe and the pose of the current frame can be obtained.

[0080] The keyframes of the vehicle point cloud data consist of at least two frames. Step S200 specifically includes:

[0081] Step S210: Based on the vehicle point cloud data and pose of the current frame and the vehicle point cloud data and pose of the key frame, determine the potential dynamic points of the vehicle point cloud data of the current frame.

[0082] Step S220: Based on the vehicle point cloud data of the remaining key frames, vote on the potential dynamic points in the vehicle point cloud data of the current frame to obtain the dynamic points in the vehicle point cloud data of the current frame.

[0083] Specifically, determining the pose of the current frame and the keyframes facilitates the alignment and comparison of the vehicle point cloud data of the current frame and the keyframes, thereby determining dynamic points. The vehicle point cloud data of a keyframe can consist of one or more frames. If there is only one frame of vehicle point cloud data for a keyframe, the dynamic points of the current frame's vehicle point cloud data are determined based on the vehicle point cloud data of the current frame and the keyframes. If there are two or more frames of vehicle point cloud data for a keyframe, the potential dynamic points of the current frame's vehicle point cloud data are first determined based on the vehicle point cloud data of one keyframe and the current frame's vehicle point cloud data. In the vehicle point cloud data of the keyframe, the position of the corresponding static obstacle point is the same as or close to the position of the corresponding static obstacle point in the vehicle point cloud data of the current frame. Similarly, the position of the corresponding non-obstacle point in the vehicle point cloud data of the keyframe is the same as or close to the position of the corresponding non-obstacle point in the vehicle point cloud data of the current frame. However, the position of the corresponding dynamic obstacle point in the vehicle point cloud data of the keyframe is different from the position of the corresponding dynamic obstacle point in the vehicle point cloud data of the current frame. The nearest neighbor distance between a point in the vehicle point cloud data of the current frame and the vehicle point cloud data of the keyframe is calculated. If the nearest neighbor distance is greater than a first preset threshold, the point in the vehicle point cloud data of the current frame is considered a potential dynamic point. If the nearest neighbor distance is less than or equal to the first preset threshold, the point in the vehicle point cloud data of the current frame is not considered a potential dynamic point.

[0084] In the voting process, the voting score is determined based on the nearest neighbor distance between the potential dynamic point and the vehicle point cloud data of the remaining keyframes. Potential dynamic points with voting scores greater than a preset score are designated as dynamic points in the vehicle point cloud data of the current frame. For each potential dynamic point, the nearest neighbor distance between that point and the vehicle point cloud data of the remaining keyframes is calculated, and all potential dynamic points participate in the voting. A larger nearest neighbor distance between a dynamic point and the vehicle point cloud data of the remaining keyframes results in a larger voting score; conversely, a smaller nearest neighbor distance results in a smaller voting score. When a potential dynamic point's voting score is greater than the preset score, it is designated as a dynamic point; when its voting score is less than or equal to the preset score, it is not designated as a dynamic point, thus obtaining more accurate dynamic points.

[0085] like Figure 2As shown, the solid squares within the sliding window represent the vehicle point cloud data of the keyframes, while the dashed squares represent the vehicle point cloud data of the remaining keyframes. The solid squares outside the sliding window represent the vehicle point cloud data of the current frame. In terms of time, the acquisition time of the vehicle point cloud data for the remaining keyframes falls between the acquisition time of the vehicle point cloud data for the current frame and the acquisition time of the keyframes. When determining potential dynamic points, potential dynamic points are identified based on the vehicle point cloud data of the keyframe furthest apart in time and the vehicle point cloud data of the current frame. Since the points corresponding to dynamic obstacles in these two sets of vehicle point cloud data differ significantly, the number of potential dynamic points is greater, allowing for the inclusion of as many dynamic obstacles as possible and improving the accuracy of dynamic obstacle identification.

[0086] Step S300 specifically includes:

[0087] Step S310: Cluster the dynamic points in the vehicle point cloud data of the current frame to obtain several clustered point clouds.

[0088] Step S320: Determine the characteristics of the clustered point cloud; wherein, the characteristics of the clustered point cloud include at least one of the following: the number of points in the clustered point cloud, the coordinates of the center point of the clustered point cloud, the length of the bounding rectangle of the clustered point cloud, and the width of the bounding rectangle of the clustered point cloud.

[0089] Step S330: Based on the characteristics of the clustered point cloud, determine the dynamic obstacles in the vehicle point cloud data of the current frame.

[0090] Specifically, dynamic points are clustered to obtain several clustered point clouds. These clustered point clouds are then further filtered to determine their characteristics. Based on these characteristics, the clustered point cloud corresponding to the dynamic obstacle is identified from these clustered point clouds. In other words, the category of these clustered point clouds is dynamic obstacle, while the category of other clustered point clouds is not dynamic obstacle. Therefore, the accuracy of identifying dynamic obstacles can be further improved. When the characteristics of a clustered point cloud meet preset requirements, that clustered point cloud is designated as the clustered point cloud corresponding to the dynamic obstacle, i.e., the category of that clustered point cloud is configured as dynamic obstacle.

[0091] During clustering, the distance between one dynamic point and another in the vehicle point cloud data of the current frame is calculated. If the distance is less than or equal to a second preset threshold, the two dynamic points are assigned to the same cluster point cloud; if the distance is greater than the second preset threshold, the two dynamic points are assigned to different cluster point clouds. Clustering methods such as K-means clustering can be used.

[0092] Step S330 specifically includes:

[0093] Step S331: When the height of the center point coordinates of the clustered point cloud is within the preset height range, configure the category of the clustered point cloud as a dynamic obstacle.

[0094] Step S332: When the number of points in the clustered point cloud is greater than the preset number of points, configure the category of the clustered point cloud as a dynamic obstacle.

[0095] Step S333: When the ratio of the length of the bounding rectangle of the clustered point cloud to the width of the bounding rectangle of the clustered point cloud is less than a preset ratio, the category of the clustered point cloud is configured as a dynamic obstacle.

[0096] Specifically, due to potential biases in determining dynamic points, some dynamic points may be improperly clustered during the clustering process. Therefore, the clustered point clouds need to be filtered to identify the point clouds containing dynamic obstacles. Since vehicle-mounted point cloud data is obtained by scanning the vehicle's environment, it contains a horizon line. Above the horizon is the sky or ceiling, and below the horizon are the ground and objects on the ground. The center point coordinates of the clustered point clouds corresponding to dynamic obstacles such as people, animals, and other vehicles should be close to the horizon, neither too high nor too low. In other words, if the center point coordinates of a clustered point cloud are not within the preset height range, the clustered point cloud will not be classified as a dynamic obstacle; only when the center point coordinates are within the preset height range will the clustered point cloud be classified as a dynamic obstacle. If the number of point clouds corresponding to dynamic obstacles such as people, animals, and other vehicles is too large or too small, these clustered point clouds will not be used as the clustered point clouds corresponding to dynamic obstacles. Therefore, if the number of points in a clustered point cloud is less than or equal to the preset number, the clustered point cloud will not be classified as a dynamic obstacle; only when the number of points in a clustered point cloud is greater than the preset number will the clustered point cloud be classified as a dynamic obstacle. The aspect ratio of the bounding rectangle of the clustered point cloud corresponding to dynamic obstacles such as people, animals, and other vehicles is within a certain range and will not be excessively large. Therefore, for clustered point clouds with excessively large aspect ratios—for example, if the aspect ratio of the bounding rectangle of a clustered point cloud is greater than or equal to the preset ratio—the clustered point cloud will not be classified as a dynamic obstacle; if the aspect ratio of the bounding rectangle of a clustered point cloud is less than the preset ratio, the clustered point cloud will be classified as a dynamic obstacle. By analyzing the height of the center point coordinates of the clustered point cloud, the number of points in the clustered point cloud, and the aspect ratio of the bounding rectangle of the clustered point cloud, moving obstacles such as people, animals, and other vehicles are filtered out as dynamic obstacles, further improving the accuracy of identifying dynamic obstacles.

[0097] The determination method also includes:

[0098] Step S400: Delete the points corresponding to dynamic obstacles in the vehicle point cloud data of the current frame.

[0099] Specifically, dynamic obstacles in the vehicle point cloud data of the current frame may move away during vehicle movement and will not hinder the vehicle's movement. Therefore, in the mapping and localization process of automatic parking, it is necessary to delete the points corresponding to dynamic obstacles in the vehicle point cloud data of the current frame to facilitate mapping and localization of automatic parking and avoid dynamic obstacles affecting the accuracy of mapping and localization.

[0100] Furthermore, based on the point cloud-based dynamic obstacle determination method of any of the above embodiments, embodiments of this application also propose an embodiment of a dynamic obstacle determination device, wherein the vehicle includes:

[0101] The acquisition module is used to acquire the vehicle point cloud data and pose of the current frame, as well as the vehicle point cloud data and pose of several key frames; wherein, the vehicle point cloud data of the key frames is the vehicle point cloud data within a preset distance of vehicle movement before the current frame.

[0102] The dynamic point module is used to determine the dynamic points in the vehicle point cloud data of the current frame based on the vehicle point cloud data and pose of the current frame and the vehicle point cloud data and pose of the key frame.

[0103] The determination module is used to cluster dynamic points in the vehicle point cloud data of the current frame in order to determine dynamic obstacles in the vehicle point cloud data of the current frame.

[0104] The device for determining dynamic obstacles also includes:

[0105] The deletion module is used to delete points corresponding to dynamic obstacles in the vehicle point cloud data of the current frame.

[0106] Based on the point cloud-based dynamic obstacle determination method of any of the above embodiments, embodiments of this application also provide a vehicle-mounted terminal. The vehicle-mounted terminal may include:

[0107] Memory, processor, and computer programs stored in memory and capable of running on the processor.

[0108] When the processor executes the program, it implements the point cloud-based method for determining dynamic obstacles provided in the above embodiments.

[0109] Furthermore, the vehicle-mounted terminal also includes:

[0110] A communication interface used for communication between the memory and the processor.

[0111] The memory may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage device.

[0112] If the memory, processor, and communication interface are implemented independently, they can be interconnected via a bus to communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc.

[0113] Specifically, if the memory, processor, and communication interface are integrated on a single chip, then the memory, processor, and communication interface can communicate with each other through internal interfaces.

[0114] The processor may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.

[0115] This application also provides a vehicle, which includes: a dynamic obstacle determination device as described in the above embodiments or an on-board terminal as described in the above embodiments.

[0116] In the description of this specification, the references to terms such as "embodiment," "any embodiment," or "implementation" indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or implementation is included in at least one embodiment or implementation of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or implementation. Furthermore, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or implementations. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or implementations described in this specification, as well as the features of different embodiments or implementations.

[0117] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0118] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0119] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

Claims

1. A method for determining dynamic obstacles based on point clouds, characterized in that, The method comprises the following steps: obtaining vehicle-mounted point cloud data and pose of a current frame and vehicle-mounted point cloud data and pose of several key frames; wherein the vehicle-mounted point cloud data of the key frames is vehicle-mounted point cloud data within a preset distance of movement of the vehicle before the current frame; determining dynamic points in the vehicle-mounted point cloud data of the current frame according to the vehicle-mounted point cloud data and pose of the current frame and the vehicle-mounted point cloud data and pose of the several key frames; clustering the dynamic points in the vehicle-mounted point cloud data of the current frame to determine dynamic obstacles in the vehicle-mounted point cloud data of the current frame; the vehicle-mounted point cloud data of the key frames has at least two frames; the determination of the dynamic points in the vehicle-mounted point cloud data of the current frame according to the vehicle-mounted point cloud data and pose of the current frame and the vehicle-mounted point cloud data and pose of the key frames comprises: determining potential dynamic points in the vehicle-mounted point cloud data of the current frame according to the vehicle-mounted point cloud data and pose of the current frame and the vehicle-mounted point cloud data and pose of the key frames; voting the potential dynamic points in the vehicle-mounted point cloud data of the current frame based on the vehicle-mounted point cloud data of the remaining key frames to obtain the dynamic points in the vehicle-mounted point cloud data of the current frame; the collection time of the vehicle-mounted point cloud data of the remaining key frames is between the collection time of the vehicle-mounted point cloud data of the current frame and the collection time of the vehicle-mounted point cloud data of the key frames; in the voting, the voting score is determined according to the nearest neighbor distance between the potential dynamic points and the vehicle-mounted point cloud data of the remaining key frames, and the potential dynamic points with a voting score greater than a preset score are taken as the dynamic points in the vehicle-mounted point cloud data of the current frame.

2. The method of claim 1, wherein, the clustering of the dynamic points in the vehicle-mounted point cloud data of the current frame to determine the dynamic obstacles in the vehicle-mounted point cloud data of the current frame comprises: clustering the dynamic points in the vehicle-mounted point cloud data of the current frame to obtain several clustered point clouds; determining the features of the clustered point clouds; wherein the features of the clustered point clouds comprise at least one of the following: the point cloud quantity of the clustered point cloud, the center point coordinates of the clustered point cloud, the length of the circumscribed rectangle of the clustered point cloud, and the width of the circumscribed rectangle of the clustered point cloud; determining the dynamic obstacles in the vehicle-mounted point cloud data of the current frame according to the features of the clustered point clouds.

3. The method of claim 2, wherein, the determination of the dynamic obstacles in the vehicle-mounted point cloud data of the current frame according to the features of the clustered point clouds comprises: when the height of the center point coordinates of the clustered point cloud is within a preset height range, the category of the clustered point cloud is configured as a dynamic obstacle; when the point cloud quantity of the clustered point cloud is greater than a preset point cloud quantity, the category of the clustered point cloud is configured as a dynamic obstacle; when the ratio of the length of the circumscribed rectangle of the clustered point cloud and the width of the circumscribed rectangle of the clustered point cloud is less than a preset ratio, the category of the clustered point cloud is configured as a dynamic obstacle.

4. The method of claim 1, wherein, the obtaining of the vehicle-mounted point cloud data and pose of the current frame and the vehicle-mounted point cloud data and pose of the several key frames comprises: Obtain a plurality of frames of original point cloud data and IMU data, and preprocess the original point cloud data of a current frame to obtain vehicle-mounted point cloud data of the current frame; wherein the preprocessing includes at least one of the following: voxel filtering of the vehicle-mounted point cloud data of the current frame, deleting noise in the vehicle-mounted point cloud data of the current frame, and deleting ground points in the vehicle-mounted point cloud data of the current frame; Based on the original point cloud data of the current frame, determine vehicle-mounted point cloud data of a plurality of key frames from the plurality of frames of original point cloud data; Determine the pose of each key frame; Determine the pose of the current frame based on the pose of the previous key frame of the current frame and the IMU data of the current frame.

5. The method of claim 1-4, wherein, After the dynamic points in the vehicle-mounted point cloud data of the current frame are clustered to determine the dynamic obstacles in the vehicle-mounted point cloud data of the current frame, the determination method further comprises: Delete the points corresponding to the dynamic obstacles in the vehicle-mounted point cloud data of the current frame.

6. A dynamic obstacle determination apparatus characterized by comprising: Comprise: An acquisition module for acquiring vehicle-mounted point cloud data and pose of a current frame and vehicle-mounted point cloud data and pose of a plurality of key frames; wherein the vehicle-mounted point cloud data of the key frame is the vehicle-mounted point cloud data within a preset distance of the vehicle moving before the current frame; A dynamic point module for determining dynamic points in the vehicle-mounted point cloud data of the current frame based on the vehicle-mounted point cloud data and pose of the current frame and the vehicle-mounted point cloud data and pose of the key frame; A determination module for clustering the dynamic points in the vehicle-mounted point cloud data of the current frame to determine the dynamic obstacles in the vehicle-mounted point cloud data of the current frame; The vehicle-mounted point cloud data of the key frame has at least two frames; the dynamic point module is specifically configured to determine potential dynamic points in the vehicle-mounted point cloud data of the current frame based on the vehicle-mounted point cloud data and pose of the current frame and the vehicle-mounted point cloud data and pose of the key frame; vote the potential dynamic points in the vehicle-mounted point cloud data of the current frame based on the vehicle-mounted point cloud data of the remaining key frames to obtain the dynamic points in the vehicle-mounted point cloud data of the current frame; the acquisition time of the vehicle-mounted point cloud data of the remaining key frames is between the acquisition time of the vehicle-mounted point cloud data of the current frame and the acquisition time of the vehicle-mounted point cloud data of the key frame; in the voting, the voting score is determined according to the nearest neighbor distance between the potential dynamic points and the vehicle-mounted point cloud data of the remaining key frames, and the potential dynamic points with a voting score greater than a preset score are taken as the dynamic points in the vehicle-mounted point cloud data of the current frame.

7. A vehicle terminal, characterized by Comprise: A memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the determination method of the dynamic obstacle based on the point cloud according to any one of claims 1-5.

8. A vehicle characterized by comprising: Comprise: The dynamic obstacle determination device according to claim 6 or the vehicle-mounted terminal according to claim 7.

Images