Point cloud target detection method and device, and electronic device

Abstract

The application discloses a point cloud target detection method and device and electronic equipment. The method comprises the following steps: acquiring laser 3D point cloud data to be processed; mapping the laser 3D point cloud data to a distance image; filtering out ground points according to the distance image to obtain object points; and obtaining a target segmentation result according to the object points. The application can filter out the ground points and achieve the effect of target segmentation, improve the detection accuracy, and ensure good real-time performance.

Description

Technical Field

[0001] This application relates to the fields of autonomous driving and positioning perception technology, and in particular to a point cloud target detection method, device, and electronic device. Background Technology

[0002] In the field of autonomous driving, environmental perception is extremely important. The quality of perception directly affects subsequent decision-making and planning, and thus the overall control performance of the autonomous driving system. Therefore, target detection, which is part of environmental perception, is particularly crucial.

[0003] Object detection can generally be divided into two main categories: one is based on deep learning neural networks, and the other is based on traditional object clustering methods. Considering hardware performance, development cycle, and data costs, the industry currently uses traditional methods more extensively. Meanwhile, LiDAR, due to its superior detection performance, has become an indispensable perception sensor in the field of autonomous driving.

[0004] Furthermore, target detection technologies based on LiDAR can be divided into two categories: one is feature extraction schemes based on deep learning, such as PointPillar and VoxelNet. The other is based on traditional point cloud processing algorithms, specifically including ground segmentation and target segmentation. Common methods for ground segmentation include planar grid methods, point cloud normal vector methods, and model fitting methods, while target segmentation often employs edge-based, attribute-based, and model-based segmentation methods.

[0005] The target detection technology has the following problems:

[0006] (a) Deep learning-based approaches have high requirements for hardware performance, and the development cycle and data costs are relatively large.

[0007] (b) Traditional raster-based point cloud segmentation methods have the problem of incomplete ground filtering, and ground points can interfere with the object point set located on the ground, which in turn leads to inaccurate target segmentation and affects the final target detection effect. Summary of the Invention

[0008] This application provides a point cloud target detection method, apparatus, and electronic device to better achieve ground filtering and target segmentation, while ensuring good real-time performance.

[0009] The embodiments of this application adopt the following technical solutions:

[0010] In a first aspect, embodiments of this application provide a point cloud target detection method, wherein the method includes:

[0011] Acquire the laser 3D point cloud data to be processed;

[0012] Map the laser 3D point cloud data to a distance image;

[0013] Based on the distance image, ground points are filtered out to obtain object points;

[0014] Based on the object points, the target segmentation result is obtained.

[0015] In some embodiments, mapping the laser 3D point cloud data to a distance image includes:

[0016] A distance image is constructed based on the field of view (FOV) and resolution of the laser sensor, wherein the number of rows in the distance image is the ratio of the vertical field of view to the vertical resolution, and the number of columns in the distance image is the ratio of the horizontal field of view to the horizontal resolution.

[0017] Calculate the number of rows and columns of the distance image corresponding to each object point in the laser 3D point cloud data, and store the number of rows and columns of the distance image corresponding to each point as an index;

[0018] The distance value from the object point in the laser 3D point cloud data to the laser sensor is used as the corresponding pixel coordinate position in the distance image.

[0019] In some embodiments, the step of filtering out ground points to obtain object points based on the distance image includes:

[0020] Based on the laser beam direction of the laser sensor, the points in the distance image containing the ground are traversed, and the pixel values ​​of each column in the distance image are processed into the tilt angle formed by two adjacent laser beams and the ground.

[0021] If the tilt angle is less than a set threshold, the index being traversed is considered a ground point;

[0022] The ground points obtained by segmentation based on the tilt angle are fitted to a plane to obtain a first plane. The points in the first plane include a plane composed of an interior point determined when the angle threshold requirement is met. The angle threshold is determined based on the angle threshold between the plane normal vector extracted from the first plane and the reference normal vector.

[0023] A threshold distance is set by traversing each point in the distance image and calculating the distance to the first plane.

[0024] In some embodiments, the step of filtering out ground points to obtain object points based on the distance image includes:

[0025] The points in the distance image are divided into near points and far points based on the longitudinal distance of the laser sensor.

[0026] If the distance between each point in the distance image and the first plane satisfies the threshold distance, and it is a nearby point, then it is taken as a new nearby ground point.

[0027] If the distance between each point in the distance image and the first plane satisfies the threshold distance, and it belongs to a point at a distance, then it is stored.

[0028] The stored distant points are subjected to a second plane fitting to obtain the second plane;

[0029] If the distance between each point in the distance image and the second plane satisfies the threshold distance, and it belongs to a distant point, then it is regarded as a new distant ground point;

[0030] The ground points are updated based on the new distant ground points and the new nearby ground points to obtain the entire point cloud ground points. After filtering, the object points are obtained.

[0031] In some embodiments, acquiring laser 3D point cloud data to be processed includes:

[0032] The laser 3D point cloud data is preprocessed to obtain laser 3D point cloud data to be processed. The preprocessing includes at least one of the following operations: ROI filtering, invalid point removal, and coordinate system transformation.

[0033] In some embodiments, obtaining the target segmentation result based on the object points includes:

[0034] Iterate through each of the object points in the distance image;

[0035] The search is performed with the row and column of each object point as the center, and the angle between two adjacent object points and the laser sensor is calculated.

[0036] If the included angle is greater than a preset angle threshold, then the two adjacent object points are clustered into the same target point;

[0037] Based on the clustering results and the laser beam of the laser sensor, noise filtering is performed to obtain the target segmentation result.

[0038] In some embodiments, obtaining the target segmentation result based on the object points includes:

[0039] Calculate the maximum longitudinal distance between all targets identified on both sides of the vehicle equipped with the laser device;

[0040] Determine whether the maximum length of the longitudinal distance of the target meets a preset length threshold;

[0041] If the conditions are met, it is assumed that a road boundary exists, and targets outside the road boundary are filtered out.

[0042] The targets within the road boundary are subjected to secondary filtering to obtain the target segmentation result.

[0043] In some embodiments, obtaining the target segmentation result based on the object points includes:

[0044] After secondary filtering, the targets are sorted according to the vertical distance of the centroid of the point cloud, and the 3DIoU between two adjacent targets is calculated.

[0045] If the 3DIoU is greater than the preset 3DIoU threshold, the two targets are clustered into a new target, and the target's attribute information is updated.

[0046] Secondly, embodiments of this application also provide a point cloud target detection device, wherein the device includes:

[0047] The acquisition module is used to acquire the laser 3D point cloud data to be processed;

[0048] The mapping module is used to map the laser 3D point cloud data to a distance image;

[0049] The filtering module is used to filter out ground points to obtain object points based on the distance image;

[0050] The segmentation module is used to obtain the target segmentation result based on the object points.

[0051] Thirdly, embodiments of this application also provide an electronic device, including: a processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform the above-described method.

[0052] Fourthly, embodiments of this application also provide a computer-readable storage medium that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform the above-described method.

[0053] The at least one technical solution adopted in this application embodiment can achieve the following beneficial effects: by acquiring laser 3D point cloud data to be processed, and then mapping the laser 3D point cloud data to a distance image. Further, based on the distance image, ground points are filtered out to obtain object points; finally, based on the object points, a target segmentation result is obtained. Object points are obtained by filtering ground points from the distance image through angle segmentation and piecewise fitting, and target clustering is achieved using a relevant point cloud optimization scheme to obtain the target segmentation and detection result. Attached Figure Description

[0054] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0055] Figure 1 This is a flowchart illustrating the point cloud target detection method in the embodiments of this application;

[0056] Figure 2 This is a schematic diagram of the point cloud target detection device in the embodiments of this application;

[0057] Figure 3 This is a schematic diagram illustrating the implementation principle of the point cloud target detection method in the embodiments of this application;

[0058] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0060] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0061] This application provides a point cloud target detection method, such as... Figure 1 The diagram shows a flowchart of a point cloud target detection method in an embodiment of this application. The method includes at least the following steps S110 to S140:

[0062] Step S110: Obtain the laser 3D point cloud data to be processed.

[0063] A common application of LiDAR is in autonomous vehicles. Autonomous vehicles are built upon conventional vehicles by adding advanced sensors (such as radar and cameras), controllers, and actuators, and integrating network and communication technologies. Through onboard sensing systems and information terminals, they achieve vehicle-to-vehicle, vehicle-to-infrastructure, and vehicle-to-person information sharing and intelligent information exchange, enabling vehicles to possess intelligent perception and decision-making capabilities. Therefore, the acquired 3D point cloud data is the result of scanning using LiDAR.

[0064] To make laser 3D point cloud data easier to process, some preprocessing can be performed in advance to obtain the laser 3D point cloud data to be processed.

[0065] Step S120: Map the laser 3D point cloud data to a distance image.

[0066] The laser 3D point cloud data is mapped onto a distance image. The distance image is divided into a grid of equal size according to a preset method, and the number of rows and columns in each grid serves as an index for the point cloud data. The index allows location of the point cloud data within a specific grid. Furthermore, the grid also stores the pixel coordinates of the laser point cloud.

[0067] In addition, the grid can also include lidar parameters such as the x-axis, y-axis, and z-axis coordinates of the laser point cloud in different directions, as well as reflectivity.

[0068] Step S130: Based on the distance image, filter out the ground points to obtain the object points.

[0069] Based on the distance image obtained above (which includes ground points and object points), global ground points are obtained through correlation filtering, and object points are obtained after filtering out these global ground points.

[0070] It should be noted that "object points" are target recognition results obtained through laser point cloud data, and the "object points" include the relevant point cloud data.

[0071] Step S140: Obtain the target segmentation result based on the object points.

[0072] Based on the object points in the distance image obtained above, target point clouds are segmented after target segmentation processing. These target point clouds can then be filtered and over-segmented to improve detection accuracy. This approach combines good real-time performance with excellent detection results.

[0073] The above methods can more effectively filter out interference from ground points, thereby improving the accuracy of target detection in laser point clouds.

[0074] The above method maps laser 3D point cloud data to a distance image, which facilitates angle segmentation and segmented fitting in the distance image. The entire point cloud ground points are obtained by continuously updating the ground points, and the distance image containing only object points is obtained by filtering out the entire point cloud ground points.

[0075] By employing the above method, combined with ground filtering processing involving angle and piecewise fitting, it is possible to detect smooth roads and ground surfaces with a certain slope, thereby filtering out ground points to obtain object points. Subsequently, oversegmentation processing is applied to the object points to reduce the occurrence of oversegmentation.

[0076] Unlike related technologies, deep learning-based methods have high hardware performance requirements, large development cycles, and high data costs. The methods described above have lower hardware performance requirements, shorter development cycles, lower data costs, and superior algorithm processing speed compared to deep learning. This ensures the real-time performance of positioning and perception results in autonomous driving systems.

[0077] Unlike traditional grid-based point cloud segmentation methods, which often suffer from incomplete ground filtering and whose ground points interfere with object points on the ground, leading to inaccurate target segmentation and affecting the final target detection results, the method described above effectively removes ground points compared to traditional grid-based processing. It can also handle terrain with a certain slope, reducing the false detection rate. Furthermore, it improves detection accuracy and achieves better target clustering, resulting in superior real-time performance for the target detection algorithm.

[0078] In one embodiment of this application, mapping the laser 3D point cloud data to a distance image includes: constructing a distance image based on the field of view (FOV) and resolution of the laser sensor, wherein the number of rows in the distance image is the ratio of the vertical field of view to the vertical resolution, and the number of columns in the distance image is the ratio of the horizontal field of view to the horizontal resolution; calculating the number of rows and columns in the distance image corresponding to each object point in the laser 3D point cloud data, and storing the number of rows and columns in the distance image corresponding to each point as an index; and using the distance value from the object point in the laser 3D point cloud data to the laser sensor as the corresponding pixel coordinate position in the distance image.

[0079] It is understandable that the field of view (FOV) of a laser sensor is related to its scanning range, and its resolution is the same as the display resolution. Based on the FOV and resolution of the LiDAR, a distance image can be constructed, thereby mapping the 3D point cloud data to be processed onto a 2.5D distance image.

[0080] In practice, the number of rows in the distance image is the ratio of the vertical field of view to the vertical resolution; if it is not an integer, it is rounded up. The number of columns is the ratio of the horizontal field of view to the horizontal resolution. Then, the row and column of the distance image corresponding to each point in the point cloud are calculated, the point cloud data corresponding to that index is stored, and the distance value from the object point to the laser sensor is used as the distance value of each pixel in this distance image.

[0081] It's understandable that if you know the row and column (index), you know which position in the image grid corresponds to the current distance.

[0082] For example, each point in the 3D point cloud data is mapped to a certain position in the distance image according to the algorithm logic. This specific position is determined by the row and column index. For instance, if the index of a point cloud point mapped to the distance image is (m, n), then the point cloud point corresponds to the position in the distance image at row m and column n. The pixel value of the distance image, which is the pixel value corresponding to the index position at row m and column n, is the Euclidean distance of that point cloud point from the laser.

[0083] In one embodiment of this application, the step of filtering out ground points to obtain object points based on the distance image includes: traversing the points in the distance image containing the ground according to the laser beam direction of the laser sensor, and processing each column of pixel values ​​in the distance image to the tilt angle formed by two adjacent laser beams and the ground; if the tilt angle is less than a set threshold, the traversed index is considered a ground point; performing plane fitting on the ground points obtained by the segmentation processing based on the tilt angle to obtain a first plane, wherein the points in the first plane include a plane composed of an interior point determined when the angle threshold requirement is met, and the angle threshold is determined based on the angle threshold set between the plane normal vector extracted from the first plane and the reference normal vector; and setting a threshold distance by traversing each point in the distance image and calculating the distance to the first plane.

[0084] First, perform angle segmentation processing.

[0085] a. Based on the vertical characteristic of the laser beam, the laser beam is constrained, and distance image points containing the ground are traversed in the distance image. Each column of pixel values ​​in the distance image is then processed to represent the tilt angle formed by adjacent laser beams and the ground. Using the ground obtained from angle segmentation as a priori, plane fitting can be further employed.

[0086] b. Determine whether the threshold condition is met based on the tilt angle. If the tilt angle is less than the set threshold, the index point being traversed is initially considered to be the ground point.

[0087] This step completes the angle segmentation process, resulting in a preliminary point cloud plane.

[0088] Secondly, perform plane fitting based on the initial fitting plane.

[0089] c. Perform plane fitting processing on the ground points initially obtained by the angle segmentation to obtain the fitted plane, and set an angle threshold between the extracted plane normal vector (the current plane normal vector refers to the fitted plane normal vector) and the reference normal vector. If the angle threshold requirement is met, a plane composed of interior points is obtained.

[0090] d. Then iterate through each point in the distance image and calculate the distance to the fitted plane, and set the distance threshold based on this.

[0091] In one embodiment of this application, the step of filtering out ground points to obtain object points based on the distance image includes: dividing the points in the distance image into near points and far points according to the longitudinal distance of the laser sensor; if the distance between each point in the distance image and the first plane meets the threshold distance threshold and is a near point, then it is designated as a new near ground point; if the distance between each point in the distance image and the first plane meets the threshold distance threshold and is a far point, then it is stored; performing a second plane fitting on the stored far points to obtain a second plane; if the distance between each point in the distance image and the second plane meets the threshold distance threshold and is a far point, then it is designated as a new far ground point; updating the ground points based on the new far ground points and the new near ground points to obtain the entire point cloud ground points, and then filtering them to obtain the object points.

[0092] First, perform the initial operations of piecewise fitting.

[0093] Using the longitudinal distance of the laser as the boundary, the point cloud points are divided into two segments: near and far. If the distance threshold requirement in step d is met and the point is in the near segment, it is considered a new near ground point and the ground point is updated. If it is a far point, it is stored.

[0094] Next, the stored distant points are subjected to a second plane fitting to obtain a second plane. An angle threshold is set between the extracted plane normal vector and the reference normal vector. If the angle threshold requirement is met, a plane composed of interior points is obtained. Then, the points in the distance image that meet the distance threshold requirement and are in the distant segment are traversed as new distant ground points. The ground points are updated, and then the ground points of the entire point cloud are obtained.

[0095] It should be noted that "first plane" and "second plane" are just examples and do not represent any specific fitting plane.

[0096] The principle behind the piecewise fitting process is as follows: The main idea is to first obtain the near ground points, and then obtain the distant ground points. First, a pre-selected point cloud of the ground is obtained through angle segmentation. Then, a first plane fitting is performed on this point cloud. Points in the fitted plane point cloud that satisfy the near longitudinal distance are taken as the near ground, while the points in the distant point cloud are retained and stored and sent to the second plane fitting to obtain the distant ground points.

[0097] The above steps first use the ground obtained from angle segmentation as a priori, and then perform plane fitting. Then, the distance of each point in the distance image to the fitted plane is calculated. Point clouds that meet the threshold requirements are then segmented into planes based on the longitudinal distance, and finally, global ground points are filtered out in the distance image.

[0098] In one embodiment of this application, the acquired laser 3D point cloud data to be processed includes: preprocessing the laser 3D point cloud data to obtain laser 3D point cloud data to be processed, wherein the preprocessing includes at least one of the following operations: ROI filtering, invalid point removal, and coordinate system transformation.

[0099] The obtained laser 3D point cloud data to be processed can be preprocessed.

[0100] ROI filtering typically refers to the extraction of ROI (Region of Interest) and the deletion of point cloud data that does not belong to a specific region. For example, point clouds that are more than 10m away from a vehicle need to be filtered out, and the final result is a downsampled and filtered point cloud.

[0101] Invalid point removal typically refers to filtering out point clouds that are isolated or belong to the background.

[0102] Coordinate system transformation unifies point data into a single coordinate system. For example, a vehicle coordinate system or a ground coordinate system.

[0103] In one embodiment of this application, obtaining the target segmentation result based on the object points includes: traversing each object point in the distance image; searching with the row and column of each traversed object point as the center, and calculating the angle formed between two adjacent object points and the laser sensor; if the angle is greater than a preset angle threshold, clustering the two adjacent object points into the same target point; and performing noise filtering based on the clustering result and the laser beam of the laser sensor to obtain the target segmentation result.

[0104] In practice, each object point in the distance image is traversed. A breadth-first search (BFS) is performed, centered on the row and column of each traversed point. The angle formed by adjacent points and the laser sensor is calculated (usually the angle formed by the longer side, but this can be chosen based on the actual situation). If the angle is greater than a certain threshold, the two points are clustered into the same target point. Noise filtering is then performed using the number of clustered points and the number of laser lines. The points that meet the requirements are the segmented target point cloud. Although the target point cloud is obtained through segmentation, there may still be oversegmentation or other non-target point clouds, which need to be filtered out.

[0105] In one embodiment of this application, obtaining the target segmentation result based on the object point includes: calculating the maximum length of the longitudinal distance between all targets identified on both sides of the vehicle body equipped with the laser device; determining whether the maximum length of the longitudinal distance between the targets meets a preset length threshold; if it does, it is considered that there is a road boundary, and targets outside the road boundary are filtered out; and the targets within the road boundary are subjected to secondary filtering to obtain the target segmentation result.

[0106] To obtain more accurate target segmentation results, secondary filtering is required.

[0107] a. Calculate the maximum longitudinal distance of all targets identified on both sides of the vehicle equipped with the laser device, and determine whether the longitudinal and lateral lengths meet the defined thresholds. If they do, a road boundary is considered to exist, and targets outside the boundary are filtered out. Then, calculate the distance between the centroid of the target in the near segment and the minimum point relative to the ground, thereby achieving the filtering of aerial noise. It should be noted that the "minimum point" is only an optional implementation method and is not used to limit the scope of protection in the embodiments of this application.

[0108] b. By restricting the target's length, width, height, area, volume, density, and other characteristics, secondary filtering of the target within the boundary is achieved.

[0109] In one embodiment of this application, obtaining the target segmentation result based on the object points includes: sorting the targets after secondary filtering according to the longitudinal distance of the centroid of the point cloud, calculating the 3DIoU between two adjacent targets; if the 3DIoU is greater than a preset 3DIoU threshold, then clustering the two targets into a new target and updating the target's attribute information.

[0110] It is understandable that "target" refers to the identification result of laser point cloud data. "Target" contains multiple point clouds and has been filtered out of noise.

[0111] IoU (Intersection over Union) is a metric for measuring the accuracy of detecting corresponding objects in a given dataset; it is also known as the crossover ratio. 3DIoU is the 3D crossover ratio between two targets.

[0112] After the above two-stage filtering process, the targets are sorted in ascending order of their centroid longitudinal distance. The 3DIoU between two adjacent targets is calculated. If it is greater than a certain threshold, the two targets are clustered into a new target and their attribute information is updated, thereby reducing the occurrence of oversegmentation to a certain extent.

[0113] The above method achieves target clustering by dividing the angle and determining the 3DIoU threshold.

[0114] This application embodiment also provides a point cloud target detection device 200, such as... Figure 2 As shown, a schematic diagram of the point cloud target detection device in this embodiment of the application is provided. The point cloud target detection device 200 includes at least: an acquisition module 210, a mapping module 220, a filtering module 230, and a segmentation module 240, wherein:

[0115] In one embodiment of this application, the acquisition module 210 is specifically used to: acquire laser 3D point cloud data to be processed.

[0116] A common application of LiDAR is in autonomous vehicles. Autonomous vehicles are built upon conventional vehicles by adding advanced sensors (such as radar and cameras), controllers, and actuators, and integrating network and communication technologies. Through onboard sensing systems and information terminals, they achieve vehicle-to-vehicle, vehicle-to-infrastructure, and vehicle-to-person information sharing and intelligent information exchange, enabling vehicles to possess intelligent perception and decision-making capabilities. Therefore, the acquired 3D point cloud data is the result of scanning using LiDAR.

[0117] To make laser 3D point cloud data easier to process, some preprocessing can be performed in advance to obtain the laser 3D point cloud data to be processed.

[0118] In one embodiment of this application, the mapping module 220 is specifically used to: map the laser 3D point cloud data to a distance image.

[0119] The laser 3D point cloud data is mapped onto a distance image. The distance image is divided into a grid of equal size according to a preset method, and the number of rows and columns in each grid serves as an index for the point cloud data. The index allows location of the point cloud data within a specific grid. Furthermore, the grid also stores the pixel coordinates of the laser point cloud.

[0120] In addition, the grid can also include lidar parameters such as the x-axis, y-axis, and z-axis coordinates of the laser point cloud in different directions, as well as reflectivity.

[0121] In one embodiment of this application, the filtering module 230 is specifically used to: filter out ground points to obtain object points based on the distance image.

[0122] Based on the distance image obtained above (which includes ground points and object points), global ground points are obtained through correlation filtering, and object points are obtained after filtering out these global ground points.

[0123] It should be noted that "object points" are target recognition results obtained through laser point cloud data, and the "object points" include the relevant point cloud data.

[0124] In one embodiment of this application, the segmentation module 240 is specifically used to: obtain a target segmentation result based on the object points.

[0125] Based on the object points in the distance image obtained above, target point clouds are segmented after target segmentation processing. These target point clouds can then be filtered and over-segmented to improve detection accuracy, achieving both good real-time performance and excellent detection results.

[0126] It is understood that the above-described point cloud target detection device can implement all the steps of the point cloud target detection method provided in the foregoing embodiments. The relevant explanations of the point cloud target detection method are applicable to the point cloud target detection device, and will not be repeated here.

[0127] To better understand the implementation principle of the point cloud target detection method in this application, ground filtering processing combining angle and piecewise fitting can achieve detection of flat roads and ground with a certain slope. Simultaneously, the angle segmentation algorithm in this proposal can reduce over-segmentation and improve detection accuracy, achieving both good real-time performance and excellent detection results. Please refer to... Figure 3 Specifically, it includes the following steps:

[0128] (1) Laser point cloud preprocessing. This includes ROI filtering, invalid point removal, and coordinate system transformation.

[0129] (2) Map the preprocessed 3D point cloud onto a 2.5D range image. Construct a range image based on the FOV and resolution of the lidar. The number of rows is the ratio of the vertical field of view to the vertical resolution. If it is not an integer, round it up. The number of columns is the ratio of the horizontal field of view to the horizontal resolution.

[0130] (3) Ground detection and filtration.

[0131] Based on the laser beam direction of the laser sensor, the points in the distance image containing the ground are traversed, and the pixel values ​​of each column in the distance image are processed into the tilt angle formed by two adjacent laser beams and the ground.

[0132] If the tilt angle is less than a set threshold, the index being traversed is considered a ground point;

[0133] The ground points obtained by segmentation based on the tilt angle are fitted to a plane to obtain a first plane. The points in the first plane include a plane composed of an interior point determined when the angle threshold requirement is met. The angle threshold is determined based on the angle threshold between the plane normal vector extracted from the first plane and the reference normal vector.

[0134] By traversing each point in the distance image and calculating the distance to the first plane, a threshold distance is set. The ground obtained from angle segmentation is used as a priori, and plane fitting is further applied. Then, the distance of each point in the depth image to the fitted plane is calculated. The point cloud that meets the threshold requirement is then segmented into planes based on the longitudinal distance to obtain the global ground points.

[0135] The points in the distance image are divided into near points and far points based on the longitudinal distance of the laser sensor.

[0136] If the distance between each point in the distance image and the first plane satisfies the threshold distance, and it is a nearby point, then it is taken as a new nearby ground point.

[0137] If the distance between each point in the distance image and the first plane satisfies the threshold distance, and it belongs to a point at a distance, then it is stored.

[0138] The stored distant points are subjected to a second plane fitting to obtain the second plane;

[0139] If the distance between each point in the distance image and the second plane satisfies the threshold distance, and it belongs to a distant point, then it is regarded as a new distant ground point;

[0140] The ground points are updated based on the new distant ground points and the new nearby ground points to obtain the entire point cloud ground points. After filtering, the object points are obtained.

[0141] (4) Target segmentation.

[0142] Iterate through each of the object points in the distance image;

[0143] The search is performed with the row and column of each object point as the center, and the angle between two adjacent object points and the laser sensor is calculated.

[0144] If the included angle is greater than a preset angle threshold, then the two adjacent object points are clustered into the same target point;

[0145] Based on the clustering results and the laser beam of the laser sensor, noise filtering is performed to obtain the target segmentation result.

[0146] (5) Filter filtering.

[0147] Calculate the maximum longitudinal distance between all targets identified on both sides of the vehicle equipped with the laser device;

[0148] Determine whether the maximum length of the longitudinal distance of the target meets a preset length threshold;

[0149] If the conditions are met, it is assumed that a road boundary exists, and targets outside the road boundary are filtered out.

[0150] The targets within the road boundary are subjected to secondary filtering to obtain the target segmentation result.

[0151] (6) Over-segmentation processing.

[0152] After secondary filtering, the targets are sorted according to the vertical distance of the centroid of the point cloud, and the 3DIoU between two adjacent targets is calculated.

[0153] If the 3DIoU is greater than the preset 3DIoU threshold, the two targets are clustered into a new target, and the target's attribute information is updated.

[0154] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 4 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0155] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0156] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0157] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming a point cloud target detection device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0158] Acquire the laser 3D point cloud data to be processed;

[0159] Map the laser 3D point cloud data to a distance image;

[0160] Based on the distance image, ground points are filtered out to obtain object points;

[0161] Based on the object points, the target segmentation result is obtained.

[0162] The above is as stated in this application. Figure 1The method executed by the point cloud target detection device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0163] The electronic device can also perform Figure 1 The method for executing a point cloud target detection device, and the implementation of the point cloud target detection device in... Figure 1 The functions of the embodiments shown are not described again in this application.

[0164] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 1 The method executed by the point cloud target detection device in the illustrated embodiment is specifically used to perform the following:

[0165] Acquire the laser 3D point cloud data to be processed;

[0166] Map the laser 3D point cloud data to a distance image;

[0167] Based on the distance image, ground points are filtered out to obtain object points;

[0168] Based on the object points, the target segmentation result is obtained.

[0169] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0170] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0171] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0172] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0173] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0174] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0175] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0176] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0177] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0178] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A point cloud target detection method, wherein, The method includes: Acquire the laser 3D point cloud data to be processed; Map the laser 3D point cloud data to a distance image; Based on the distance image, ground points are filtered out to obtain object points; Based on the object points, the target segmentation result is obtained; The step of obtaining the target segmentation result based on the object points includes: Calculate the maximum longitudinal distance between all targets identified on both sides of the vehicle equipped with the laser device; Determine whether the maximum length of the longitudinal distance of the target meets a preset length threshold; If the conditions are met, it is assumed that a road boundary exists, and targets outside the road boundary are filtered out. The targets within the road boundary are subjected to secondary filtering to obtain the target segmentation result; After secondary filtering, the targets are sorted according to the vertical distance of the centroid of the point cloud, and the 3DIoU between two adjacent targets is calculated. If the 3DIoU is greater than the preset 3DIoU threshold, the two targets are clustered into a new target, and the target's attribute information is updated.

2. The method as described in claim 1, wherein, The step of mapping the laser 3D point cloud data to a distance image includes: A distance image is constructed based on the field of view (FOV) and resolution of the laser sensor, wherein the number of rows in the distance image is the ratio of the vertical field of view to the vertical resolution, and the number of columns in the distance image is the ratio of the horizontal field of view to the horizontal resolution. Calculate the number of rows and columns of the distance image corresponding to each object point in the laser 3D point cloud data, and store the number of rows and columns of the distance image corresponding to each point as an index; The distance value from the object point in the laser 3D point cloud data to the laser sensor is used as the corresponding pixel coordinate position in the distance image.

3. The method as described in claim 2, wherein, The step of filtering out ground points to obtain object points based on the distance image includes: Based on the laser beam direction of the laser sensor, the points in the distance image containing the ground are traversed, and the pixel values ​​of each column in the distance image are processed into the tilt angle formed by two adjacent laser beams and the ground. If the tilt angle is less than a set threshold, the index being traversed is considered a ground point; The ground points obtained by segmentation based on the tilt angle are fitted to a plane to obtain a first plane. The points in the first plane include a plane composed of an interior point determined when the angle threshold requirement is met. The angle threshold is determined based on the angle threshold between the plane normal vector extracted from the first plane and the reference normal vector. A threshold distance is set by traversing each point in the distance image and calculating the distance to the first plane.

4. The method as described in claim 3, wherein, The step of filtering out ground points to obtain object points based on the distance image includes: The points in the distance image are divided into near points and far points based on the longitudinal distance of the laser sensor. If the distance between each point in the distance image and the first plane satisfies the threshold distance, and it is a nearby point, then it is taken as a new nearby ground point. If the distance between each point in the distance image and the first plane satisfies the threshold distance, and it belongs to a point at a distance, then it is stored. The stored distant points are subjected to a second plane fitting to obtain the second plane; If the distance between each point in the distance image and the second plane satisfies the threshold distance, and it belongs to a distant point, then it is regarded as a new distant ground point; The ground points are updated based on the new distant ground points and the new nearby ground points to obtain the entire point cloud ground points. After filtering, the object points are obtained.

5. The method as described in claim 1, wherein, Acquire the laser 3D point cloud data to be processed, including: The laser 3D point cloud data is preprocessed to obtain laser 3D point cloud data to be processed. The preprocessing includes at least one of the following operations: ROI filtering, invalid point removal, and coordinate system transformation.

6. A point cloud target detection device, wherein, The device includes: The acquisition module is used to acquire the laser 3D point cloud data to be processed; The mapping module is used to map the laser 3D point cloud data to a distance image; The filtering module is used to filter out ground points to obtain object points based on the distance image; The segmentation module is used to obtain the target segmentation result based on the object points; The step of obtaining the target segmentation result based on the object points includes: Calculate the maximum longitudinal distance between all targets identified on both sides of the vehicle equipped with the laser device; Determine whether the maximum length of the longitudinal distance of the target meets a preset length threshold; If the conditions are met, it is assumed that a road boundary exists, and targets outside the road boundary are filtered out. The targets within the road boundary are subjected to secondary filtering to obtain the target segmentation result; After secondary filtering, the targets are sorted according to the vertical distance of the centroid of the point cloud, and the 3DIoU between two adjacent targets is calculated. If the 3DIoU is greater than the preset 3DIoU threshold, the two targets are clustered into a new target, and the target's attribute information is updated.

7. An electronic device, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 5.

Images