Method and device for stripping foreground and background of point cloud

Abstract

The invention discloses a method and a device for stripping a foreground and a background of a point cloud. The bounding box and the boundary contour of the obstacle in the 2D space are identified from the image, and a clear reference is provided for stripping background point cloud. A bounding box is expanded to a 3D space to obtain a first view cone, a point cloud located in the first view cone is obtained to serve as a first stripping reserved point cloud, and most background point clouds are effectively filtered. According to the method, a boundary contour is extracted from a 3D space to obtain a first view cone, a boundary contour is expanded to the 3D space to obtain a second view cone, point clouds located in the second view cone are obtained from first-time stripping reserved point clouds to serve as second-time stripping reserved point clouds, background point clouds are further removed finely, accurate point cloud input is provided for 3D target detection, and the data size is reduced. According to the invention, the method achieves the high-precision stripping of the foreground and background of the point cloud in a complex environment, solves the problems in the prior art that the foreground and background separation precision and processing speed are insufficient, solves the problem of high requirements for the computing power and environment, and improves the universality and effectiveness of the technology for stripping the foreground and background of the point cloud.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for separating a point cloud foreground and background. Background Art

[0002] With the development of perception technology, accurate and real-time perception of the surrounding environment is crucial in the fields of autonomous driving, robot navigation, etc. Foreground and background separation of point cloud data is an important processing step, the goal is to distinguish foreground objects (such as obstacles) and background environment in the scene. Common point cloud foreground and background separation technologies include separation algorithms based on lidar or cameras, but the three-dimensional data collected by lidar has limited semantic classification capabilities for objects, and the two-dimensional data collected by cameras lacks depth and distance perception capabilities.

[0003] Therefore, there are still stripping methods based on the fusion of camera and lidar in the existing technology, which can complement the advantages of camera and lidar, but have disadvantages such as complex operation, high computing resource requirements, and susceptibility to environmental influences, which limits the breadth and effectiveness of existing technologies in practical applications. Therefore, how to improve the breadth and effectiveness of point cloud foreground and background stripping technology in complex environments has become an urgent problem to be solved. Summary of the invention

[0004] Based on the above problems, the present application provides a method and device for separating the foreground and background of a point cloud, so as to improve the breadth and effectiveness of the point cloud foreground and background separation technology in complex environments.

[0005] The present application discloses a method for separating a point cloud foreground and background, the method comprising:

[0006] Identify a bounding box and a bounding outline of an obstacle from an image acquired by a camera; the bounding box and the bounding outline exist in a 2D space;

[0007] Expanding the bounding box into 3D space to obtain a first viewing cone;

[0008] In the point cloud acquired by the laser radar, a point cloud located in the first visual cone is acquired as the first stripping and retaining point cloud;

[0009] Expanding the boundary contour into 3D space to obtain a second viewing cone;

[0010] In the first stripping retained point cloud, obtaining the point cloud located in the second visual cone as the second stripping retained point cloud;

[0011] The re-stripped retained point cloud is output as an obstacle point cloud, completing the stripping of the point cloud foreground and background.

[0012] Optionally, identifying a bounding box and a boundary contour of an obstacle from an image acquired from a camera includes:

[0013] After processing the image into a fixed size, the image is input into an obstacle recognition model; the obstacle recognition model uses a multi-scale prediction technology;

[0014] The obstacle recognition model outputs the bounding box and the bounding contour, and category information of obstacles corresponding to each of the bounding box and the bounding contour.

[0015] Optionally, after identifying the boundary box and boundary contour of the obstacle from the image acquired by the camera, the method further includes:

[0016] Remove overlapping bounding boxes using the non-maximum suppression algorithm;

[0017] The bounding boxes corresponding to categories whose confidence level is lower than a preset threshold are removed.

[0018] Optionally, acquiring a point cloud located in the first visual cone from the point cloud acquired by the laser radar includes:

[0019] The point cloud acquired by the laser radar is used as the original point cloud, and the original point cloud is preprocessed; the preprocessing includes statistical filtering, voxel grid filtering, ground segmentation and edge feature enhancement;

[0020] The point cloud located in the first viewing cone in the preprocessed point cloud is obtained.

[0021] Optionally, the statistical filtering includes:

[0022] Calculate the average value and standard deviation of the distance between each point in the original point cloud and other points in the neighborhood;

[0023] Outliers are located and removed based on the mean value and the standard deviation.

[0024] Optionally, the voxel grid filtering includes:

[0025] Divide the 3D space where the statistically filtered point cloud is located into voxels and calculate the centroid coordinates of each voxel;

[0026] The point cloud coordinates in each voxel are replaced by the centroid coordinates of each voxel to perform data downsampling.

[0027] Optionally, the ground segmentation includes:

[0028] The ground point cloud is identified and removed by using the plane assumption of the ground and combining it with a random sampling consensus algorithm.

[0029] Optionally, the edge feature enhancement includes:

[0030] A high-pass filter is used to enhance the edge features of the point cloud after ground segmentation, so that the boundary clarity of obstacles is enhanced.

[0031] Optionally, the obstacle recognition model is based on YOLOv10.

[0032] Based on the above method for stripping a point cloud foreground and background, the present application also discloses a device for stripping a point cloud foreground and background, comprising: an image processing unit, a first stripping unit, a second stripping unit and an output unit;

[0033] The image processing unit identifies a bounding box and a bounding outline of an obstacle from an image acquired by a camera; the bounding box and the bounding outline exist in a 2D space;

[0034] The image processing unit is further used to expand the bounding box into 3D space to obtain a first viewing cone;

[0035] The first stripping unit is used to obtain the point cloud located in the first visual cone from the point cloud obtained by the laser radar as the first stripping retained point cloud;

[0036] The image processing unit is further used to expand the boundary contour into 3D space to obtain a second visual cone;

[0037] The second stripping unit is used to obtain the point cloud located in the second viewing cone in the first stripping retained point cloud as the second stripping retained point cloud;

[0038] The output unit is used to output the re-stripped retained point cloud as an obstacle point cloud, thereby completing the stripping of the point cloud foreground and background.

[0039] Optionally, the image processing unit includes:

[0040] An input subunit, used to process the image into a fixed size and then input it into an obstacle recognition model; the obstacle recognition model uses a multi-scale prediction technology;

[0041] The output subunit is used for the obstacle recognition model to output the bounding box and the bounding contour, and category information of the obstacle corresponding to each of the bounding box and the bounding contour.

[0042] Optionally, the device further comprises:

[0043] A post-processing unit for removing overlapping bounding boxes by a non-maximum suppression algorithm;

[0044] It is also used to remove the bounding boxes corresponding to categories whose confidence is lower than a preset threshold.

[0045] Optionally, the first stripping unit includes:

[0046] A preprocessing subunit, used to use the point cloud acquired by the laser radar as an original point cloud and preprocess the original point cloud; the preprocessing includes statistical filtering, voxel grid filtering, ground segmentation and edge feature enhancement;

[0047] The stripping subunit is used to obtain the point cloud located in the first viewing cone in the preprocessed point cloud.

[0048] Optionally, the preprocessing subunit includes:

[0049] A statistical filter subunit, used to calculate the average value and standard deviation of the distance between each point in the original point cloud and other points in the neighborhood;

[0050] It is also used to locate and remove abnormal points based on the mean value and the standard deviation.

[0051] Optionally, the preprocessing subunit includes:

[0052] A voxel grid filter subunit is used to divide the 3D space where the statistically filtered point cloud is located into voxels and calculate the centroid coordinates of each voxel;

[0053] It is also used to replace the point cloud coordinates in each voxel with the centroid coordinates of each voxel for data downsampling.

[0054] Optionally, the preprocessing subunit includes:

[0055] The ground segmentation subunit is used to identify and remove ground point clouds by using the plane assumption of the ground and combining it with a random sampling consensus algorithm.

[0056] Optionally, the preprocessing subunit includes:

[0057] The edge feature enhancement subunit is used to enhance the edge features of the point cloud after ground segmentation using a high-pass filter, so as to enhance the boundary clarity of obstacles.

[0058] Optionally, the obstacle recognition model is based on YOLOv10.

[0059] The present application discloses a method and device for stripping the foreground and background of a point cloud. First, the bounding box and boundary contour of the obstacle in the 2D space are identified from the image acquired by the camera, and the boundary of the object is accurately defined to provide a clearer benchmark for stripping the background point cloud. Subsequently, the bounding box is expanded to the 3D space to obtain the first visual cone. In the point cloud acquired by the laser radar, the point cloud located in the first visual cone is obtained as the first stripping and retained point cloud, and the background point cloud is removed, effectively filtering out most of the background point cloud. The boundary contour is then expanded to the 3D space to obtain the second visual cone. In the first stripping and retained point cloud, the point cloud located in the second visual cone is obtained as the second stripping and retained point cloud, and the background point cloud is removed. This process reduces the noise and interference caused by the background point cloud, and significantly improves the quality of the point cloud data. The two-step stripping further and finely removes the background point cloud while also greatly reducing the amount of data, providing accurate point cloud input for subsequent target detection algorithms. In summary, the method described in this application achieves high-precision separation of point cloud foreground and background in complex environments, solves the shortcomings of existing technologies in foreground and background separation accuracy and processing speed, as well as high requirements for hardware computing power and environment, and improves the breadth and effectiveness of point cloud foreground and background separation technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0061] Figure 1 A schematic diagram of a process for separating a point cloud foreground and background disclosed in an embodiment of the present application;

[0062] Figure 2 A flowchart of another method for separating the foreground and background of a point cloud disclosed in an embodiment of the present application;

[0063] Figure 3 A schematic diagram of the structure of a device for separating the foreground and background of a point cloud disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The method described in this application needs to be implemented through a perception device. As an example, the perception device may include a camera and a laser radar. The camera selected for the method described in this application has high performance in image quality, resolution, frame rate, viewing angle width, light sensitivity, and physical and time synchronization capabilities with the laser radar. The camera provides high resolution to capture detailed and rich images, which helps to identify and classify objects. At the same time, the camera has a high enough frame rate to match the laser radar data update speed to ensure the real-time data in a dynamic environment. The camera's wide viewing angle can cover the detection range of the laser radar, while the high dynamic range and excellent low-light performance ensure that high-quality image performance can be maintained under a variety of lighting conditions. In addition, the camera selected for the method described in this application has hardware triggering and timestamp functions to achieve precise synchronization with the laser radar.

[0065] Since the method described in this application relies on distance measurement to accurately distinguish foreground targets from the background, the laser radar selected for the method described in this application can have a higher distance resolution capability. Similarly, the laser radar can use high resolution to provide denser point cloud data, enhance the spatial definition and boundary clarity of objects, and make the separation of background and foreground more accurate. The scanning range of the laser radar needs to be wide to cover the complete environment around the vehicle, and the update frequency ensures that the data stream is synchronized with the image captured by the camera in real time, supporting the detection and classification of fast-moving objects. In addition, the stability and anti-interference ability of the laser radar selected by the method described in this application under various weather conditions are also very critical.

[0066] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0067] Embodiment 1: This application discloses a method for separating the foreground and background of a point cloud.

[0068] For details, please refer to Figure 1 , a method for separating the foreground and background of a point cloud disclosed in this embodiment includes the following steps:

[0069] Step 101: Identify the boundary box and boundary contour of the obstacle from the image acquired by the camera.

[0070] In the method described in this embodiment, the bounding box and the bounding contour exist in a 2D space.

[0071] As an optional method, the obstacle recognition model is used to identify the bounding box and boundary contour of the obstacle. Specifically, the image can be processed into a fixed size (such as 640x640 pixels) and then input into the obstacle recognition model to ensure that the obstacle recognition model can accept consistent input from different sources. After the obstacle recognition model works, it outputs the bounding box of the obstacle, as well as the category and mask information of the obstacle corresponding to each bounding box. Among them, the mask information is a binary image that can reflect the boundary contour and shape of the obstacle, and supports accurate segmentation of each obstacle in the image. In this binary image, the pixel value can be set to 1 to indicate that the pixel belongs to an obstacle, and the pixel value can be set to 0 to indicate that it does not belong to an obstacle. The bounding box can represent the center point coordinates and width and height through a four-dimensional vector, and the category can represent the confidence of the obstacle category to which its corresponding bounding box may belong with a probability.

[0072] As an optional method, the obstacle recognition model is based on the YOLO (You Only Look Once) v10 algorithm. YOLOv10 is a deep learning model that integrates object detection and instance segmentation. It uses a single convolutional neural network (CNN) to simultaneously predict the information of multiple obstacles. Among them, CNN gradually extracts the features of the image through multiple convolutional layers, batch normalization layers, and activation layers. At the back end of CNN, YOLOv10 uses multi-scale prediction technology. This means that CNN outputs prediction results at multiple levels to support the detection of objects of different sizes. Each output layer predicts bounding boxes at three scales, and each scale uses specific anchor boxes (anchorboxes) to predict the position and size of the bounding box. The advantages of the obstacle recognition model based on YOLOv10 are fast speed and high accuracy. It is suitable for application scenarios that require real-time processing, so it can successfully achieve accurate object detection and foreground and background separation in complex environments.

[0073] As another optional method, the obstacle recognition model can also be based on the improved R-CNN (Region-CNN) series models, such as Faster R-CNN, Mask R-CNN, etc. These models can also provide detailed control and excellent performance when dealing with object detection and instance segmentation tasks. As another optional method, the obstacle recognition model can also be based on the Transformers architecture to process image and point cloud data. Its global dependency capture capability can provide accurate detection results in complex scenes.

[0074] The obstacle recognition model described in this embodiment takes a whole image as input and outputs various information about obstacles. This one-step method greatly improves the processing speed compared to the traditional step-by-step method (such as detecting the bounding box first and then segmenting).

[0075] As an optional method, the prediction results of the obstacle recognition model are post-processed, which may include removing overlapping bounding boxes through a non-maximum suppression algorithm to ensure that each obstacle is detected only once, and removing bounding boxes corresponding to categories with confidence levels lower than a preset threshold to ensure the accuracy of the obstacle type.

[0076] Step 102: Expand the bounding box into 3D space to obtain a first viewing cone.

[0077] In the method described in this embodiment, the 2D bounding box (usually a rectangle) detected by the obstacle recognition model is expanded into the 3D space to obtain the first view cone. The view cone is a conical area expanded into the 3D space from the camera's perspective, indicating the spatial range corresponding to the edge contour of the obstacle in the camera image, and is used for background removal when filtering point cloud data. Coordinate transformation technology can be used to unify the data obtained by different sensors (camera and lidar) into the same coordinate system.

[0078] In the method described in this embodiment, as an optional method, in addition to spatial conversion and fusion, a data fusion method based on time synchronization can be used to fuse the timestamps of the synchronously collected image and point cloud data to improve the accuracy of data processing. As another optional method, a deep learning model can be used to directly fuse at the feature layer instead of at the decision layer, which can automatically learn the intrinsic connection between the two types of data through the network, thereby improving the robustness and effect of the system.

[0079] Step 103: From the point cloud acquired by the laser radar, obtain the point cloud located in the first viewing cone as the first stripping and retaining point cloud.

[0080] In the method described in this embodiment, the point cloud acquired by the laser radar is preprocessed. This includes statistical filtering, voxel grid filtering, ground segmentation and edge feature enhancement, with the purpose of optimizing the point cloud data to make it more suitable for deep learning and other computationally intensive tasks. As an optional method, statistical filtering can specifically analyze the statistical characteristics of the neighborhood point set of each point to identify and remove noise, that is, calculate the mean and standard deviation of the distance between each point and other points in the neighborhood in the point cloud acquired by the laser radar. Then decide whether to retain the point based on the mean and standard deviation, and realize the function of locating and removing outliers. The statistical filtering step can significantly reduce the outliers in the point cloud data and improve the overall quality of the data.

[0081] As an optional method, voxel grid filtering can specifically divide the 3D space where the point cloud after statistical filtering is located into voxels, and calculate the centroid coordinates of each voxel. The centroid coordinates of each voxel are used to replace the point cloud coordinates in each voxel to perform data downsampling. The voxel grid filtering step can effectively reduce the total number of points, thereby reducing the computational burden of subsequent processing steps. It also helps to even out the density of the point cloud and improve the uniformity of the data.

[0082] As an optional method, ground segmentation can be specifically to use the plane assumption of the ground and combine it with the random sampling consensus algorithm to identify and remove the ground point cloud. Through ground segmentation, non-ground objects (such as vehicles, buildings, and pedestrians) will be highlighted, which facilitates the subsequent foreground and background separation tasks.

[0083] As an optional method, edge feature enhancement can be specifically to use a high-pass filter to enhance the edge features of the point cloud after ground segmentation processing, so that the boundary clarity of the obstacle is enhanced. This step is intended to improve the feature expression ability of the point cloud data and make the boundaries of each object clearer.

[0084] In the method described in this embodiment, as an optional method, the above preprocessing can also be replaced by an optical flow algorithm, which can predict the movement of each point in the point cloud data, thereby accurately performing denoising and compression of the point cloud data. As another optional method, a multi-scale processing strategy can also be introduced to capture more details at different scales, which helps to improve the quality and use of the processed data.

[0085] In the method described in this embodiment, the point cloud located in the first viewing cone in the preprocessed point cloud is obtained. This step can remove most of the background point cloud.

[0086] Step 104: Expand the boundary contour into 3D space to obtain a second viewing cone.

[0087] In the method described in this embodiment, the boundary contour of the obstacle is obtained from the mask information detected by the obstacle recognition model, and is extended to the 3D space to obtain the second visual cone.

[0088] Step 105: In the first stripping retained point cloud, obtain the point cloud located in the second viewing cone as the second stripping retained point cloud.

[0089] In the method described in this embodiment, the second stripping can further refine the removal of the background point cloud and retain only the obstacle point cloud. At the same time, after two strippings, the amount of point cloud data is greatly reduced, which can also improve the efficiency of the subsequent target detection algorithm and improve the accuracy.

[0090] Step 106: The re-stripped and retained point cloud is output as an obstacle point cloud, thereby completing the stripping of the point cloud foreground and background.

[0091] The method described in this embodiment aims to improve the ability to perceive obstacles in complex environments, overcome the limitations of computing resources at the algorithm level, and achieve high-precision and real-time environmental perception. Obstacle recognition is performed based on the improved YOLOv10 model, and the 2D boundary contour of the object is accurately extracted from the image acquired by the camera through the instance segmentation algorithm. The advantages of deep learning are used to improve the accuracy of object recognition and classification. For point cloud data, the method described in this embodiment designs a comprehensive preprocessing algorithm, including denoising, voxel grid filtering, ground segmentation and feature enhancement steps, which optimizes data quality and reduces the computational burden. The method described in this embodiment uses two background point cloud stripping. The first stripping can remove most of the background point cloud, and the second stripping is based on the 2D boundary contour of the obstacle to further refine the background point cloud. After two strippings, the amount of point cloud data is greatly reduced. In view of the problem that the amount of point cloud data is huge and the fine stripping of background point cloud consumes a lot of computing resources, the method described in this embodiment is more efficient and fast, and significantly reduces the consumption of computing resources. In summary, this embodiment achieves high-precision and real-time perception of obstacles in complex environments through multi-level data fusion and algorithm optimization. It helps autonomous driving and robot navigation systems to perform object recognition and classification more accurately, plan paths and avoid obstacles more effectively when dealing with complex or dense traffic environments, and enhances the accuracy and reliability of environmental models.

[0092] Embodiment 2: This application discloses another method for separating the foreground and background of a point cloud. Figure 2 The method described in this embodiment introduces the whole process of separating the foreground and the background.

[0093] Step 201: Acquire a 2D image through a camera.

[0094] Step 202: Input the 2D image into the obstacle recognition model to obtain the boundary box and boundary contour of the obstacle in the 2D space.

[0095] Step 203: Expand the bounding box of the obstacle to the 3D space to obtain the first viewing cone.

[0096] Step 204: Expand the boundary outline of the obstacle to the 3D space to obtain a second viewing cone.

[0097] Step 205: Acquire a 3D point cloud through a laser radar.

[0098] Step 206: pre-process the 3D point cloud, select the 3D point cloud located in the first viewing cone, retain it as the first stripping point cloud, and remove the 3D point cloud outside the first viewing cone as the background point cloud.

[0099] Step 207: Select the first stripping retained point cloud located in the second viewing cone and retain it as the obstacle point cloud, and remove the 3D point cloud outside the second viewing cone as the background point cloud.

[0100] Step 208: Output the obstacle point cloud to complete the separation of the point cloud foreground and background.

[0101] Based on the method for separating the foreground and background of a point cloud disclosed in the above embodiment, this embodiment correspondingly discloses a device for separating the foreground and background of a point cloud. Figure 3 , the device for stripping the foreground and background of a point cloud comprises: an image processing unit 301, a first stripping unit 302, a second stripping unit 303 and an output unit 304;

[0102] The image processing unit 301 identifies a boundary box and a boundary outline of an obstacle from an image acquired by a camera; the boundary box and the boundary outline exist in a 2D space;

[0103] The image processing unit 301 is further used to expand the bounding box into 3D space to obtain a first viewing cone;

[0104] The first stripping unit 302 is used to obtain the point cloud located in the first visual cone from the point cloud obtained by the laser radar as the first stripping retained point cloud;

[0105] The image processing unit is further used to expand the boundary contour into 3D space to obtain a second visual cone;

[0106] The second stripping unit 303 is used to obtain the point cloud located in the second viewing cone in the first stripping retained point cloud as the second stripping retained point cloud;

[0107] The output unit 304 is used to output the re-stripped retained point cloud as an obstacle point cloud, thereby completing the stripping of the point cloud foreground and background.

[0108] Optionally, the image processing unit 301 includes:

[0109] An input subunit, used to process the image into a fixed size and then input it into an obstacle recognition model; the obstacle recognition model uses a multi-scale prediction technology;

[0110] The output subunit is used for the obstacle recognition model to output the bounding box and the bounding contour, and category information of the obstacle corresponding to each of the bounding box and the bounding contour.

[0111] Optionally, the device further comprises:

[0112] A post-processing unit for removing overlapping bounding boxes by a non-maximum suppression algorithm;

[0113] It is also used to remove the bounding boxes corresponding to categories whose confidence is lower than a preset threshold.

[0114] Optionally, the first stripping unit 302 includes:

[0115] A preprocessing subunit, used to use the point cloud acquired by the laser radar as an original point cloud and preprocess the original point cloud; the preprocessing includes statistical filtering, voxel grid filtering, ground segmentation and edge feature enhancement;

[0116] The stripping subunit is used to obtain the point cloud located in the first viewing cone in the preprocessed point cloud.

[0117] Optionally, the preprocessing subunit includes:

[0118] A statistical filter subunit, used to calculate the average value and standard deviation of the distance between each point in the original point cloud and other points in the neighborhood;

[0119] It is also used to locate and remove abnormal points based on the mean value and the standard deviation.

[0120] Optionally, the preprocessing subunit includes:

[0121] A voxel grid filter subunit is used to divide the 3D space where the statistically filtered point cloud is located into voxels and calculate the centroid coordinates of each voxel;

[0122] It is also used to replace the point cloud coordinates in each voxel with the centroid coordinates of each voxel for data downsampling.

[0123] Optionally, the preprocessing subunit includes:

[0124] The ground segmentation subunit is used to identify and remove ground point clouds by using the plane assumption of the ground and combining it with a random sampling consensus algorithm.

[0125] Optionally, the preprocessing subunit includes:

[0126] The edge feature enhancement subunit is used to enhance the edge features of the point cloud after ground segmentation using a high-pass filter, so as to enhance the boundary clarity of obstacles.

[0127] Optionally, the obstacle recognition model is based on YOLOv10.

[0128] The embodiments in this specification are described in a progressive manner. As for the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the methods.

[0129] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0130] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0131] The features described in the embodiments of this specification can be replaced or combined with each other, so that professional and technical personnel in this field can implement or use the present application.

[0132] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for separating the foreground and background of a point cloud, characterized in that: include: Identify a bounding box and a bounding outline of an obstacle from an image acquired by a camera; the bounding box and the bounding outline exist in a 2D space; Expanding the bounding box into 3D space to obtain a first viewing cone; In the point cloud acquired by the laser radar, a point cloud located in the first visual cone is acquired as the first stripping and retaining point cloud; Expanding the boundary contour into 3D space to obtain a second viewing cone; In the first stripping retained point cloud, obtaining the point cloud located in the second visual cone as the second stripping retained point cloud; The re-stripped retained point cloud is output as an obstacle point cloud, completing the stripping of the point cloud foreground and background.

2. The method according to claim 1, characterized in that The step of identifying a bounding box and a boundary contour of an obstacle from an image acquired by a camera includes: After processing the image into a fixed size, the image is input into an obstacle recognition model; the obstacle recognition model uses a multi-scale prediction technology; The obstacle recognition model outputs the bounding box and the bounding contour, and category information of obstacles corresponding to each of the bounding box and the bounding contour.

3. The method according to claim 2, characterized in that After identifying the bounding box and the bounding outline of the obstacle from the image acquired by the camera, the method further includes: Remove overlapping bounding boxes using the non-maximum suppression algorithm; The bounding boxes corresponding to categories whose confidence level is lower than a preset threshold are removed.

4. The method according to claim 1, characterized in that: The step of acquiring a point cloud located in the first visual cone from the point cloud acquired by the laser radar includes: The point cloud acquired by the laser radar is used as the original point cloud, and the original point cloud is preprocessed; the preprocessing includes statistical filtering, voxel grid filtering, ground segmentation and edge feature enhancement; The point cloud located in the first viewing cone in the preprocessed point cloud is obtained.

5. The method according to claim 4, characterized in that The statistical filtering comprises: Calculate the average value and standard deviation of the distance between each point in the original point cloud and other points in the neighborhood; Outliers are located and removed based on the mean value and the standard deviation.

6. The method according to claim 5, characterized in that The voxel grid filtering comprises: Divide the 3D space where the statistically filtered point cloud is located into voxels and calculate the centroid coordinates of each voxel; The point cloud coordinates in each voxel are replaced by the centroid coordinates of each voxel to perform data downsampling.

7. The method according to claim 6, characterized in that The ground segmentation comprises: The ground point cloud is identified and removed by using the plane assumption of the ground and combining it with a random sampling consensus algorithm.

8. The method according to claim 7, characterized in that The edge feature enhancement includes: A high-pass filter is used to enhance the edge features of the point cloud after ground segmentation, so that the boundary clarity of obstacles is enhanced.

9. The method according to claim 2, characterized in that: The obstacle recognition model is based on YOLOv10.

10. A device for separating the foreground and background of a point cloud, characterized in that: include: An image processing unit, a first peeling unit, a second peeling unit and an output unit; The image processing unit is used to identify a bounding box and a bounding outline of an obstacle from an image acquired by a camera; the bounding box and the bounding outline exist in a 2D space; The image processing unit is further used to expand the bounding box into 3D space to obtain a first viewing cone; The first stripping unit is used to obtain the point cloud located in the first visual cone from the point cloud obtained by the laser radar as the first stripping retained point cloud; The image processing unit is further used to expand the boundary contour into 3D space to obtain a second visual cone; The second stripping unit is used to obtain the point cloud located in the second viewing cone in the first stripping retained point cloud as the second stripping retained point cloud; The output unit is used to output the re-stripped retained point cloud as an obstacle point cloud, thereby completing the stripping of the point cloud foreground and background.

Images