A point cloud data filtering method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a point cloud data filtering method and device, electronic equipment and storage medium, relates to the technical field of computers, and particularly relates to the technical field of high-definition maps, autonomous driving, artificial intelligence and the like. The specific implementation scheme is as follows: acquiring a plurality of frames of point cloud data collected in a sensor movement process, and sensor movement trajectory information corresponding to a time period of each frame of point cloud data collected; for each frame of point cloud data, extracting ground points in the frame of point cloud data; based on each sensor movement trajectory information, splicing the extracted ground points in each frame of point cloud data to obtain a ground point cloud model; based on each sensor movement trajectory information, splicing points in each frame of point cloud data to obtain a global point cloud model; filtering points in the global point cloud model that are above the ground point cloud model to obtain filtered point cloud data, thereby achieving filtering of dynamic point cloud.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computers, further relates to the technical fields of high-precision maps, autonomous driving, artificial intelligence, and the like, and in particular relates to a point cloud data filtering method and device, an electronic device, and a storage medium. BACKGROUND

[0002] With the development of computer technology, autonomous driving has become an important direction for the future development of vehicles. In the field of autonomous driving, a high-precision map is one of the important elements. In the process of drawing a high-precision map, it is necessary to filter the point cloud data corresponding to dynamic objects without drawing the point cloud data corresponding to dynamic objects. SUMMARY

[0003] The present disclosure provides a point cloud data filtering method and device, an electronic device, and a storage medium.

[0004] According to an aspect of the present disclosure, a point cloud data filtering method is provided, comprising:

[0005] acquiring a plurality of frames of point cloud data collected in a sensor movement process, and sensor movement trajectory information corresponding to a time period of each frame of point cloud data collected;

[0006] extracting ground points in each frame of point cloud data;

[0007] splicing the extracted ground points in each frame of point cloud data based on each of the sensor movement trajectory information to obtain a ground point cloud model;

[0008] splicing points in each frame of point cloud data based on each of the sensor movement trajectory information to obtain a global point cloud model;

[0009] filtering points in the global point cloud model that are above the ground point cloud model to obtain filtered point cloud data.

[0010] According to another aspect of the present disclosure, a point cloud data filtering device is provided, comprising:

[0011] a data acquisition module configured to acquire a plurality of frames of point cloud data collected in a sensor movement process, and sensor movement trajectory information corresponding to a time period of each frame of point cloud data collected;

[0012] a ground point extraction module configured to extract ground points in each frame of point cloud data;

[0013] a ground point splicing module configured to splice the extracted ground points in each frame of point cloud data based on each of the sensor movement trajectory information to obtain a ground point cloud model;

[0014] The point cloud splicing module is configured to splice points in each frame of point cloud data based on the motion trajectory information of each sensor to obtain a global point cloud model.

[0015] The point cloud filtering module is configured to filter points in the global point cloud model that are above the ground point cloud model to obtain filtered point cloud data.

[0016] According to another aspect of the present disclosure, an electronic device is provided, comprising:

[0017] at least one processor; and

[0018] a memory in communication with the at least one processor; wherein

[0019] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of the present disclosure.

[0020] According to another aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to perform the method of any one of the present disclosure.

[0021] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method of any one of the present disclosure.

[0022] The embodiments of the present disclosure achieve filtering of dynamic point clouds.

[0023] It should be understood that the contents described in this part are not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent through the following description. BRIEF DESCRIPTION OF DRAWINGS

[0024] The accompanying drawings are used to better understand the present scheme and do not limit the present disclosure. Among them:

[0025] Figure 1 is a schematic diagram of a point cloud data filtering method according to the present disclosure;

[0026] Figure 2 is another schematic diagram of a point cloud data filtering method according to the present disclosure;

[0027] Figure 3 is a schematic diagram of a surface feature point determination method according to the present disclosure;

[0028] Figure 4 is still another schematic diagram of a point cloud data filtering method according to the present disclosure;

[0029] Figure 5 FIG. 1 is a schematic diagram of a point cloud data filtering device according to the present disclosure;

[0030] Figure 6 FIG. 2 is a block diagram of an electronic device for implementing a point cloud data filtering method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] Exemplary embodiments of the present disclosure are described herein below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to assist in the understanding thereof. These should be considered in a descriptive sense only and not for purposes of limiting the scope of the present disclosure. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the principles of the disclosure and, as such, within the scope of the disclosure. Similarly, it will be appreciated that, in the case of conjugated or similar structures, specific functional details are not necessarily limited to the illustrated examples.

[0032] In the process of drawing a high-precision map, for the collected point cloud data, since it is not necessary to record the point cloud data corresponding to dynamic objects, and the point cloud data corresponding to dynamic objects will cause certain interference to the splicing of the point cloud data corresponding to non-dynamic objects, affecting the accuracy of point cloud splicing, and further affecting the accuracy of high-precision map drawing, therefore, it is necessary to filter the point cloud data corresponding to dynamic objects in the collected point cloud data. The dynamic objects are, for example, vehicles, pedestrians, etc.

[0033] In the related art, for the collected point cloud data, a classification method of deep learning is used to classify the point cloud data, such as classifying dynamic point clouds and non-dynamic point clouds in the point cloud data, and then filtering the dynamic point clouds according to the classification results. However, using the classification method of deep learning requires collecting and labeling a large amount of sample data for training of a deep learning model, which is relatively high in implementation cost.

[0034] To realize low-cost dynamic point cloud filtering, the present disclosure provides a point cloud data filtering method, which acquires multiple frames of point cloud data collected in a sensor motion process, and sensor motion trajectory information corresponding to a time period of each frame of point cloud data; for each frame of point cloud data, extracts ground points in the frame of point cloud data; based on each of the sensor motion trajectory information, splices the extracted ground points in each frame of point cloud data to obtain a ground point cloud model; based on each of the sensor motion trajectory information, splices the points in each frame of point cloud data to obtain a global point cloud model; filters the points in the global point cloud model that are above the ground point cloud model to obtain filtered point cloud data.

[0035] In the embodiments of the present disclosure, the ground points in each frame of point cloud data are spliced to obtain a ground point cloud model, and the points in each frame of point cloud data are spliced to obtain a global point cloud model, which are completed based on the sensor motion trajectory information of the corresponding time period of each frame of collected point cloud data, so that the ground points in the ground point cloud model and the points in the global point cloud model are in the same coordinate system. By filtering the points in the global point cloud model above the ground point cloud model, dynamic point cloud filtering can be realized without the need for a large amount of sample data to train the model, thereby realizing effective filtering of dynamic point cloud on the basis of reducing cost.

[0036] The point cloud data filtering method provided by the embodiments of the present disclosure will be described in detail below.

[0037] The point cloud data filtering method provided by the embodiments of the present disclosure can be applied to electronic devices such as server devices, intelligent terminal devices, and the like. In one example, it can be applied to a point cloud filtering application program in an electronic device or a mapping system, etc. The point cloud data filtering method provided by the embodiments of the present disclosure can be applied to point cloud data filtering in technical fields such as high-precision maps, autonomous driving, artificial intelligence, etc.

[0038] Referring to Figure 1 , Figure 1 A flowchart of a point cloud data filtering method provided by the embodiments of the present disclosure includes the following steps:

[0039] S101, acquiring a plurality of frames of point cloud data collected in a sensor motion process, and sensor motion trajectory information corresponding to a time period of each frame of collected point cloud data.

[0040] In one example, the point cloud data can be collected by a UAV or a vehicle carrying a sensor to obtain a plurality of frames of point cloud data collected in a sensor motion process. Then, the electronic device acquires a plurality of frames of point cloud data collected in a sensor motion process, and sensor motion trajectory information corresponding to a time period of each frame of collected point cloud data.

[0041] The sensor can be, for example, a LiDAR (Laser Radar) sensor or an image sensor, etc. In the embodiments of the present disclosure, the LiDAR sensor is taken as an example for description, and the sensor motion trajectory information can be a function information of the LiDAR sensor pose state with respect to time.

[0042] S102, extracting ground points in each frame of point cloud data.

[0043] For each frame of acquired point cloud data, the ground points in the frame of point cloud data can be extracted by a direct calculation method (such as determining the ground points by calculating the change in the pitch angle of adjacent points), a plane fitting method (determining the ground points by fitting a plane), a deep learning method, or a filtering method (filtering the points above and below the ground points).

[0044] In a possible implementation, the implementation process of extracting the ground points in each frame of point cloud data can include: for each frame of point cloud data, extracting the ground points in the frame of point cloud data by using a RANSAC (Random Sample Consensus) method of a plane.

[0045] For each frame of acquired point cloud data, the ground points in the frame of point cloud data are extracted by using a RANSAC method of a plane. The principle of RANSAC is that several points are randomly selected from each frame of original point cloud data to form a plane, when the point cloud is close to the plane, the points are added to the plane, and the iteration is continuously performed, and finally the number of plane points reaches a certain number, and the extraction of the plane points is completed.

[0046] In the embodiments of the present disclosure, the RANSAC method of the plane can efficiently extract the ground points in each frame of point cloud data.

[0047] In S103, the ground points in each frame of extracted point cloud data are spliced based on the motion trajectory information of each sensor, to obtain a ground point cloud model.

[0048] In S104, the points in each frame of point cloud data are spliced based on the motion trajectory information of each sensor, to obtain a global point cloud model.

[0049] The multiple frames of point cloud data collected by the sensor in the motion process should be in the same spatial coordinate system as the sensor, and the motion trajectories of the sensors for collecting different frames of point cloud data are different. Since the sensor is in a motion state during the process of collecting the point cloud data, the spatial coordinates of the sensor are changing, and thus the spatial coordinates of the points in the multiple frames of acquired point cloud data are also changing with the sensor. Therefore, in the embodiments of the present disclosure, the ground points in each frame of extracted point cloud data are projected in a specified coordinate system in time sequence according to the motion trajectory information of the sensor in the corresponding time period of each frame of point cloud data, to obtain a ground point cloud model in the specified coordinate system, and the points in each frame of point cloud data are projected in the specified coordinate system in time sequence according to the motion trajectory information of the sensor in the corresponding time period of each frame of point cloud data, to obtain a global point cloud model in the specified coordinate system.

[0050] The ground point cloud model includes ground points, and the global point cloud model includes ground points and non-ground points. For example, the non-ground points can include buildings, vehicles, pedestrians, fences, and the like, the ground points refer to points on the road surface, and the ground point cloud model refers to a road surface point cloud model.

[0051] In a possible implementation, the sensor motion trajectory information includes pose state information of the sensor at different time points (i.e., information of the sensor pose state as a function of time), and then the mapping relationship between the coordinate system in which the sensor is located and the global coordinate system at different time points can be determined according to the pose state information of the sensor at different time points.

[0052] The sensor is in a motion state during the process of collecting point cloud data, and the pose state of the sensor at different time points is different. The mapping relationship between the coordinate system in which the sensor is located and the global coordinate system at different time points can be determined by projecting the pose state of the sensor at different time points into the global coordinate system.

[0053] The global coordinate system is a specified coordinate system. For example, the global coordinate system can be a coordinate system with a real coordinate of a specified target ground object as a coordinate origin. The target ground object can be set according to actual needs, and the disclosure embodiments are not limited in this regard.

[0054] In one example, the pose state information can include spatial coordinate information (horizontal axis, vertical axis, and vertical axis) and orientation information (direction angle relative to the horizontal axis, vertical axis, and vertical axis, respectively), and the mapping relationship between the coordinate system in which the sensor is located and the global coordinate system at different time points can include the mapping relationship between the spatial coordinates (horizontal axis, vertical axis, and vertical axis) and the orientation (direction angle relative to the horizontal axis, vertical axis, and vertical axis, respectively) of the coordinate system in which the sensor is located and the global coordinate system at different time points.

[0055] Accordingly, the step S103 of splicing the ground points in the extracted frame point cloud data to obtain the ground point cloud model based on the sensor motion trajectory information can include: projecting the ground points in the extracted frame point cloud data into the global coordinate system in time sequence according to the mapping relationship to obtain the ground point cloud model.

[0056] The step S104 of splicing the points in the frame point cloud data to obtain the global point cloud model based on the sensor motion trajectory information can include: projecting the points in the frame point cloud data into the global coordinate system in time sequence according to the mapping relationship to obtain the global point cloud model.

[0057] According to the mapping relationship between the coordinate system of the sensor at different time and the global coordinate system, the ground points in the extracted point cloud data of each frame are projected into the global coordinate system in time sequence to splice all the extracted ground points and obtain the ground point cloud model. According to the mapping relationship between the coordinate system of the sensor at different time and the global coordinate system, the points in the point cloud data of each frame are projected into the global coordinate system in time sequence to splice all the obtained point cloud data and obtain the global point cloud model, so that the ground points in the ground point cloud model and the points in the global point cloud model are in the global coordinate system, thereby more accurately removing the dynamic points in the global point cloud model.

[0058] In the embodiments of the present disclosure, the mapping relationship between the coordinate system of the sensor at different time and the global coordinate system is determined, according to the mapping relationship, the ground points in the extracted point cloud data of each frame are projected into the global coordinate system in time sequence to obtain the ground point cloud model, and the points in the point cloud data of each frame are projected into the global coordinate system in time sequence to obtain the global point cloud model, so that the ground points in the ground point cloud model and the points in the global point cloud model are in the global coordinate system, thereby more accurately removing the dynamic points in the global point cloud model.

[0059] The step S103 and the step S104 can be executed synchronously or asynchronously, and the embodiments of the present disclosure do not limit this.

[0060] Referring to Figure 1 S105, filtering the points in the global point cloud model above the ground point cloud model to obtain the filtered point cloud data.

[0061] Dynamic objects such as vehicles, pedestrians, etc. are usually above the road surface, that is, the point cloud data corresponding to the dynamic objects should be above the ground point cloud data. The ground points in the ground point cloud model and the points in the global point cloud model are in the global coordinate system, and the points in the global point cloud model above the ground point cloud model are determined as dynamic points to be filtered, that is, the point cloud data after filtering the dynamic points is obtained.

[0062] In a possible implementation, the implementation process of the step S105 filtering the points in the global point cloud model above the ground point cloud model to obtain the filtered point cloud data includes: filtering the points in the global point cloud model and the ground point cloud model with the same horizontal axis coordinate and vertical axis coordinate and different vertical axis coordinates to obtain the filtered point cloud data.

[0063] The ground points in the ground point cloud model and the points in the global point cloud model are in the global coordinate system. The points in the global point cloud model and the ground point cloud model that have the same horizontal axis coordinate and vertical axis coordinate and different vertical axis coordinates (i.e., points above the ground) are determined as dynamic points to be filtered, to obtain the point cloud data after filtering the dynamic points. For example, if the ground is a road surface, the points above the road surface are filtered.

[0064] In the embodiments of the present disclosure, the ground points in the ground point cloud model and the points in the global point cloud model are in the global coordinate system. The points in the global point cloud model and the ground point cloud model that have the same horizontal axis coordinate and vertical axis coordinate and different vertical axis coordinates are filtered, which can effectively filter out the dynamic points above the ground.

[0065] In the embodiments of the present disclosure, the ground points in the extracted frame point cloud data are spliced to obtain the ground point cloud model, and the points in the frame point cloud data are spliced to obtain the global point cloud model, which are both based on the sensor motion trajectory information of the corresponding time period of the collected frame point cloud data. The ground points in the spliced ground point cloud model and the points in the global point cloud model are in the global coordinate system. The points in the global point cloud model above the ground point cloud model are filtered, which can realize the filtering of dynamic point cloud without the need for a large amount of sample data to train the model, thereby realizing the effective filtering of dynamic point cloud on the basis of reducing the cost.

[0066] Referring to Figure 2 , Figure 2 The flowchart of another point cloud data filtering method provided by the embodiments of the present disclosure includes the following steps:

[0067] S201, acquiring a plurality of frames of point cloud data collected in the sensor motion process, and sensor motion trajectory information corresponding to a time period of each frame of point cloud data.

[0068] S202, for each frame of point cloud data, extracting ground points in the frame of point cloud data.

[0069] S203, based on each sensor motion trajectory information, splicing the ground points in the extracted frame point cloud data to obtain a ground point cloud model.

[0070] S204, based on each sensor motion trajectory information, splicing the points in each frame of point cloud data to obtain a global point cloud model.

[0071] The specific implementation process of steps S201-S204 is described above with reference to the implementation process of steps S101-S104, and the embodiments of the present disclosure will not be described here.

[0072] S205, for each frame of point cloud data, the target object contained in the frame of point cloud data is segmented to obtain a point cloud block containing the target object.

[0073] For each frame of point cloud data, the target object contained in the frame of point cloud data can be segmented by using an object segmentation method to obtain a point cloud block containing the target object. The object segmentation method may, for example, be a threshold method, an edge detection method, or a deep learning segmentation model, and the target object may, for example, be a vehicle, a pedestrian, or the like.

[0074] S206, for each point cloud block of the target object, the point cloud block of the target object is tracked in each frame of point cloud data to obtain a tracking result of the point cloud block of the target object.

[0075] Each point cloud block of the target object segmented is tracked in each frame of point cloud data to determine the frame of point cloud data to which each point cloud block of the target object belongs, and a tracking result of each point cloud block of the target object is obtained, thereby realizing tracking of the point cloud block of the same target object in different frames of point cloud data. The tracking result corresponding to a point cloud block of a target object may, for example, contain an identifier of the frame of point cloud data to which the point cloud block of the target object belongs.

[0076] In one example, the point cloud block of the target object can be tracked in each frame of point cloud data using a nearest neighbor search method, a method based on motion speed estimation, or a deep learning method.

[0077] S207, based on the tracking result and the sensor motion trajectory information, determine the points belonging to the surface feature in each frame of point cloud data.

[0078] In one example, the tracking result corresponding to a point cloud block of a target object may, for example, contain an identifier of the frame of point cloud data to which the point cloud block of the target object belongs. Accordingly, based on the tracking result and the sensor motion trajectory information corresponding to the time period of each frame of point cloud data, the sensor motion trajectory information corresponding to the tracking result corresponding to the point cloud block of the target object can be determined, and the point cloud block of the target object can be further spliced based on the determined sensor motion trajectory information. Further, it can be determined whether the spliced result belongs to a surface. In the case where the spliced result belongs to a surface, the points contained in the point cloud block of the target object are points belonging to the surface feature. For example, the points belonging to the surface feature may, for example, be a fence on the road surface.

[0079] The steps S202-S204 and the steps S205-S207 can be executed synchronously or asynchronously, and the present disclosure does not limit the execution of the steps.

[0080] S208, filtering points in the global point cloud model above the ground point cloud model except points belonging to the surface feature, to obtain filtered point cloud data.

[0081] In the embodiments of the present disclosure, the target object contained in each frame of point cloud data is segmented, and the point cloud blocks of the same target object in different frames of point cloud data are tracked, and the points belonging to the surface feature in each frame of point cloud data are further determined based on the tracking result, so as to determine the static points belonging to the surface feature in each frame of point cloud data, thereby avoiding the false filtering of the point cloud data. Moreover, the ground points in each frame of point cloud data are spliced to obtain the ground point cloud model, and the points in each frame of point cloud data are spliced to obtain the global point cloud model, which are both completed based on the sensor motion trajectory information of the corresponding time period of the collected each frame of point cloud data, so that the ground points in the ground point cloud model and the points in the global point cloud model are in the same coordinate system, and the points in the global point cloud model above the ground point cloud model except the points belonging to the surface feature are filtered, without the need of training a model by means of a large amount of sample data, thereby realizing the effective and accurate filtering of the dynamic point cloud of the points belonging to the surface feature on the basis of reducing the cost.

[0082] In a possible implementation, referring to Figure 3 The implementation process of the above step S207, the tracking result and the sensor motion trajectory information, to determine the points belonging to the surface feature in each frame of point cloud data, includes:

[0083] S301, determining the target sensor motion trajectory information corresponding to the tracking result based on the frame point cloud data to which the point cloud block of the target object corresponding to the tracking result belongs.

[0084] S302, splicing the point cloud block of the target object corresponding to the tracking result based on the target sensor motion trajectory information, to obtain the target object point cloud.

[0085] The sensor motion trajectory information includes the pose state information of the sensor at different time points, and the mapping relationship between the coordinate system of the sensor at different time points and the global coordinate system can be determined according to the pose state information of the sensor at different time points. According to the mapping relationship, the points in the point cloud block of the target object corresponding to the tracking result are projected into the global coordinate system in time sequence, so as to splice the point cloud block of the target object, to obtain the target object point cloud.

[0086] S303, determining the shape and thickness of the target object point cloud.

[0087] In one example, the shape of the target object point cloud is detected by a target detection method or a deep learning model method to determine the shape of the target object point cloud, and the thickness of the target object point cloud is calculated by a PCA (Principal Component Analysis) method. For example, the determined shape of the target object point cloud can be a regular shape or an irregular shape, such as a cluster shape.

[0088] In S304, in a case where the shape and the thickness of the target object point cloud satisfy a preset condition, the feature of the target object point cloud is determined as a surface feature, and the points included in the target object point cloud are determined as points belonging to the surface feature.

[0089] In one example, in a case where the shape of the target object point cloud is a regular shape and the thickness is less than a preset threshold, it is determined that the shape and the thickness of the target object point cloud satisfy the preset condition, and at this time, the feature of the target object point cloud is determined as a surface feature, and the points included in the target object point cloud are determined as points belonging to the surface feature. The preset condition and the preset threshold can be set according to requirements. For example, the preset condition can be that the shape of the target object point cloud is a regular shape and the thickness is less than a preset threshold, and the preset threshold can be, for example, 10 cm, 20 cm, or 30 cm.

[0090] In the embodiments of the present disclosure, based on the frame point cloud data to which the point cloud block of the target object corresponding to the tracking result belongs, the target sensor motion trajectory information corresponding to the tracking result is determined, and then based on the target sensor motion trajectory information, the point cloud block of the target object corresponding to the tracking result is spliced to determine the shape and the thickness of the target object point cloud obtained by splicing. According to the shape and the thickness of the target object point cloud, the points belonging to the surface feature are determined to retain the static points belonging to the surface feature in the process of filtering the point cloud data, and the accuracy of the point cloud data filtering is improved.

[0091] Referring to Figure 4 , Figure 4 A flowchart of another point cloud data filtering method provided by the embodiments of the present disclosure includes the following steps:

[0092] In S401, a plurality of frames of point cloud data collected in a sensor motion process and sensor motion trajectory information corresponding to a time period of each frame of point cloud data are obtained.

[0093] The specific implementation process of step S401 is described above with reference to the implementation process of step S101, and the embodiments of the present disclosure will not be described here.

[0094] In S402, for each frame of point cloud data, whether the frame of point cloud data has been motion compensated is determined according to the sensor motion trajectory information corresponding to the frame of point cloud data.

[0095] The acquired multi-frame point cloud data is collected during sensor motion, and the acquired multi-frame point cloud data is motion-compensated point cloud data in the case of motion compensation of the sensor itself, or is non-motion-compensated point cloud data.

[0096] The multi-frame point cloud data collected by the sensor during motion, each frame of point cloud data should be in the same spatial coordinate system as the sensor, and then, whether the frame of point cloud data has been motion-compensated can be determined through the sensor motion trajectory information corresponding to each frame of point cloud data.

[0097] For example, for each frame of point cloud data, whether the coordinates of each point in the frame of point cloud data are consistent with the coordinates of the sensor at different time points can be determined according to the pose state information of the sensor at different time points included in the sensor motion trajectory information corresponding to the frame of point cloud data, if consistent, it is determined that the frame of point cloud data has been motion-compensated.

[0098] S403, in the case that the frame of point cloud data has not been motion-compensated, motion-compensating the frame of point cloud data according to the sensor motion trajectory information corresponding to the frame of point cloud data, and saving the compensation index information of the motion-compensated frame of point cloud data.

[0099] In one example, in the case that the frame of point cloud data has not been motion-compensated, the coordinate transformation relationship between the coordinates of the sensor at different time points is determined according to the pose state information of the sensor at different time points included in the sensor motion trajectory information corresponding to the frame of point cloud data, and then the coordinates of the corresponding points in the frame of point cloud data are transformed one by one using the determined coordinate transformation relationship, to realize motion compensation of the frame of point cloud data, and save the compensation index information of the motion-compensated frame of point cloud data. The compensation index information can be the order index of each point in the frame of point cloud data, etc.

[0100] S404, for each frame of point cloud data, extracting the ground points in the frame of point cloud data.

[0101] S405, based on the sensor motion trajectory information, splicing the extracted ground points in each frame of point cloud data to obtain a ground point cloud model.

[0102] S406, based on the sensor motion trajectory information, splicing the points in each frame of point cloud data to obtain a global point cloud model.

[0103] S407, filtering the points in the global point cloud model that are above the ground point cloud model to obtain filtered point cloud data.

[0104] The specific implementation process of steps S404-S407 can refer to the implementation process of steps S102-S105, which will not be described here again.

[0105] S408, determining the original coordinates of the points in the filtered point cloud data based on the compensation index information of the motion compensation of each frame of point cloud data.

[0106] The point cloud data targeted in steps S404-S407 is the point cloud data after motion compensation, and accordingly, the obtained filtered point cloud data is the point cloud data after motion compensation. Since the compensation index information of the motion compensation of each frame of point cloud data is saved when the motion compensation is performed on each frame of point cloud data, the original coordinates of the points in the filtered point cloud data can be determined based on the compensation index information of the motion compensation of each frame of point cloud data after the point cloud data filtering is completed. This process can be regarded as the inverse process of the motion compensation.

[0107] If it is determined in step S402 that the motion compensation is performed on the frame of point cloud data, steps S403 and S408 do not need to be performed.

[0108] In the embodiments of the present disclosure, the ground points in the extracted frames of point cloud data are spliced to obtain a ground point cloud model, and the points in the frames of point cloud data are spliced to obtain a global point cloud model, which are both based on the sensor motion trajectory information corresponding to the time period of the collected frames of point cloud data. Therefore, the ground points in the ground point cloud model and the points in the global point cloud model are in the same coordinate system, and the points in the global point cloud model above the ground point cloud model are filtered without the need of training a model with a large amount of sample data, thereby achieving effective filtering of dynamic point cloud on the basis of reducing cost. In addition, if it is determined that the point cloud data is not motion compensated, the point cloud data is motion compensated based on the sensor motion trajectory information corresponding to the point cloud data, and the compensation index information is saved, so that the original coordinates of the points in the filtered point cloud data can be determined based on the compensation index information after the point cloud data filtering is completed, thereby achieving the filtering of the original point cloud data.

[0109] The embodiments of the present disclosure also provide a point cloud data filtering device, which can refer to Figure 5 The device comprises:

[0110] The data acquisition module 501 is configured to acquire a plurality of frames of point cloud data collected in the motion process of a sensor, and sensor motion trajectory information corresponding to a time period of each frame of point cloud data.

[0111] The ground point extraction module 502 is configured to extract ground points in each frame of point cloud data.

[0112] The ground point splicing module 503 is configured to splice the ground points in the extracted frame point cloud data based on the sensor motion trajectory information, to obtain a ground point cloud model.

[0113] The point cloud splicing module 504 is configured to splice the points in the frame point cloud data based on the sensor motion trajectory information, to obtain a global point cloud model.

[0114] The point cloud filtering module 505 is configured to filter the points in the global point cloud model that are above the ground point cloud model, to obtain filtered point cloud data.

[0115] In the embodiments of the present disclosure, the ground points in the extracted frame point cloud data are spliced to obtain a ground point cloud model, and the points in the frame point cloud data are spliced to obtain a global point cloud model, both of which are completed based on the sensor motion trajectory information of the corresponding time period of the collected frame point cloud data, so that the ground points in the ground point cloud model and the points in the global point cloud model are in the same coordinate system. Filtering the points in the global point cloud model that are above the ground point cloud model can achieve filtering of dynamic point cloud without the need for training a model with a large amount of sample data, thereby achieving effective filtering of dynamic point cloud on the basis of reducing cost.

[0116] In a possible implementation, the apparatus further includes:

[0117] The target segmentation module is configured to segment, for each frame of point cloud data, a target object included in the frame of point cloud data, to obtain a point cloud block including the target object.

[0118] The target tracking module is configured to track, for each point cloud block of a target object, the point cloud block of the target object in the frame of point cloud data, to obtain a tracking result of the point cloud block of the target object.

[0119] The surface feature point determination module is configured to determine, based on the tracking result and the sensor motion trajectory information, points in the frame of point cloud data that belong to a surface feature.

[0120] The point cloud filtering module 505 is specifically configured to filter the points in the global point cloud model that are above the ground point cloud model, except for the points that belong to the surface feature, to obtain the filtered point cloud data.

[0121] In a possible implementation, the surface feature point determination module is specifically configured to:

[0122] Determine, based on the frame of point cloud data to which the point cloud block of the target object corresponding to the tracking result belongs, target sensor motion trajectory information corresponding to the tracking result.

[0123] Based on the target sensor motion trajectory information, the point cloud of the target object corresponding to the tracking result is spliced to obtain a target object point cloud.

[0124] The shape and thickness of the target object point cloud are determined.

[0125] In a case where the shape and thickness of the target object point cloud satisfy a preset condition, a feature of the target object point cloud is determined as a surface feature, and points included in the target object point cloud are determined as points belonging to the surface feature.

[0126] In a possible implementation, the ground point extraction module 502 is specifically configured to extract, for each frame of point cloud data, ground points in the frame of point cloud data by using a random sample consensus (RANSAC) method of a plane.

[0127] In a possible implementation, the sensor motion trajectory information includes pose state information of the sensor at different time points, and the device further includes:

[0128] The mapping relationship determination module is configured to determine, according to the pose state information of the sensor at different time points, a mapping relationship between a coordinate system of the sensor at different time points and a global coordinate system, the global coordinate system being a specified coordinate system.

[0129] The ground point splicing module 503 is specifically configured to project, according to the mapping relationship, the extracted ground points in each frame of point cloud data into the global coordinate system in time sequence to obtain a ground point cloud model.

[0130] The point cloud splicing module 504 is specifically configured to project, according to the mapping relationship, points in each frame of point cloud data into the global coordinate system in time sequence to obtain a global point cloud model.

[0131] In a possible implementation, the point cloud filtering module 505 is specifically configured to filter points with the same horizontal axis coordinate and vertical axis coordinate but different vertical axis coordinates in the global point cloud model and the ground point cloud model to obtain filtered point cloud data.

[0132] In a possible implementation, the device further includes:

[0133] The data determination module is configured to determine, for each frame of point cloud data, whether the frame of point cloud data has been subjected to motion compensation according to sensor motion trajectory information corresponding to the frame of point cloud data.

[0134] The motion compensation module is configured to, in a case where the frame of point cloud data has not been subjected to motion compensation, perform motion compensation on the frame of point cloud data according to the sensor motion trajectory information corresponding to the frame of point cloud data, and save compensation index information of the motion compensation on the frame of point cloud data.

[0135] The coordinate restoration module is configured to determine original coordinates of the points in the filtered point cloud data based on compensation index information of motion compensation of each frame of point cloud data.

[0136] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information comply with relevant laws and regulations and do not violate public order and good customs. It should be noted that the head model in the present embodiment is not a head model of a specific user and cannot reflect the personal information of a specific user. It should be noted that the two-dimensional face image in the present embodiment comes from a public data set.

[0137] According to embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0138] The electronic device provided by the present disclosure comprises:

[0139] at least one processor; and

[0140] a memory in communication with the at least one processor; wherein

[0141] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of the present disclosure.

[0142] The present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to make a computer execute the method of any one of the present disclosure.

[0143] The present disclosure provides a computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the method of any one of the present disclosure.

[0144] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the present disclosure described and / or claimed in this document.

[0145] As Figure 6As shown, the device 600 includes a computing unit 601 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from the storage unit 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the device 600 can also be stored in the RAM 603. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0146] Various components in the device 600 are connected to the I / O interface 605, including an input unit 606 such as a keyboard, a mouse, etc., an output unit 607 such as various types of displays, speakers, etc., a storage unit 608 such as a magnetic disk, an optical disk, etc., and a communication unit 609 such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0147] The computing unit 601 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 601 performs various methods and processes described above, such as the point cloud data filtering method. For example, in some embodiments, the point cloud data filtering method can be implemented as a computer software program that is tangibly embodied in a machine-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the point cloud data filtering method described above can be performed. Alternatively, in other embodiments, the computing unit 601 can be configured to perform the point cloud data filtering method by any other appropriate means, such as by means of firmware.

[0148] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0149] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, or entirely on a remote machine or server.

[0150] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0151] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0152] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0153] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0154] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, without departing from the desired results of the technology disclosed in the present disclosure, and are not limited herein.

[0155] The specific embodiments described above are not intended to be limiting, and persons skilled in the art will appreciate that various modifications, combinations, sub-combinations and alternatives can be made to the specific embodiments without departing from the spirit and principles of the disclosure. Accordingly, the disclosure is not limited to the specific embodiments described above.

Claims

1. A point cloud data filtering method, comprising: Acquire multiple frames of point cloud data collected during the sensor's movement, as well as sensor motion trajectory information for the corresponding time period of each frame of point cloud data. For each frame of point cloud data, extract the ground points from that frame of point cloud data; Based on the motion trajectory information of each sensor, the ground points in each frame of extracted point cloud data are stitched together to obtain a ground point cloud model. Based on the motion trajectory information of each sensor, the points in each frame of point cloud data are stitched together to obtain a global point cloud model. For each frame of point cloud data, the target objects contained in the frame of point cloud data are segmented to obtain point cloud blocks containing the target objects; For each point cloud block of a target object, the point cloud block of the target object is tracked in each frame of point cloud data to obtain the tracking result of the point cloud block of the target object; Based on the point cloud data of the frame to which the point cloud block of the target object corresponding to the tracking result belongs, the target sensor motion trajectory information corresponding to the tracking result is determined. Based on the target sensor motion trajectory information, the point cloud blocks of the target object corresponding to the tracking result are stitched together to obtain the target object point cloud; Determine the shape and thickness of the point cloud of the target object; When the shape and thickness of the target object point cloud meet the preset conditions, the feature of the target object point cloud is determined as a surface feature, and the points contained in the target object point cloud are determined as points belonging to the surface feature. Filtering points in the global point cloud model that are above the ground point cloud model to obtain filtered point cloud data includes: filtering points in the global point cloud model that are above the ground point cloud model, excluding points belonging to surface features, to obtain filtered point cloud data.

2. The method according to claim 1, wherein, The step of extracting ground points from each frame of point cloud data includes: For each frame of point cloud data, the Random Sample Consensus (RANSAC) method for planes is used to extract ground points from the point cloud data of that frame.

3. The method according to claim 1, wherein, The sensor motion trajectory information includes: the sensor's pose state information at different times. The method further includes: determining the mapping relationship between the sensor's coordinate system and the global coordinate system at different times based on the sensor's pose state information at different times, wherein the global coordinate system is a specified coordinate system. The step of stitching together ground points from each frame of point cloud data based on the motion trajectory information of each sensor to obtain a ground point cloud model includes: According to the mapping relationship, the ground points in each frame of extracted point cloud data are projected into the global coordinate system in a time sequence to obtain a ground point cloud model. The step of stitching together points from each frame of point cloud data based on the motion trajectory information of each sensor to obtain a global point cloud model includes: Based on the mapping relationship, the points in each frame of point cloud data are projected onto the global coordinate system according to the time sequence to obtain the global point cloud model.

4. The method according to claim 3, wherein, The step of filtering points in the global point cloud model that are above the ground point cloud model to obtain filtered point cloud data includes: Points in the global point cloud model and the ground point cloud model that have the same horizontal and vertical coordinates but different vertical coordinates are filtered to obtain filtered point cloud data.

5. The method according to any one of claims 1-4, further comprising: For each frame of point cloud data, determine whether motion compensation has been performed on the point cloud data based on the sensor motion trajectory information corresponding to that frame of point cloud data. If no motion compensation is performed on the point cloud data of this frame, motion compensation is performed on the point cloud data of this frame based on the sensor motion trajectory information corresponding to the point cloud data of this frame, and the compensation index information of motion compensation of the point cloud data of this frame is saved. Based on the compensation index information of motion compensation of each frame of point cloud data, the original coordinates of the points in the filtered point cloud data are determined.

6. A point cloud data filtering device, comprising: The data acquisition module is used to acquire multiple frames of point cloud data collected during the sensor's movement, as well as the sensor's motion trajectory information for the corresponding time period of each frame of point cloud data. The ground point extraction module is used to extract ground points from each frame of point cloud data. The ground point stitching module is used to stitch the ground points in each frame of extracted point cloud data based on the motion trajectory information of each sensor to obtain a ground point cloud model. The point cloud stitching module is used to stitch together the points in each frame of point cloud data based on the motion trajectory information of each sensor to obtain a global point cloud model. The target segmentation module is used to segment the target objects contained in each frame of point cloud data to obtain point cloud blocks containing the target objects. The target tracking module is used to track the point cloud block of each target object in each frame of point cloud data to obtain the tracking result of the point cloud block of the target object. The surface feature point determination module is used to determine the target sensor motion trajectory information corresponding to the tracking result based on the frame point cloud data to which the point cloud block of the target object belongs; Based on the target sensor motion trajectory information, the point cloud blocks of the target object corresponding to the tracking result are stitched together to obtain the target object point cloud; Determine the shape and thickness of the target object point cloud; if the shape and thickness of the target object point cloud meet preset conditions, determine the feature of the target object point cloud as a surface feature, and determine the points contained in the target object point cloud as points belonging to the surface feature; The point cloud filtering module is used to filter points in the global point cloud model that are above the ground point cloud model to obtain filtered point cloud data. Specifically, the point cloud filtering module is used to filter points in the global point cloud model that are above the ground point cloud model, excluding the points that belong to surface features, to obtain filtered point cloud data.

7. The apparatus according to claim 6, wherein, The ground point extraction module is specifically used to extract ground points from each frame of point cloud data using the Random Sample Consensus (RANSAC) method for the plane.

8. The apparatus according to claim 6, wherein, The sensor motion trajectory information includes: sensor pose state information at different times; the device further includes: The mapping relationship determination module is used to determine the mapping relationship between the sensor's coordinate system and the global coordinate system at different times based on the sensor's pose state information at different times. The global coordinate system is a specified coordinate system. The ground point stitching module is specifically used to project the ground points in each frame of extracted point cloud data into the global coordinate system according to the mapping relationship and in a time sequence to obtain a ground point cloud model. The point cloud stitching module is specifically used to project the points in each frame of point cloud data into the global coordinate system according to the mapping relationship and in a time sequence to obtain a global point cloud model.

9. The apparatus according to claim 8, wherein, The point cloud filtering module is specifically used to filter points in the global point cloud model and the ground point cloud model that have the same horizontal and vertical coordinates but different vertical coordinates, to obtain filtered point cloud data.

10. The apparatus according to any one of claims 6-9, further comprising: The data determination module is used to determine whether motion compensation has been performed on each frame of point cloud data based on the sensor motion trajectory information corresponding to that frame of point cloud data. The motion compensation module is used to perform motion compensation on the point cloud data frame based on the sensor motion trajectory information corresponding to the point cloud data frame when no motion compensation has been performed on the point cloud data frame, and to save the compensation index information of the motion compensation of the point cloud data frame. The coordinate restoration module is used to determine the original coordinates of points in the filtered point cloud data based on the compensation index information of motion compensation for each frame of point cloud data.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-5.

Images