Three-dimensional point cloud segmentation method and device, and movable platform

Abstract

A three-dimensional point cloud segmentation method and device, and a movable platform, are used to perform point cloud segmentation on a three-dimensional point cloud collected by the movable platform. The method comprises: obtaining a plurality of candidate points (301) in the three-dimensional point cloud; searching the plurality of candidate points on a v-disparity plane to determine a target candidate point (302) located on a driving road surface of the movable platform among the plurality of candidate points; fitting a model of the driving road surface based on the target candidate point, and performing a second point cloud segmentation on the three-dimensional point cloud based on the model of the driving road surface on a u-disparity plane to obtain a point cloud segmentation result (303).

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to a three-dimensional point cloud segmentation method and device, and a movable platform. Background Art

[0002] During the driving process of the mobile platform, the path planning module on the mobile platform can be used to make decisions and plans for the driving state of the mobile platform (for example, posture and speed). In order for the planning module to complete the decision-making and planning, the point cloud acquisition device on the mobile platform needs to collect the three-dimensional point cloud of the surrounding environment and perform point cloud segmentation to distinguish the ground and obstacles in the three-dimensional point cloud, and further distinguish dynamic objects from static objects from obstacles. Therefore, point cloud segmentation is an important part of making decisions and plans for the driving state of the mobile platform.

[0003] Traditional point cloud segmentation methods generally segment ground points and non-ground points based on the local features of 3D point clouds. However, the above methods will suffer from obvious degradation when processing 3D point clouds at a long distance, and the segmentation accuracy is low. Summary of the invention

[0004] In view of this, the embodiments of the present disclosure propose a three-dimensional point cloud segmentation method and apparatus, and a movable platform to accurately perform point cloud segmentation on the three-dimensional point cloud collected by the movable platform.

[0005] According to a first aspect of an embodiment of the present disclosure, a three-dimensional point cloud segmentation method is provided for performing point cloud segmentation on a three-dimensional point cloud collected by a movable platform, the method comprising: acquiring a plurality of candidate points in the three-dimensional point cloud; searching the plurality of candidate points on a v-disparity plane to determine a target candidate point located on a driving road surface of the movable platform among the plurality of candidate points; fitting a model of the driving road surface based on the target candidate point, and performing a second point cloud segmentation on the three-dimensional point cloud based on the model of the driving road surface on a u-disparity plane to obtain a point cloud segmentation result.

[0006] According to a second aspect of an embodiment of the present disclosure, a three-dimensional point cloud segmentation device is provided, including a processor, wherein the three-dimensional point cloud segmentation device is used to perform point cloud segmentation on a three-dimensional point cloud collected by a movable platform, and the processor is used to execute the following steps: acquiring multiple candidate points in the three-dimensional point cloud; searching the multiple candidate points on a v-disparity plane to determine a target candidate point among the multiple candidate points that is located on a driving road surface of the movable platform; fitting a model of the driving road surface based on the target candidate point, and performing a second point cloud segmentation on the three-dimensional point cloud based on the model of the driving road surface on a u-disparity plane.

[0007] According to a third aspect of an embodiment of the present disclosure, a movable platform is provided, characterized in that it includes: a shell; a point cloud acquisition device, disposed on the shell and used to acquire a three-dimensional point cloud; and a three-dimensional point cloud segmentation device, disposed in the shell and used to execute the method described in any embodiment of the present disclosure.

[0008] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described in any embodiment of the present disclosure is implemented.

[0009] By applying the embodiment scheme of the present disclosure, candidate points are first searched on the v-disparity plane to determine a target candidate point located on the driving road surface of the movable platform among the multiple candidate points, and then a model of the driving road surface is fitted based on the target candidate point. The model is used as a reference for point cloud segmentation, and a second point cloud segmentation is performed on the three-dimensional point cloud based on the model of the driving road surface on the u-disparity plane, thereby improving the accuracy of point cloud segmentation and allowing areas in the three-dimensional point cloud that are difficult to segment, such as slopes and distant areas, to be accurately segmented. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 is a schematic diagram of a point cloud segmentation process in some embodiments.

[0012] Figure 2 It is a schematic diagram of the decision-making and planning process during the driving process of a movable platform in some embodiments.

[0013] Figure 3 It is a flowchart of the point cloud segmentation method according to an embodiment of the present disclosure.

[0014] Figure 4A and Figure 4B They are respectively schematic diagrams of the UVD coordinate system of the embodiments of the present disclosure.

[0015] Figure 5 It is a schematic diagram of the projection process of the u-disparity plane of the embodiment of the present disclosure.

[0016] Figure 6 Schematic diagram of the relationship between parallax and depth according to an embodiment of the present disclosure.

[0017] Figure 7is a schematic diagram of a point cloud segmentation device according to an embodiment of the present disclosure.

[0018] Figure 8 is a schematic diagram of a movable platform according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0020] The terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0021] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0022] During the driving process of the mobile platform, the path planning module on the mobile platform can be used to make decisions and plans for the driving state of the mobile platform. Among them, point cloud segmentation is an important part of making decisions and plans for the driving state of the mobile platform. Figure 1As shown, it is a schematic diagram of the point cloud segmentation process of some embodiments. In step 101, a three-dimensional point cloud can be collected by a point cloud collection device on a movable platform. Then, in step 102, for a movable platform traveling on the ground (such as an unmanned vehicle), the collected three-dimensional point cloud can be ground segmented, that is, the three-dimensional points in the three-dimensional point cloud are divided into ground points and non-ground points. For other types of movable platforms (such as mobile robots), the collected three-dimensional point cloud can be segmented to divide the three-dimensional points in the three-dimensional point cloud into points on the road surface on which the movable platform travels and points not on the road surface on which the movable platform travels. For ease of description, the following description is based on the road surface as the ground. In step 103, if a three-dimensional point is a ground point, step 104 is executed to add a ground point label to the three-dimensional point, otherwise step 105 is executed to perform dynamic and static segmentation on the three-dimensional point, that is, to segment the three-dimensional point into a static point that is stationary and a dynamic point that moves. In step 106, if a 3D point is a static point, step 107 is executed to add a static point label to the 3D point, otherwise step 108 is executed to add a dynamic point label to the 3D point, and the 3D point cloud with labels is output to the downstream module in step 109. Among them, all or part of the 3D points in the 3D point cloud can be labeled. The label can include at least one of a first label for characterizing whether the 3D point is a ground point and a second label for characterizing whether the 3D point is a static point, and can also include a label for characterizing other information of the 3D point.

[0023] The downstream module may be a planning module on the mobile platform, such as an electronic control unit (ECU), a central processing unit (CPU), etc. After receiving the labeled three-dimensional point cloud, the planning module may make a decision plan for the driving state of the mobile platform based on the labels of the three-dimensional points. The driving state may include at least one of the position and speed of the mobile platform. Figure 2As shown, it is a schematic diagram of the decision-making planning process of some embodiments. In step 201 and step 202, the planning module can receive a three-dimensional point cloud and read the label carried in the three-dimensional point cloud. In step 203, it can be determined based on the label whether the three-dimensional point in the three-dimensional point cloud is a point on the road surface (e.g., the ground) on which the movable platform is traveling. Taking the ground point as an example, if it is, step 204 is executed to identify the three-dimensional point belonging to the lane line from the ground point, and determine the posture of the movable platform according to the direction of the lane line, so that the movable platform can travel along the direction of the lane line. If it is a non-ground point, step 205 is executed to determine whether the non-ground point is a static point. If so, step 206 is executed to determine the posture of the movable platform according to the orientation of the static point. For example, it is determined whether the static point is on a pre-planned driving path. If so, the path is replanned to avoid collision between the movable platform and the static point. If the non-ground point is a dynamic point, step 207 is executed to determine at least one of the posture and speed of the movable platform according to the orientation and speed of the static point. For example, if the dynamic point is on the pre-planned driving path of the movable platform, and the moving speed of the dynamic point is less than or equal to the moving speed of the movable platform, the movable platform is controlled to decelerate, or the posture of the movable platform is adjusted so that the movable platform can bypass the dynamic point. For another example, the movable platform can be controlled to travel at the same speed as the dynamic point.

[0024] It can be seen that point cloud segmentation is an important part of decision-making and planning for the driving state of a movable platform. Accurate point cloud segmentation helps to make accurate decision-making and planning for the driving state of a movable platform. The current point cloud segmentation method is mainly based on local features. Specifically, the three-dimensional point cloud is transformed into the xyz space, and rasterization or proximity search is performed in the xyz space. Find the adjacent points around the candidate point, and determine the probability that the candidate point belongs to the ground point based on the thickness, height, and normal vector of the adjacent point cloud. This method will experience obvious degradation when processing three-dimensional point clouds that are far away, the segmentation accuracy is low, and it is difficult to establish a global ground model. It is impossible to make correct judgments for non-ground planes.

[0025] Based on this, the present disclosure provides a three-dimensional point cloud segmentation method for segmenting a three-dimensional point cloud collected by a movable platform, such as Figure 3 As shown, the method includes:

[0026] Step 301: Acquire multiple candidate points in the three-dimensional point cloud;

[0027] Step 302: searching the plurality of candidate points on the v-disparity plane to determine a target candidate point located on the road surface on which the movable platform can travel among the plurality of candidate points;

[0028] Step 303: Fitting a model of the driving road surface based on the target candidate points, and performing a second point cloud segmentation on the three-dimensional point cloud based on the model of the driving road surface on the u-disparity plane to obtain a point cloud segmentation result.

[0029] In step 301, a point cloud acquisition device (such as a laser radar, a visual sensor, etc.) on a movable platform may acquire a three-dimensional point cloud. The movable platform may be an unmanned vehicle, an unmanned aerial vehicle, an unmanned ship, a movable robot, etc.

[0030] The candidate points may be some or all of the points in the three-dimensional point cloud. Optionally, the three-dimensional point cloud may be semantically segmented, and multiple candidate points in the three-dimensional point cloud may be obtained based on the semantic segmentation result. Through semantic segmentation, the categories of multiple three-dimensional points in the three-dimensional point cloud may be obtained, such as vehicle category, traffic light category, pedestrian category, lane line category, etc. Then, the candidate points may be determined based on the category of the three-dimensional points. For example, the three-dimensional points of the lane line category are determined as candidate points.

[0031] Optionally, the three-dimensional point cloud can also be pre-segmented on the u-disparity plane, and candidate points can be determined based on the results of the pre-segmentation. Specifically, multiple candidate points in the three-dimensional point cloud can be obtained based on the projection density of the three-dimensional point cloud on the u-disparity plane and the first reference projection density of the plane model of the driving road surface on the u-disparity plane. In this embodiment, the driving road surface of the movable platform is first assumed to be a plane, the first reference projection density is determined based on the plane, and then pre-segmentation is performed based on the first reference projection density to determine the candidate points on the driving road surface of the movable platform. Pre-segmentation on the u-disparity plane can improve the signal-to-noise ratio of the candidate points, so that the candidate points can be selected in long-distance areas (with less signal), and improve the distance and accuracy of the search on the v-disparity plane.

[0032] The coordinate axes in the UVD space can be determined based on the direction of the road surface on which the movable platform is traveling. Figure 4A In the example, the movable platform 401 is traveling on the horizontal ground 402, and the u-axis, v-axis, and d-axis (i.e., the disparity axis) can be coordinate axes on the ground perpendicular to the traveling direction of the movable platform, coordinate axes on the ground parallel to the traveling direction of the movable platform, and coordinate axes in the height direction of the movable platform (i.e., vertically upward). Figure 4BIn the figure, the movable platform 404 is a glass-wiping robot traveling on the vertical glass plane 403, and the u-axis, v-axis, and d-axis can be coordinate axes on the glass plane perpendicular to the traveling direction of the movable platform, coordinate axes on the glass plane parallel to the traveling direction of the movable platform, and coordinate axes in the height direction (i.e., horizontal direction) of the movable platform, respectively. In addition, each coordinate axis of the uvd space can also point to other directions, and the specific direction can be set according to actual needs, and the present disclosure does not limit this.

[0033] In some embodiments, the u-disparity plane may be pre-divided into a plurality of grids. In a first grid among the plurality of grids, if the ratio of the projection density to the first reference projection density is greater than or equal to a first preset ratio, a point in the three-dimensional point cloud projected into the first pixel grid is determined as a candidate point. The first preset ratio is greater than 1.

[0034] like Figure 5 As shown, the u-disparity plane can be pre-divided into multiple grids. In order to facilitate the comparison of projection density, the size of each grid can be the same. Each black dot represents a projection point of a 3D point in the 3D point cloud on the u-disparity plane. The number of projection points in a grid is equal to the number of 3D points in the 3D point cloud projected to the grid. The projection density in a grid can be determined as the ratio of the number of projection points in the grid to the area of ​​the grid. Taking a vehicle traveling on the road as an example, when the vehicle is traveling, there is no obstacle in the area d1 to d2 away from the vehicle (called the first area). Then, all the 3D points in the first area are 3D points on the road, and the plane where they are located is parallel to the driving direction of the vehicle (that is, the direction of the disparity coordinate axis) or at a small angle. These 3D points extend along the disparity coordinate axis, and the parallax variation range is large, that is, the density of the 3D points in the first area is small. In the area d3 to d4 away from the vehicle (called the second area), there is an obstacle. The plane where the obstacle is located is generally perpendicular to the driving direction of the vehicle or at a large angle, which will hinder the vehicle from moving forward. That is to say, the parallax variation range of the 3D points in the second area is small, and the density of the 3D points in the second area is large. Therefore, as long as the first reference density of the movable platform's driving plane is known, it can be roughly inferred whether a 3D point is a point on the driving plane or a point outside the driving plane (such as an obstacle).

[0035] Furthermore, since the actual driving road surface may not be a plane, the preset ratio λ (i.e., the redundancy of the segmentation) is set to a value greater than 1, thereby providing a certain amount of redundancy and reducing the selection error of the candidate points. Among them, λ can be fixedly set according to the model of the visual sensor, or it can be dynamically set according to the actual application scenario. When the reliability of the visual sensor is low, λ can be set to a larger value, otherwise λ can be set to a smaller value. For example, when the focal length of the visual sensor is long, or the surrounding light is dim, λ can be set to a larger value.

[0036] In some embodiments, the first reference projection density of the plane model on the u-disparity plane is proportional to the disparity value of the point on the plane model. Specifically, the first reference projection density can be determined according to the baseline length of the visual sensor, the end of the first coordinate axis of the plane model in the coordinate system of the visual sensor, and the disparity value of the point on the plane model. Assume that the plane model of the driving road is:

[0037] y=αz+h

[0038] Wherein, α is the slope of the driving road. The ratio of the intercept to the baseline length of the visual sensor can be calculated, and the product of the ratio and the disparity value of the point on the plane model is determined as the first reference projection density. Then the first reference projection density of the point on the plane model on the u-disparity plane can be recorded as:

[0039]

[0040] Where h is the intercept, b is the baseline length, Δd is the density of projection points within a grid on the u-disparity plane, and Δv g is the first reference projection density in the corresponding grid. The first coordinate axis is the coordinate axis in the height direction of the movable platform. For example, when the movable platform is Figure 4A When the movable platform is traveling on the ground shown in FIG. 1 , the first coordinate axis may be a vertically upward coordinate axis; for example, when the movable platform is traveling on the ground shown in FIG. Figure 4B When driving on the glass plane shown, the first coordinate axis may be a coordinate axis in the horizontal direction.

[0041] Using the above principle, it is possible to obtain which points are on the driving road surface when the model of the driving road surface is a plane model, so as to extract the points on the driving road surface as candidate points. It can be seen that the first reference projection density is only related to the intercept, baseline length and disparity value, and has nothing to do with the distance of the ground (z value). Therefore, here the driving road surface of the movable platform is first assumed to be a plane, and then the first reference projection density is determined based on the plane model to perform segmentation on the u-disparity plane to determine the candidate points. On the one hand, the amount of calculation is reduced, and on the other hand, the signal-to-noise ratio of the candidate points is improved, so that the candidate points can be selected when the distance is far (the amount of signal is small), and the subsequent search distance and accuracy on the v-disparity plane are improved, so that areas with greater segmentation difficulty such as slopes and distant areas can also be accurately cut out.

[0042] In some embodiments, due to reasons such as clock reset, the point cloud acquisition device fails to acquire a complete point cloud frame. Therefore, the acquired three-dimensional point cloud may include both valid points and invalid points. In order to improve the reliability of obtaining candidate points, multiple candidate points can be obtained only from valid points in the three-dimensional point cloud. Invalid points can be set to invalid to avoid being selected as candidate points.

[0043] In some embodiments, outlier points can be filtered out from the three-dimensional points to obtain multiple candidate points in the filtered three-dimensional point cloud. Outlier points are points whose value range is outside the valid range. Outlier points can be filtered out from the three-dimensional points by filtering.

[0044] In some embodiments, before obtaining multiple candidate points in the three-dimensional point cloud based on a first projection density of the three-dimensional point cloud on the u-disparity plane and a first reference projection density of the plane model of the driving road surface on the u-disparity plane, preset scale transformation parameters can be obtained, and based on the scale transformation parameters, the u-coordinate values ​​of each three-dimensional point in the three-dimensional point cloud are scaled, and the scale-transformed three-dimensional point cloud is projected onto the u-disparity plane.

[0045] Among them, the scale transformation parameter scale of a 3D point is used to enlarge or reduce the u coordinate value of the 3D point. When the scale transformation parameter scale is greater than 1, it means that the u coordinate value of the 3D point is enlarged, that is, the 1-row projection point on the u-disparity plane is mapped to the scale row after transformation. When the scale transformation parameter scale is less than 1, it means that the u coordinate value of the 3D point is reduced, that is, the scale row projection point on the u-disparity plane is mapped to the first row after transformation.

[0046] The value of the scale transformation parameter of a three-dimensional point corresponds to the u coordinate value of the three-dimensional point. For example, if the u coordinate value of the first three-dimensional point in the three-dimensional point cloud is less than the first preset coordinate value, the scale transformation parameter of the first three-dimensional point is greater than 1. For another example, if the u coordinate value of the first three-dimensional point in the three-dimensional point cloud is greater than or equal to the second preset coordinate value, the scale transformation parameter of the first three-dimensional point is less than 1. Among them, the first preset coordinate value may be less than or equal to the second preset coordinate value. Further, if the first preset coordinate value is less than the second preset coordinate value, then when the u coordinate value of the first three-dimensional point in the three-dimensional point cloud is greater than or equal to the first preset coordinate value and less than the second preset coordinate value, the scale transformation parameter of the first three-dimensional point is equal to 1.

[0047] Furthermore, multiple scale transformation parameters greater than 1 and / or multiple scale transformation parameters less than 1 may also be set. For example, if the u coordinate value of the first three-dimensional point is less than the third preset coordinate value, the scale transformation parameter of the first three-dimensional point is set to the first parameter value; if the u coordinate value of the first three-dimensional point is greater than or equal to the third preset coordinate value and less than the first preset coordinate value, the scale transformation parameter of the first three-dimensional point is set to the second parameter value. Wherein, both the first parameter value and the second parameter value are greater than 1, and the first parameter value is greater than the second parameter value, and the third preset coordinate value is less than the first preset coordinate value.

[0048] For another example, if the u coordinate value of the first three-dimensional point is greater than or equal to the fourth preset coordinate value, the scale transformation parameter of the first three-dimensional point is set to the third parameter value, and if the u coordinate value of the first three-dimensional point is less than the fourth preset coordinate value and greater than or equal to the second preset coordinate value, the scale transformation parameter of the first three-dimensional point is set to the fourth parameter value. The fourth preset coordinate value is greater than the second preset coordinate value, and the fourth parameter value is less than the third parameter value.

[0049] Table 1 shows the corresponding relationship between the scale parameter and the number of rows before and after the transformation in some embodiments. Those skilled in the art will understand that what is shown in Table 1 is only an exemplary description and is not intended to limit the present disclosure. For example, when the scale transformation parameter scale is 4.0, 1 row of projection points before the transformation is mapped to 4 rows of projection points after the transformation. For example, the 3rd row of projection points before the transformation is mapped to the 0th to 3rd row of projection points after the transformation, the 4th row of projection points before the transformation is mapped to the 4th to 7th row of projection points after the transformation, and so on.

[0050] It should be noted that the starting row number of the projection points before transformation is 3 because the error corresponding to the point with too small disparity value is large. Therefore, only the points with disparity value greater than or equal to 3 are processed here. Those skilled in the art can understand that, without considering the error, the points before the 3rd row can also be used; in other cases, the starting row number of the projection points before transformation can also be set to a value greater than 3.

[0051] Table 1 Correspondence between scale parameter and number of rows before and after transformation

[0052] scale 4.0 2.0 1.0 0.5 0.25 The transformed row range [0,108) [108,128) [128,148) [148,162) [162,172) The row range before transformation [3,30) [30,40) [40,60) [60,88) [88,128)

[0053] Since the relationship between disparity and depth is:

[0054] z=b*f / d

[0055] Where f is the focal length of the visual sensor used to collect the 3D point cloud, b is the focal length of the visual sensor, d is the parallax value, and z is the depth. It can be seen that the relationship between z and d is an inverse proportional relationship, such as Figure 6 As shown. Here, the u coordinate value is scaled by using the scale transformation parameter to counter the characteristic of nonlinear change in depth corresponding to disparity, compress the high-resolution area in the near distance, and give full play to the sub-pixel accuracy in the far distance, thereby improving the accuracy of point cloud segmentation. In addition, through the above-mentioned scale transformation method, the number of rows of 3D points after the transformation is close to the number of rows before the transformation, avoiding the situation where the scale transformation causes too many rows, thereby greatly increasing the computing power, and achieving a balance between computing power and point cloud segmentation accuracy.

[0056] In step 302, the candidate points selected in step 301 may be projected onto the v-disparity plane. Unlike the u-disparity plane, the v-disparity plane is of equal scale, i.e., the number of rows of projection points on the v-disparity plane corresponds one-to-one to the number of rows of the disparity map, and the number of columns of projection points on the v-disparity plane corresponds one-to-one to the integer value of the valid range of the disparity value.

[0057] The multiple candidate points can be searched on the v-disparity plane to determine a target candidate point located on the road surface of the movable platform among the multiple candidate points. The following uses the dynamic programming method as an example to illustrate the process of determining the target candidate point. In actual applications, other methods can also be used to determine the target candidate point, which will not be described in detail here.

[0058] For each of the plurality of candidate points, a search cost of the candidate point may be determined, and a target candidate point may be determined from the candidate points based on the search cost of the candidate point. If the search cost of the candidate point is less than a preset cost, the candidate point is determined as the target candidate point.

[0059] In some embodiments, the search cost includes a first search cost and a second search cost; wherein, the first search cost is used to characterize whether the target candidate point is observed at the candidate point, and the second search cost is used to characterize whether the candidate point and the neighboring target candidate points of the candidate point are smooth.

[0060] The density cost can be calculated as follows:

[0061]

[0062] p is a point on the v-disparity image, cost is the density cost of the point, and th is related to the parameters of the visual sensor. For example, when the road surface of the movable platform is the ground, the width of the lane line on the ground is about 3 meters. At a depth of 100 meters, the 3-meter-wide area captured by the visual sensor generally includes 5 pixels in a frame of the 3D point cloud image. Therefore, the value of th can be 5. In other cases, th can also be set to other values ​​according to actual conditions.

[0063] After obtaining the target candidate points, in step 303, a model of the driving road surface can be fitted based on the target candidate points. For example, a polynomial fitting can be performed on the target candidate points based on the least squares method to obtain a polynomial model of the driving road surface. The obtained model can be expressed as:

[0064] y=A+B*z+C*z 2 +D*z 3 +E*z 4

[0065] The slope of the tangent line to the model at a point is:

[0066] y'=B+2*C*z+3*D*z 2 +4*E*z 3

[0067] Among them, A, B, C, D and E are all constants, and z is the depth. The above model is only an exemplary description. According to the actual application scenario, the above model can be adjusted to a cubic polynomial model or a quintic polynomial model. Then, the slope of the model of the driving road surface can be obtained; based on the slope, the second reference projection density of the driving road surface on the u-disparity plane is determined; and the second point cloud segmentation of the three-dimensional point cloud is performed based on the second reference projection density.

[0068] The method of performing the second point cloud segmentation on the three-dimensional point cloud based on the second reference projection density is similar to the method of performing the point cloud segmentation based on the first reference projection density, that is, in the second pixel grid on the u-disparity plane, if the ratio of the projection density of the three-dimensional point cloud on the u-disparity plane to the second reference projection density is greater than or equal to the second preset ratio, the points in the three-dimensional point cloud projected into the second pixel grid are segmented as target points on the driving road surface. Here, the second preset ratio can be set to a value greater than or equal to 1, and the second preset value can be the same as or different from the first preset value.

[0069] The second reference projection density may be determined based on the model of the driving road surface, the slope of the model, and the depth of the driving road surface. For example, the product of the slope of the model and the depth of the driving road surface may be calculated, and the difference between the model of the driving road surface and the product may be calculated, and the second reference projection density may be determined based on the ratio of the difference to the baseline length of the visual sensor, as follows:

[0070] Δv g =(yz*y') / b

[0071] The above process performs search and model fitting on the v-disparity plane and performs point cloud segmentation on the u-disparity plane. The two steps are iteratively performed. The segmentation of the u-disparity plane can improve the signal-to-noise ratio of the candidate points on the ground, so that the candidate points can be selected in the long-distance area (with less signal), and the search distance and accuracy on the v-disparity plane are improved. The obtained model can provide important information of benchmark density for u-disparity segmentation, so that areas with greater segmentation difficulty such as slopes and distant areas can also be accurately segmented.

[0072] In some embodiments, each 3D point in the 3D point cloud can be labeled based on the point cloud segmentation result, and a label of a 3D point is used to characterize the category of the 3D point. The category may include a first category and a second category, wherein the first category is used to characterize that the 3D point belongs to a point on the road surface on which the movable platform can travel, and the second category is used to characterize that the 3D point belongs to a point on an obstacle. Furthermore, the category may also include a third category, which is used to characterize that the 3D point is neither a point on the road surface nor a point on an obstacle. The points of the third category may be reflection points, or points whose categories cannot be determined, etc.

[0073] In some embodiments, each 3D point in the 3D point cloud can be labeled based on the point cloud segmentation result and the height of each 3D point in the 3D point cloud. Here, the 3D points are labeled based on the height of the 3D points and the point cloud segmentation result, thereby improving the accuracy of the labels. Specifically, if the height of a 3D point is lower than the height of the driving road surface, the label of the 3D point can be determined as a first label, and the first label is used to characterize the 3D point as a reflection point. If the height of a 3D point is not lower than the height of the driving road surface, the labeling can be further combined with the point cloud segmentation result.

[0074] For example, for each 3D point in the 3D point cloud, a first confidence level that the 3D point is a point on the driving road surface can be determined based on the height of the 3D point; a second confidence level that the 3D point is a point on the driving road surface can be determined based on the point cloud segmentation result; and the 3D point can be labeled based on the first confidence level and the second confidence level of the 3D point.

[0075] In the case where the height of a 3D point is not lower than the height of the driving road surface, if the height of a 3D point is higher, the first confidence that the 3D point is a point on the driving road surface is lower, otherwise the first confidence is higher. If the ratio of the projection density of a 3D point to the second reference projection density is larger, the second confidence that the 3D point is a point on the driving road surface is lower, otherwise the second confidence is higher.

[0076] Different methods can be selected for labeling in combination with different scenarios. For example, when the reliability and accuracy of the label are high, the label of the three-dimensional point can be determined to be a first label when the first confidence and the second confidence of a three-dimensional point are both greater than the corresponding confidence thresholds, and the first label is used to indicate that the three-dimensional point is a point on the driving road. When the reliability and accuracy of the label are low, the label of the three-dimensional point can be determined to be a first label when at least one of the first confidence and the second confidence of a three-dimensional point is greater than the corresponding confidence threshold. The label of the three-dimensional point can also be determined based on other methods, which are not listed here one by one.

[0077] In practical applications, the point cloud segmentation result can be used by a planning unit on the mobile platform to plan the driving state of the mobile platform. For example, the planning unit can determine whether there are obstacles on the driving path based on the labels obtained from the point cloud segmentation result, thereby deciding whether it is necessary to control the speed and posture of the mobile platform to avoid the obstacles.

[0078] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0079] The present disclosure also provides a point cloud segmentation device, including a processor, wherein the processor is configured to perform the following steps:

[0080] Acquire multiple candidate points in the three-dimensional point cloud;

[0081] Searching the multiple candidate points on a v-disparity plane to determine a target candidate point located on a driving road surface of the movable platform among the multiple candidate points;

[0082] A model of the driving road surface is fitted based on the target candidate points, and a second point cloud segmentation is performed on the three-dimensional point cloud based on the model of the driving road surface on a u-disparity plane.

[0083] In some embodiments, the processor is used to: determine all three-dimensional points in the three-dimensional point cloud as candidate points; or perform semantic segmentation on the three-dimensional point cloud and obtain multiple candidate points in the three-dimensional point cloud based on the semantic segmentation result; or obtain multiple candidate points in the three-dimensional point cloud based on the projection density of the three-dimensional point cloud on the u-disparity plane and the first baseline projection density of the plane model of the driving road surface on the u-disparity plane.

[0084] In some embodiments, the processor is used to: in a first pixel grid on the u-disparity plane, if the ratio of the projection density to the first reference projection density is greater than or equal to a first preset ratio, determine the point in the three-dimensional point cloud projected into the first pixel grid as a candidate point, and the first preset ratio is greater than 1.

[0085] In some embodiments, a first reference projection density of the planar model on the u-disparity plane is proportional to a disparity value of a point on the planar model.

[0086] In some embodiments, the three-dimensional point cloud is collected by a visual sensor on the movable platform; the processor is used to: obtain the intercept of the first coordinate axis of the plane model in the coordinate system of the visual sensor, the first coordinate axis being the coordinate axis in the height direction of the movable platform; determine the first reference projection density based on the intercept, the baseline length of the visual sensor and the parallax value of the point on the plane model.

[0087] In some embodiments, the processor is used to: calculate the ratio of the intercept to the baseline length of the visual sensor; and determine the product of the ratio and the disparity value of the point on the plane model as the first reference projection density.

[0088] In some embodiments, the three-dimensional point cloud includes valid points and invalid points; and the processor is used to: obtain multiple candidate points from the valid points in the three-dimensional point cloud.

[0089] In some embodiments, the processor is used to: filter out outlier points from the three-dimensional points; and obtain a plurality of candidate points in the filtered three-dimensional point cloud.

[0090] In some embodiments, the processor is further used to: obtain preset scale transformation parameters; based on the scale transformation parameters, scale the u coordinate value of each three-dimensional point in the three-dimensional point cloud; and project the scale-transformed three-dimensional point cloud onto the u-disparity plane.

[0091] In some embodiments, the scale transformation parameter of a 3D point corresponds to the u-coordinate value of the 3D point.

[0092] In some embodiments, if the u coordinate value of the first three-dimensional point in the three-dimensional point cloud is less than the first preset coordinate value, the scale transformation parameter of the first three-dimensional point is greater than 1; and / or if the u coordinate value of the first three-dimensional point in the three-dimensional point cloud is greater than the second preset coordinate value, the scale transformation parameter of the first three-dimensional point is less than 1.

[0093] In some embodiments, the processor is used to: determine, for each candidate point among the multiple candidate points, a search cost of the candidate point; and determine a target candidate point from the candidate points based on the search cost of the candidate point.

[0094] In some embodiments, the processor is configured to: if the search cost of the candidate point is less than a preset cost, determine the candidate point as a target candidate point.

[0095] In some embodiments, the search cost includes a first search cost and a second search cost; wherein, the first search cost is used to characterize whether the target candidate point is observed at the candidate point, and the second search cost is used to characterize whether the candidate point and the neighboring target candidate points of the candidate point are smooth.

[0096] In some embodiments, the processor is used to: perform polynomial fitting on the target candidate points based on the least squares method to obtain a polynomial model of the driving road surface.

[0097] In some embodiments, the processor is used to: obtain the slope of the model of the driving road surface; determine a second reference projection density of the driving road surface on the u-disparity plane based on the slope; and perform a second point cloud segmentation on the three-dimensional point cloud based on the second reference projection density.

[0098] In some embodiments, the processor is used to: in a second pixel grid on the u-disparity plane, if the ratio of the projection density of the three-dimensional point cloud on the u-disparity plane to the second reference projection density is greater than or equal to a second preset ratio, segment the points in the three-dimensional point cloud projected into the second pixel grid as target points on the driving road surface, and the second preset ratio is greater than or equal to 1.

[0099] In some embodiments, the processor is used to: determine the depth of the driving road surface; and determine the second reference projection density based on a model of the driving road surface, a slope of the model, and the depth of the driving road surface.

[0100] In some embodiments, the three-dimensional point cloud is collected by a visual sensor on the movable platform; the processor is used to: calculate the product of the slope of the model and the depth of the driving road surface; calculate the difference between the model of the driving road surface and the product; and determine the second reference projection density based on the ratio of the difference to the baseline length of the visual sensor.

[0101] In some embodiments, the processor is further used to: label each 3D point in the 3D point cloud based on the point cloud segmentation result, and a label of a 3D point is used to characterize a category of the 3D point.

[0102] In some embodiments, the processor is used to: label each 3D point in the 3D point cloud based on the point cloud segmentation result and the height of each 3D point in the 3D point cloud.

[0103] In some embodiments, the processor is used to: if the height of the three-dimensional point is lower than the height of the driving road surface, determine the label of the three-dimensional point as a first label, and the first label is used to characterize that the three-dimensional point is a reflection point.

[0104] In some embodiments, the processor is used to: for each three-dimensional point in the three-dimensional point cloud, determine a first confidence level that the three-dimensional point is a point on the driving road surface based on the height of the three-dimensional point; determine a second confidence level that the three-dimensional point is a point on the driving road surface based on the point cloud segmentation result; and label the three-dimensional point based on the first confidence level and the second confidence level of the three-dimensional point.

[0105] In some embodiments, the processor is used to: if at least one of the first confidence and the second confidence of the three-dimensional point is greater than a preset confidence, determine that the label of the three-dimensional point is a first label, and the first label is used to indicate that the three-dimensional point is a point on the driving road surface.

[0106] In some embodiments, the three-dimensional point cloud is collected based on a visual sensor or a lidar installed on the movable platform; and / or the point cloud segmentation result is used by a planning unit on the movable platform to plan the driving state of the movable platform.

[0107] The specific embodiments of the method executed by the processor in the point cloud segmentation device of the embodiment of the present disclosure can be found in the aforementioned method embodiments, which will not be described again here.

[0108] Figure 7 A more specific hardware structure diagram of a data processing device provided in an embodiment of this specification is shown, and the device may include: a processor 701, a memory 702, an input / output interface 703, a communication interface 704, and a bus 705. The processor 701, the memory 702, the input / output interface 703, and the communication interface 704 are connected to each other in communication within the device through the bus 705.

[0109] The processor 701 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0110] The memory 702 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 702 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 702 and are called and executed by the processor 701.

[0111] The input / output interface 703 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0112] The communication interface 704 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0113] The bus 705 includes a path for transmitting information between various components of the device (eg, the processor 701 , the memory 702 , the input / output interface 703 , and the communication interface 704 ).

[0114] It should be noted that, although the above device only shows the processor 701, the memory 702, the input / output interface 703, the communication interface 704 and the bus 705, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0115] like Figure 8 As shown, the embodiment of the present disclosure further provides a movable platform 800, including a housing 801; a point cloud acquisition device 802, disposed on the housing 801, for acquiring a three-dimensional point cloud; and a three-dimensional point cloud segmentation device 803, disposed in the housing 801, for executing the method described in any embodiment of the present disclosure. The movable platform 800 may be a drone, an unmanned vehicle, an unmanned ship, a movable robot, or other equipment, and the point cloud acquisition device 802 may be a visual sensor (e.g., a binocular visual sensor, a trinocular visual sensor, etc.) or a laser radar.

[0116] The present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps performed by the second processing unit in the method described in any of the above embodiments are implemented.

[0117] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0118] It can be known from the above description of the implementation mode that the technicians in this field can clearly understand that the embodiments of this specification can be implemented by means of software plus the necessary general hardware platform. Based on such an understanding, the technical solution of the embodiments of this specification can be essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments of this specification.

[0119] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.

[0120] The various technical features in the above embodiments can be arbitrarily combined as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they are not described one by one. Therefore, any combination of the various technical features in the above embodiments also falls within the scope of this disclosure.

[0121] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the disclosure and practicing the specifications disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0122] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0123] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A three-dimensional point cloud segmentation method, characterized in that: For performing point cloud segmentation on a three-dimensional point cloud collected by a movable platform, the method comprises: Acquire multiple candidate points in the three-dimensional point cloud; Searching the multiple candidate points on a v-disparity plane to determine a target candidate point located on a driving road surface of the movable platform among the multiple candidate points; Fitting a model of the driving road surface based on the target candidate points, and performing a second point cloud segmentation on the three-dimensional point cloud based on the model of the driving road surface on a u-disparity plane to obtain a point cloud segmentation result; The performing a second point cloud segmentation on the three-dimensional point cloud based on the model of the driving road surface comprises: Obtaining a slope of a model of the driving road surface; Determining a second reference projection density of the driving road surface on the u-disparity plane based on the slope; Perform a second point cloud segmentation on the three-dimensional point cloud based on the second reference projection density.

2. The method according to claim 1, characterized in that Acquiring multiple candidate points in the three-dimensional point cloud, including: Determine all three-dimensional points in the three-dimensional point cloud as candidate points; or Performing semantic segmentation on the three-dimensional point cloud, and obtaining a plurality of candidate points in the three-dimensional point cloud based on the semantic segmentation result; or Based on the projection density of the three-dimensional point cloud on the u-disparity plane and the first reference projection density of the plane model of the driving road surface on the u-disparity plane, a plurality of candidate points in the three-dimensional point cloud are acquired.

3. The method according to claim 2, characterized in that The acquiring of a plurality of candidate points in the three-dimensional point cloud based on the projection density of the three-dimensional point cloud on the u-disparity plane and the first reference projection density of the plane model of the driving road surface on the u-disparity plane comprises: In the first pixel grid on the u-disparity plane, if the ratio of the projection density to the first benchmark projection density is greater than or equal to a first preset ratio, the point in the three-dimensional point cloud projected into the first pixel grid is determined as a candidate point, and the first preset ratio is greater than 1.

4. The method according to claim 2, characterized in that: The first reference projection density of the plane model on the u-disparity plane is proportional to the disparity value of the point on the plane model.

5. The method according to claim 4, characterized in that The three-dimensional point cloud is acquired by a visual sensor on the movable platform; the first reference projection density is determined based on the following method: Acquire an intercept of a first coordinate axis of the plane model in a coordinate system of the visual sensor, where the first coordinate axis is a coordinate axis in a height direction of the movable platform; The first reference projection density is determined based on the intercept, a baseline length of the vision sensor, and a disparity value of a point on the plane model.

6. The method according to claim 5, characterized in that The determining the first reference projection density based on the intercept, the baseline length of the visual sensor, and the parallax value of the point on the plane model comprises: Calculating a ratio of the intercept to a baseline length of the visual sensor; The product of the ratio and the parallax value of the point on the plane model is determined as the first reference projection density.

7. The method according to claim 1, characterized in that The three-dimensional point cloud includes valid points and invalid points; the step of obtaining a plurality of candidate points in the three-dimensional point cloud includes: A plurality of candidate points are obtained from valid points in the three-dimensional point cloud.

8. The method according to claim 1, characterized in that The obtaining of a plurality of candidate points in the three-dimensional point cloud comprises: Filtering outlier points from the three-dimensional point cloud; A plurality of candidate points in the filtered three-dimensional point cloud is obtained.

9. The method according to claim 2, characterized in that: Before acquiring a plurality of candidate points in the three-dimensional point cloud based on a first projection density of the three-dimensional point cloud on a u-disparity plane and a first reference projection density of the plane model of the driving road surface on the u-disparity plane, the method further includes: Get the preset scale transformation parameters; Based on the scale transformation parameter, scale transformation is performed on the u coordinate value of each three-dimensional point in the three-dimensional point cloud; The scale-transformed three-dimensional point cloud is projected onto the u-disparity plane.

10. The method according to claim 9, characterized in that The scale transformation parameter of a 3D point corresponds to the u coordinate value of the 3D point.

11. The method according to claim 10, characterized in that If the u coordinate value of the first three-dimensional point in the three-dimensional point cloud is less than the first preset coordinate value, the scale transformation parameter of the first three-dimensional point is greater than 1; and / or If the u coordinate value of a first three-dimensional point in the three-dimensional point cloud is greater than or equal to a second preset coordinate value, the scale transformation parameter of the first three-dimensional point is less than 1.

12. The method according to claim 1, characterized in that The searching for the multiple candidate points on the v-disparity plane includes: For each candidate point among the multiple candidate points, determining a search cost of the candidate point; A target candidate point is determined from the candidate points based on the search costs of the candidate points.

13. The method according to claim 12, characterized in that The determining a target candidate point from the candidate points based on the search cost of the candidate point includes: If the search cost of the candidate point is less than the preset cost, the candidate point is determined as the target candidate point.

14. The method according to claim 12, characterized in that The search cost includes a first search cost and a second search cost; wherein the first search cost is used to characterize whether a target candidate point is observed at the candidate point, and the second search cost is used to characterize whether the candidate point and the neighboring target candidate points of the candidate point are smooth.

15. The method according to claim 1, characterized in that The step of fitting the model of the driving road surface based on the target candidate point comprises: A polynomial model of the driving road surface is obtained by performing polynomial fitting on the target candidate points based on the least square method.

16. The method according to claim 1, characterized in that The performing a second point cloud segmentation on the three-dimensional point cloud based on the second reference projection density includes: In the second pixel grid on the u-disparity plane, if the ratio of the projection density of the three-dimensional point cloud on the u-disparity plane to the second reference projection density is greater than or equal to a second preset ratio, the points in the three-dimensional point cloud projected onto the second pixel grid are segmented as target points on the driving road surface, and the second preset ratio is greater than or equal to 1.

17. The method according to claim 1, characterized in that The determining, based on the slope, a second reference projection density of the driving road surface on the u-disparity plane comprises: determining the depth of the driving road surface; The second reference projection density is determined based on a model of the driving road surface, a slope of the model, and a depth of the driving road surface.

18. The method according to claim 17, characterized in that The three-dimensional point cloud is acquired by a visual sensor on the movable platform; the second reference projection density is determined based on the model of the driving road surface, the slope of the model and the depth of the driving road surface, including: Calculating the product of the slope of the model and the depth of the driving road surface; Calculating a difference between the model of the driving road surface and the product; The second reference projection density is determined based on a ratio of the difference to a baseline length of the visual sensor.

19. The method according to claim 1, characterized in that The method further comprises: Based on the point cloud segmentation result, each 3D point in the 3D point cloud is labeled, and a label of a 3D point is used to represent the category of the 3D point.

20. The method according to claim 19, characterized in that The step of labeling each three-dimensional point in the three-dimensional point cloud based on the point cloud segmentation result includes: Label each 3D point in the 3D point cloud based on the point cloud segmentation result and the height of each 3D point in the 3D point cloud.

21. The method according to claim 20, characterized in that The step of labeling each 3D point in the 3D point cloud based on the point cloud segmentation result and the height of each 3D point in the 3D point cloud comprises: If the height of the three-dimensional point is lower than the height of the driving road surface, the label of the three-dimensional point is determined as a first label, and the first label is used to indicate that the three-dimensional point is a reflection point.

22. The method according to claim 20, characterized in that The step of labeling each 3D point in the 3D point cloud based on the point cloud segmentation result and the height of each 3D point in the 3D point cloud comprises: For each 3D point in the 3D point cloud, determining a first confidence level that the 3D point is a point on the driving road surface based on the height of the 3D point; Determining, based on the point cloud segmentation result, a second confidence level that the three-dimensional point is a point on the driving road surface; The three-dimensional point is labeled based on the first confidence level and the second confidence level of the three-dimensional point.

23. The method according to claim 22, characterized in that The step of labeling the three-dimensional point based on the first confidence level and the second confidence level of the three-dimensional point includes: If at least one of the first confidence and the second confidence of the three-dimensional point is greater than a preset confidence, the label of the three-dimensional point is determined to be a first label, and the first label is used to indicate that the three-dimensional point is a point on the driving road surface.

24. The method according to claim 1, characterized in that The three-dimensional point cloud is collected based on a visual sensor or a laser radar installed on the movable platform; and / or The point cloud segmentation result is used by a planning unit on the movable platform to plan the driving state of the movable platform.

25. A three-dimensional point cloud segmentation device, comprising a processor, characterized in that: The three-dimensional point cloud segmentation device is used to perform point cloud segmentation on the three-dimensional point cloud collected by the movable platform, and the processor is used to perform the following steps: Acquire multiple candidate points in the three-dimensional point cloud; Searching the multiple candidate points on a v-disparity plane to determine a target candidate point located on a driving road surface of the movable platform among the multiple candidate points; Fitting a model of the driving road surface based on the target candidate points, and performing a second point cloud segmentation on the three-dimensional point cloud based on the model of the driving road surface on a u-disparity plane to obtain a point cloud segmentation result; When performing a second point cloud segmentation on the three-dimensional point cloud based on the model of the driving road surface, the processor is specifically used to: Obtaining a slope of a model of the driving road surface; Determining a second reference projection density of the driving road surface on the u-disparity plane based on the slope; Perform a second point cloud segmentation on the three-dimensional point cloud based on the second reference projection density.

26. The device according to claim 25, characterized in that The processor is used to: Determine all three-dimensional points in the three-dimensional point cloud as candidate points; or Performing semantic segmentation on the three-dimensional point cloud, and obtaining a plurality of candidate points in the three-dimensional point cloud based on the semantic segmentation result; or Based on the projection density of the three-dimensional point cloud on the u-disparity plane and the first reference projection density of the plane model of the driving road surface on the u-disparity plane, a plurality of candidate points in the three-dimensional point cloud are acquired.

27. The device according to claim 26, characterized in that The processor is used to: In the first pixel grid on the u-disparity plane, if the ratio of the projection density to the first benchmark projection density is greater than or equal to a first preset ratio, the point in the three-dimensional point cloud projected into the first pixel grid is determined as a candidate point, and the first preset ratio is greater than 1.

28. The device according to claim 26, characterized in that The first reference projection density of the plane model on the u-disparity plane is proportional to the disparity value of the point on the plane model.

29. The device according to claim 28, characterized in that The three-dimensional point cloud is acquired by a visual sensor on the movable platform; the processor is used for: Acquire an intercept of a first coordinate axis of the plane model in a coordinate system of the visual sensor, where the first coordinate axis is a coordinate axis in a height direction of the movable platform; The first reference projection density is determined based on the intercept, a baseline length of the vision sensor, and a disparity value of a point on the plane model.

30. The device according to claim 29, characterized in that The processor is used to: Calculating a ratio of the intercept to a baseline length of the visual sensor; The product of the ratio and the parallax value of the point on the plane model is determined as the first reference projection density.

31. The device according to claim 25, characterized in that The three-dimensional point cloud includes valid points and invalid points; the processor is used for: A plurality of candidate points are obtained from valid points in the three-dimensional point cloud.

32. The device according to claim 25, characterized in that The processor is used to: Filtering outlier points from the three-dimensional point cloud; A plurality of candidate points in the filtered three-dimensional point cloud is obtained.

33. The device according to claim 26, characterized in that The processor is further configured to: Get the preset scale transformation parameters; Based on the scale transformation parameter, scale transformation is performed on the u coordinate value of each three-dimensional point in the three-dimensional point cloud; The scale-transformed three-dimensional point cloud is projected onto the u-disparity plane.

34. The device according to claim 33, characterized in that The scale transformation parameter of a 3D point corresponds to the u coordinate value of the 3D point.

35. The device according to claim 34, characterized in that If the u coordinate value of the first three-dimensional point in the three-dimensional point cloud is less than the first preset coordinate value, the scale transformation parameter of the first three-dimensional point is greater than 1; and / or If the u coordinate value of a first three-dimensional point in the three-dimensional point cloud is greater than a second preset coordinate value, the scale transformation parameter of the first three-dimensional point is less than 1.

36. The device according to claim 25, characterized in that The processor is used to: For each candidate point among the multiple candidate points, determining a search cost of the candidate point; A target candidate point is determined from the candidate points based on the search costs of the candidate points.

37. The device according to claim 36, characterized in that The processor is used to: If the search cost of the candidate point is less than the preset cost, the candidate point is determined as the target candidate point.

38. The device according to claim 36, characterized in that The search cost includes a first search cost and a second search cost; wherein the first search cost is used to characterize whether a target candidate point is observed at the candidate point, and the second search cost is used to characterize whether the candidate point and the neighboring target candidate points of the candidate point are smooth.

39. The device according to claim 25, characterized in that The processor is used to: A polynomial model of the driving road surface is obtained by performing polynomial fitting on the target candidate points based on the least square method.

40. The device according to claim 25, characterized in that The processor is used to: In the second pixel grid on the u-disparity plane, if the ratio of the projection density of the three-dimensional point cloud on the u-disparity plane to the second reference projection density is greater than or equal to a second preset ratio, the points in the three-dimensional point cloud projected onto the second pixel grid are segmented as target points on the driving road surface, and the second preset ratio is greater than or equal to 1.

41. The device according to claim 25, characterized in that The processor is used to: determining the depth of the driving road surface; The second reference projection density is determined based on a model of the driving road surface, a slope of the model, and a depth of the driving road surface.

42. The device according to claim 41, characterized in that The three-dimensional point cloud is acquired by a visual sensor on the movable platform; the processor is used for: Calculating the product of the slope of the model and the depth of the driving road surface; Calculating a difference between the model of the driving road surface and the product; The second reference projection density is determined based on a ratio of the difference to a baseline length of the visual sensor.

43. The device according to claim 25, characterized in that The processor is further configured to: Based on the point cloud segmentation result, each 3D point in the 3D point cloud is labeled, and a label of a 3D point is used to represent the category of the 3D point.

44. The device according to claim 43, characterized in that The processor is used to: Label each 3D point in the 3D point cloud based on the point cloud segmentation result and the height of each 3D point in the 3D point cloud.

45. The device according to claim 44, characterized in that The processor is used to: If the height of the three-dimensional point is lower than the height of the driving road surface, the label of the three-dimensional point is determined as a first label, and the first label is used to indicate that the three-dimensional point is a reflection point.

46. ​​The device according to claim 44, characterized in that The processor is used to: For each 3D point in the 3D point cloud, determining a first confidence level that the 3D point is a point on the driving road surface based on the height of the 3D point; Determining, based on the point cloud segmentation result, a second confidence level that the three-dimensional point is a point on the driving road surface; The three-dimensional point is labeled based on the first confidence level and the second confidence level of the three-dimensional point.

47. The device according to claim 46, characterized in that The processor is used to: If at least one of the first confidence and the second confidence of the three-dimensional point is greater than a preset confidence, the label of the three-dimensional point is determined to be a first label, and the first label is used to indicate that the three-dimensional point is a point on the driving road surface.

48. The device according to claim 25, characterized in that The three-dimensional point cloud is collected based on a visual sensor or a laser radar installed on the movable platform; and / or The point cloud segmentation result is used by a planning unit on the movable platform to plan the driving state of the movable platform.

49. A movable platform, characterized in that: include: case; A point cloud acquisition device, disposed on the housing, for acquiring three-dimensional point clouds; as well as A three-dimensional point cloud segmentation device is arranged in the shell and is used to execute the method described in any one of claims 1 to 24.

50. A computer-readable storage medium, characterized in that Computer instructions are stored thereon, and when the instructions are executed by a processor, the method described in any one of claims 1 to 24 is implemented.

51. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 24 is implemented.

Images