Method, apparatus, electronic device and medium for detecting a curb

Abstract

Embodiments of the present disclosure disclose a method, an apparatus, an electronic device, and a medium for detecting a curb. A specific implementation of the method includes: rasterizing a pre-acquired three-dimensional ground point cloud to generate a target raster map, where the three-dimensional ground point cloud includes point cloud data for characterizing a curb, and the target raster map includes at least one of the following: a height raster map, a density raster map; slicing the target raster map according to trajectory points corresponding to the three-dimensional ground point cloud to generate a target raster map slice image; extracting skeleton information of the curb according to the target raster map slice image; post-processing the skeleton information of the curb to generate vector information of the curb, where the vector information of the curb is used to indicate the point cloud data characterizing the curb. This implementation helps to improve the effectiveness of the curb detection method.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and more particularly, to methods, devices, electronic devices, and media for detecting curbs. Background Art

[0002] In the field of autonomous driving, curbs (also known as road shoulders) are elements that interact very frequently with the implementation of autonomous driving technology. Currently, curb detection technologies are mainly divided into two categories: one is to process laser point cloud data frame by frame and detect curb elements by judging the curvature rate in the ground point cloud; the other is to perform object detection by combining image deep learning methods and traditional image processing technologies, and then further extract edges to fit the curb contour.

[0003] However, in the prior art, the technology of identifying curbs based on the slope information in the lidar point cloud processed frame by frame is easily affected by environmental factors. For example, when a vehicle is parked on the curb or blocks the lidar near the acquisition vehicle, since this method runs in real time and cannot supplement the undetected parts of the curb, the accuracy of the finally detected curb will be reduced. In the curb detection algorithm based on image detection, information is lost due to the conversion from three-dimensional point cloud to two-dimensional image; moreover, both the camera and the lidar have external parameters, and there will be cumulative errors when converting to the three-dimensional space through back-projection and pose transformation, affecting the final accuracy. Summary of the Invention

[0004] Embodiments of the present disclosure provide methods, devices, terminals, electronic devices, and media for detecting curbs.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for detecting curbs, the method including: rasterizing a pre-acquired three-dimensional ground point cloud to generate a target raster map, where the three-dimensional ground point cloud includes point cloud data for characterizing curbs, and the target raster map includes at least one of the following: a height raster map, a density raster map; slicing the target raster map according to trajectory points corresponding to the three-dimensional ground point cloud to generate a target raster map slice image; extracting the skeleton information of the curb from the target raster map slice image; post-processing the skeleton information of the curb to generate vector information of the curb, where the vector information of the curb is used to indicate the point cloud data characterizing the curb.

[0006] In some embodiments, the above-mentioned target grid map further includes an intensity grid map; and the extraction of the curb skeleton information from the sliced images of the target grid map includes: fusing the sliced images corresponding to the target grid map to form a target grid map sliced image with a first number of channels; inputting the target grid map sliced image into a pre-trained deep learning model to generate a sliced mask map of the curb; splicing the generated sliced mask maps of the curb according to the corresponding coordinates to form a mask map of the curb; finding connected regions based on the mask map of the curb and performing erosion operations on the connected regions to extract the curb skeleton information.

[0007] In some embodiments, the above-mentioned target grid map includes a height grid map and a density grid map; and the post-processing of the curb skeleton information to generate the curb vector information includes: determining target curb pixel point pairs from the curb skeleton information based on the curb ends indicated by the curb skeleton information and the edges detected based on the Canny operator; and performing the following curb determination steps: determining the gradient direction of the curb according to the positions of the determined target curb pixel point pairs on the density grid map; determining the gradient values corresponding to a second number of candidate pixel points selected along the gradient direction of the curb from the target pixel point according to the pixel values around the pixel point, where the target pixel point is one of the pixel points in the target curb pixel point pair; selecting pixel points from both sides of the target pixel point as target candidate pixel points according to the determined gradient values; selecting the target candidate pixel point corresponding to the target pixel point as the inner edge pixel point of the curb according to the positions of the target candidate pixel points on the height grid map; in response to determining the existence of the next pixel point of the target curb pixel point selected along the curb skeleton information, forming a new target curb pixel point pair with the target curb pixel point and the next pixel point, and continuing to perform the curb determination steps; in response to determining the non-existence of the next pixel point of the target curb pixel point selected along the curb skeleton information, generating the curb vector information according to the generated inner edge pixel points of the curb.

[0008] In some embodiments, the selecting of pixel points from both sides of the target pixel point as target candidate pixel points according to the determined gradient values includes: in response to determining that the maximum value in the determined gradient values is greater than a preset gradient threshold, determining the pixel point corresponding to the maximum value in the determined gradient values as one of the target candidate pixel points; screening out pixel points whose distance from the determined target candidate pixel point is less than a preset first distance threshold; and selecting the pixel point corresponding to the maximum value of the gradient value from the remaining pixel points after screening as the other target candidate pixel point.

[0009] In some embodiments, rasterizing the pre-acquired three-dimensional ground point cloud to generate a target raster map includes: obtaining a three-dimensional point cloud data set, where the three-dimensional point cloud data set includes point cloud data for characterizing a curb; using morphological filtering to separate the ground from the three-dimensional point cloud to obtain a first quasi-ground three-dimensional point cloud; filtering out point cloud data with a distance exceeding a preset second distance threshold according to the positions indicated by the point cloud data in the first quasi-ground three-dimensional point cloud and the distances from the trajectory points corresponding to the three-dimensional point cloud to obtain second quasi-ground three-dimensional point cloud data; performing morphological filtering with a reduced window area on the second quasi-ground three-dimensional point cloud data to obtain third quasi-ground three-dimensional point cloud data; performing statistical filtering on the third quasi-ground three-dimensional point cloud data to generate a ground point cloud; and rasterizing the ground point cloud to generate at least one of the following: a height raster map, a density raster map as the target raster map.

[0010] In a second aspect, an embodiment of the present disclosure provides a device for detecting a curb. The device includes: a rasterizing unit configured to rasterize a pre-acquired three-dimensional ground point cloud to generate a target raster map, where the three-dimensional ground point cloud includes point cloud data for characterizing a curb, and the target raster map includes at least one of the following: a height raster map, a density raster map; a slicing unit configured to slice the target raster map according to the trajectory points corresponding to the three-dimensional ground point cloud to generate a target raster map slice image; an extraction unit configured to extract the skeleton information of the curb according to the target raster map slice image; and a post-processing unit configured to post-process the skeleton information of the curb to generate vector information of the curb, where the vector information of the curb is used to indicate the point cloud data characterizing the curb.

[0011] In some embodiments, the above target raster map further includes an intensity raster map; and the above extraction unit is further configured to: fuse the slice images corresponding to the target raster map to form a target raster map slice image with a first number of channels; input the target raster map slice image into a pre-trained deep learning model to generate a slice mask map of the curb; splice the generated slice mask map of the curb according to the corresponding coordinates to form a mask map of the curb; and find connected regions and perform an erosion operation on the connected regions according to the mask map of the curb to extract the skeleton information of the curb.

[0012] In some embodiments, the above-mentioned target grid map includes a height grid map and a density grid map; and the above-mentioned post-processing unit includes: a determination module configured to determine a pair of target curb pixel points from the curb skeleton information according to the curb end indicated by the curb skeleton information and the edge detected based on the Canny operator; an execution module configured to execute the following curb determination steps: determine the gradient direction of the curb according to the position of the determined pair of target curb pixel points on the density grid map; determine the gradient values corresponding to the second number of candidate pixel points selected along the gradient direction of the curb according to the pixel values around the pixel points, where the target pixel point is one of the pixel points in the pair of target curb pixel points; select pixel points from both sides of the target pixel point as target candidate pixel points according to the determined gradient values; select the target candidate pixel point corresponding to the target pixel point as the inner edge pixel point of the curb according to the position of the target candidate pixel point on the height grid map; in response to determining that there is a next pixel point of the target curb pixel point selected along the curb skeleton information, form a new pair of target curb pixel points with the target curb pixel point and the next pixel point, and continue to execute the above-mentioned curb determination steps; a generation module configured to, in response to determining that there is no next pixel point of the target curb pixel point selected along the curb skeleton information, generate vector information of the curb according to the generated inner edge pixel points of the curb.

[0013] In some embodiments, the above-mentioned execution module is further configured to: in response to determining that the maximum value among the determined gradient values is greater than a preset gradient threshold, determine the pixel point corresponding to the maximum value among the determined gradient values as one of the target candidate pixel points; filter out the pixel points whose distance from the determined target candidate pixel points is less than a preset first distance threshold; select the pixel point corresponding to the maximum value of the gradient value from the remaining pixel points after filtering as the other target candidate pixel point.

[0014] In some embodiments, the above-mentioned rasterization unit is further configured to: obtain a three-dimensional point cloud dataset, where the three-dimensional point cloud dataset includes point cloud data for characterizing the curb; use morphological filtering to separate the ground from the three-dimensional point cloud to obtain a first quasi-ground three-dimensional point cloud; filter out the point cloud data whose distance exceeds a preset second distance threshold according to the position indicated by the point cloud data in the first quasi-ground three-dimensional point cloud and the distance between the trajectory points corresponding to the three-dimensional point cloud to obtain second quasi-ground three-dimensional point cloud data; perform morphological filtering with a reduced window area on the second quasi-ground three-dimensional point cloud data to obtain third quasi-ground three-dimensional point cloud data; perform statistical filtering on the third quasi-ground three-dimensional point cloud data to generate a ground point cloud; rasterize the ground point cloud to generate at least one of the following: a height grid map, a density grid map as the target grid map.

[0015] In a third aspect, embodiments of the present disclosure provide an electronic device, which includes: one or more processors; a storage device storing one or more programs thereon; when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method described in any implementation manner of the first aspect.

[0016] In a fourth aspect, embodiments of the present disclosure provide a computer-readable medium storing a computer program thereon, and when the program is executed by a processor, the method described in any implementation manner of the first aspect is implemented.

[0017] The method, device, electronic device, and medium for detecting a curb provided by the embodiments of the present disclosure generate rasterized ground point clouds and sliced images, then extract the skeleton of the curb based on the sliced images, and post-process the skeleton of the curb to finally generate vector information of the curb. Thus, a curb detection method different from the prior art is provided. Moreover, through the combination of point cloud and image detection, and the process of generating the skeleton first and then performing post-processing, a finer curb edge can be generated, which helps to improve the effectiveness of the curb detection method. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Other features, objects, and advantages of the present disclosure will become more apparent by reading the detailed description of the non-limiting embodiments with reference to the following drawings:

[0019] Figure 1 is an exemplary system architecture diagram to which an embodiment of the present disclosure can be applied;

[0020] Figure 2 is a flowchart of an embodiment of the method for detecting a curb according to the present disclosure;

[0021] Figure 3 is a schematic diagram of an application scenario of the method for detecting a curb according to an embodiment of the present disclosure;

[0022] Figure 4 is a flowchart of another embodiment of the method for detecting a curb according to the present disclosure;

[0023] Figure 5 is a schematic structural diagram of an embodiment of the device for detecting a curb according to the present disclosure;

[0024] Figure 6 is a schematic structural diagram of an electronic device suitable for implementing the embodiments of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] The present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related invention and are not intended to limit the invention. Additionally, it should be noted that for ease of description, only parts related to the relevant invention are shown in the drawings.

[0026] It should be noted that, without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other. The present disclosure will be described in detail below with reference to the drawings and embodiments.

[0027] Figure 1 An exemplary architecture 100 is shown that can apply the method for detecting a curb or the device for detecting a curb of the present disclosure.

[0028] As Figure 1 shown, the system architecture 100 may include a terminal device 101, a network 102, and a server 103. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0029] The terminal device 101 interacts with the server 103 through the network 102 to receive or send messages, etc. The terminal device 101 can be hardware or software. When the terminal device 101 is hardware, it can be a map acquisition vehicle equipped with a lidar, an inertial navigation unit, and a navigation system. When the terminal device 101 is software, it can be installed in the above-listed electronic devices. It can be implemented as multiple software or software modules (such as software or software modules for providing distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0030] The server 103 can be a server that provides various services, such as a background server for processing the point cloud data collected by the terminal device 101. The background server can analyze and process the received point cloud data and generate a processing result (such as the vector information of the curb).

[0031] It should be noted that the server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or it can be implemented as a single server. When the server is software, it can be implemented as multiple software or software modules (such as software or software modules for providing distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0032] It should be noted that the method for detecting a curb provided by the embodiments of the present disclosure is generally executed by the server 103. Correspondingly, the device for detecting a curb is generally provided in the server 103.

[0033] It should be understood that Figure 1 the numbers of the terminal devices, networks, and servers in

[0034] Continuing to refer to Figure 2 , a flowchart 200 of an embodiment of a method for detecting a curbstone according to the present disclosure is shown. The method for detecting a curbstone includes the following steps:

[0035] Step 201, rasterize the pre-acquired three-dimensional ground point cloud to generate a target raster map.

[0036] In this embodiment, the execution subject of the method for detecting a curbstone (such as Figure 1 the server 103 shown) can pre-acquire the three-dimensional ground point cloud through a wired connection method or a wireless connection method. Among them, the above three-dimensional ground point cloud may include point cloud data for characterizing the curbstone.

[0037] As an example, the above three-dimensional ground point cloud may be a point cloud formed after ground point segmentation of the three-dimensional laser point cloud obtained by the lidar on the map data collection vehicle. Generally, an inertial navigation unit and a satellite navigation system may also be installed on the above map data collection vehicle. The above lidar can be used to collect the surrounding environment information during the vehicle driving process. The above inertial navigation unit may include an angular velocity meter and a linear acceleration meter, and is used to cooperate with the satellite navigation system to estimate the 6-degree-of-freedom pose of the above map data collection vehicle. According to the above estimated pose, the lidar point cloud data at each moment is stitched, and finally a complete three-dimensional laser point cloud is obtained. Then, ground point segmentation is performed on the above complete three-dimensional laser point cloud (for example, using the Rand-LA-Net deep learning model) to obtain the three-dimensional ground point cloud.

[0038] In this embodiment, the above execution subject may perform a raster conversion operation on the above pre-acquired three-dimensional ground point cloud in various ways, so as to generate at least one of the following as the target raster map: a height raster map, a density raster map. Among them, the above height raster map is used to characterize the height distribution of the above three-dimensional ground point cloud data. The above density raster map is used to characterize the density distribution of the above three-dimensional ground point cloud data.

[0039] In some optional implementation manners of this embodiment, the above target raster map may further include an intensity raster map. The above intensity raster map is used to characterize the reflection intensity distribution of the above three-dimensional ground point cloud data.

[0040] Step 202, slice the target raster map according to the trajectory points corresponding to the three-dimensional ground point cloud to generate a target raster map slice image.

[0041] In this embodiment, the target grid map is sliced according to the trajectory points corresponding to the three-dimensional ground point cloud obtained in advance in step 201, and the execution subject can generate the target grid map slice image in various ways. Among them, the trajectory points corresponding to the three-dimensional ground point cloud are usually the trajectory points of the device that collects the ground point cloud (such as a vehicle equipped with a lidar point map data collector).

[0042] In this embodiment, as an example, the execution subject can select a trajectory point every 5 meters, so that there is not too large an overlapping area between each slice. Then, the execution subject can use the position of the selected trajectory point as the center of a rectangular frame, and frame a region with a preset size (such as a resolution of 960*640) in the target grid map, so as to generate the target grid map slice image.

[0043] It should be noted that since the principle of slicing is to project the trajectory points onto the grid map. Therefore, when the target grid map is a height grid map or a density grid map, the generated target grid map slice image is a single-channel image, and the pixel range is 0-255. When the target grid map is a height grid map and a density grid map, the generated target grid map slice image is a two-channel image.

[0044] Step 203, extract the skeleton information of the curb according to the target grid map slice image.

[0045] In this embodiment, according to the target grid map slice image generated in the above step 202, the execution subject can extract the skeleton information of the curb in various ways. Among them, the skeleton information of the curb is used to indicate the main body of the curb and its location. As an example, the skeleton of the curb indicated by the skeleton information of the curb can be a polyline passing through multiple points.

[0046] As an example, the execution subject can use a pre-trained image detection model to determine the location of the curb (such as a detection box) from the target grid map slice image generated in the above step 202. Then, the execution subject can further identify key points from the detection box, and then connect the identified key points to generate the skeleton information of the curb.

[0047] In some optional implementation manners of this embodiment, based on the fact that the target grid map further includes an intensity grid map, the execution subject can extract the skeleton information of the curb according to the target grid map slice image according to the following steps:

[0048] The first step is to fuse the slice images corresponding to the target grid map to form a target grid map slice image with a first number of channels.

[0049] In these implementation manners, the above-mentioned execution entity may fuse the sliced images corresponding to the target grid map generated in step 201 to form the target grid map sliced images with a first number of channels.

[0050] In these implementation manners, the above-mentioned execution entity may fuse at least two of the sliced images corresponding to the height grid map, the density grid map, and the intensity grid map to form the target grid map sliced images with a first number of channels. Wherein, the above-mentioned first number may be 2 or 3.

[0051] Second, input the target grid map sliced images into a pre-trained deep learning model to generate the sliced mask map of the curb.

[0052] In these implementation manners, the above-mentioned execution entity may input the target grid map sliced images formed in the above-mentioned first step into a pre-trained deep learning model to generate the sliced mask map of the curb. Wherein, the above-mentioned deep learning model may be various image detection models for detecting curbs, such as the DeeplabV3 model. The DeeplabV3 model can obtain a larger receptive field by performing cascaded dilated convolution and spatial pyramid pooling on the input image, thereby obtaining multi-scale information and improving the segmentation accuracy.

[0053] Third, splice the generated sliced mask maps of the curb according to the corresponding coordinates to form the mask map of the curb.

[0054] In these implementation manners, the above-mentioned execution entity may splice the sliced mask maps of the curb corresponding to a plurality of target grid map sliced images generated in the above-mentioned second step according to the corresponding coordinates to form the mask map of the curb. Wherein, the above-mentioned corresponding coordinates may be, for example, those used when rasterizing the three-dimensional ground point cloud data. As an example, the above-mentioned mask map may be 10800*10800 pixels.

[0055] Fourth, find the connected regions based on the mask map of the curb and perform an erosion operation on the connected regions to extract the skeleton information of the curb.

[0056] In these implementation manners, the above-mentioned execution entity may find the connected regions in the mask map of the curb formed in the above-mentioned third step. Each connected region may be used to represent a complete curb line segment. Then, the above-mentioned execution entity may perform an erosion operation on each of the found connected regions to extract the skeleton information of the curb.

[0057] Based on the above optional implementation methods, this solution can use at least one of the height raster map and the density raster map and the intensity raster map as the input of a pre-trained deep learning model to obtain a slice mask map of the curb. Then, by finding connected components, erosion operations, etc., the skeleton information of the curb is extracted, thus providing a method for extracting the curb skeleton, enriching the extraction methods of the curb skeleton, and improving the extraction accuracy due to the combination of the raster map and the deep learning model.

[0058] Step 204: Post-process the skeleton information of the curb to generate vector information of the curb.

[0059] In this embodiment, the above execution subject can post-process the skeleton information of the curb extracted in step 203 in various ways to generate vector information of the curb. Among them, the above vector information of the curb can be used to indicate the point cloud data representing the curb. It can include information on the position of the point cloud data representing the curb, that is, separating the curb point cloud from the three-dimensional ground point cloud.

[0060] As an example, the above execution subject can use the Canny operator to generate the edge of the curb. Then, the above execution subject can correct the edge of the curb generated by using the Canny operator according to the skeleton information of the curb extracted in step 203, such as supplementing breakpoints or correcting directions, etc., so as to generate vector information of the curb.

[0061] In some optional implementation methods of this embodiment, the above execution subject can post-process the skeleton information of the curb according to the following steps to generate vector information of the curb:

[0062] S1: Determine target curb pixel pairs from the skeleton information of the curb according to the curb end points indicated by the skeleton information of the curb and the edge detected based on the Canny operator.

[0063] In these implementation methods, according to the curb end points indicated by the skeleton information of the curb extracted in step 203 and the edge detected based on the Canny operator, the above execution subject can determine target curb pixel pairs from the above skeleton information of the curb in various ways. As an example, the above execution subject can sequentially select two pixel points from the curb end points (such as the head) indicated by the skeleton information of the curb extracted in the above step 203. Then, the above execution subject can adjust the two selected pixel points based on the edge detected by the Canny operator. For example, expand a 3×3 pixel area centered on the selected pixel points respectively, and select the pixel points falling on the edge detected based on the Canny operator from the above expanded pixel areas respectively, so as to form target curb pixel pairs.

[0064] S2: Execute the following curb determination steps:

[0065] S21. Determine the gradient direction of the curb according to the position of the determined target curb pixel pair on the density grid map.

[0066] In these implementation manners, according to the position of the determined target curb pixel pair on the density grid map in step S1, the above-mentioned execution entity can determine the gradient direction of the curb in various ways. As an example, the above-mentioned execution entity can calculate the direction perpendicular to the line connecting the two pixel points in the above-mentioned target curb pixel pair in the above-mentioned density grid map as the gradient direction of the curb (the direction perpendicular to the curb). Suppose the coordinates of the two pixel points P1 and P2 in the target curb pixel pair are (x1, y1) and (x2, y2) respectively, then the unit gradient direction of the curb can be obtained according to formulas (1) and (2):

[0067]

[0068]

[0069] S22. Determine the gradient values corresponding to the second number of candidate pixels selected along the gradient direction of the curb from the target pixel according to the pixel values around the pixel.

[0070] In these implementation manners, according to the pixel values around the pixel, the above-mentioned execution entity can determine the gradient values corresponding to the second number of candidate pixels selected along the gradient direction of the curb from the target pixel in various ways. Among them, the above-mentioned target pixel is one of the pixel points in the above-mentioned target curb pixel pair.

[0071] As an example, the above-mentioned execution entity can search for the second number of candidate pixels along the gradient and the opposite direction of the gradient around the above-mentioned target pixel. For example, when the set search step size is 1 and the range is 5, the above-mentioned second number can be 10.

[0072] After that, the above-mentioned execution entity can determine the gradient values corresponding to the above-mentioned candidate pixels in various ways. As an example, for the candidate pixel among the above-mentioned second number of candidate pixels, the above-mentioned execution entity can find the candidate pixel P candidate nearby (for example, at a distance of 1 pixel) M (for example, 2) pixel points P neighbor1 and P neighbor2 , and then calculate the sum of the pixel values of the preset regions (for example, 3×3 pixels) centered on P candidate , P neighbor1 and P neighbor2 respectively, such as k1, k2, and k3. After that, according to formula (3), determine the gradient value V of the candidate pixel P candidate as:

[0073]

[0074] Thus, the above-mentioned execution entity can determine the gradient values corresponding to the second number of candidate pixel points.

[0075] S23. Select pixel points from both sides of the target pixel point as target candidate pixel points according to the determined gradient values.

[0076] In these implementation manners, according to the gradient values determined in step S22, the above-mentioned execution entity can select pixel points from both sides of the above-mentioned target pixel point as target candidate pixel points in various ways. As an example, the above-mentioned execution entity can select the pixel points with the largest gradient values from both sides (i.e., the gradient direction and the opposite gradient direction) of the above-mentioned target pixel point as target candidate pixel points.

[0077] S24. Select the target candidate pixel points corresponding to the target pixel point as the inner edge pixel points of the curb according to the positions of the target candidate pixel points on the height grid map.

[0078] In these implementation manners, according to the positions of the target candidate pixel points selected in the above-mentioned step S23 on the height grid map, the above-mentioned execution entity can select the target candidate pixel points corresponding to the target pixel point as the inner edge pixel points of the curb in various ways. Among them, the above-mentioned new curb pixel points can be used to indicate the edge of the curb close to the road side. As an example, the above-mentioned execution entity can select the target candidate pixel points with smaller heights as the inner edge pixel points of the curb according to the heights indicated by the above-mentioned height grid map. As another example, the above-mentioned execution entity can also select the target candidate pixel points with smaller heights as the inner edge pixel points of the curb in a similar manner based on the gradient information. For example, the above-mentioned execution entity can respectively select N pixel points (e.g., sampled at a step size of 1 and a range of 10) along the gradient direction and the opposite direction under the height grid map for the above-mentioned two target candidate pixel points. Then, the above-mentioned execution entity can sum the height values of the selected N pixel points, and then select the pixel point with the smaller height value as the above-mentioned inner edge pixel point of the curb.

[0079] It should be noted that the target candidate pixel points selected through the above-mentioned step S23 are usually located at the two edges of the curb respectively. That is, the two edges are respectively on the side close to the road surface and the side far from the road surface (e.g., close to the sidewalk). Since the curb has height information, when the lidar scans, more points will hit the curb. Therefore, on the density grid map, there are obvious differences between the curb pixels and the two sides. Similarly, on the height grid map, there are also obvious differences between the curb pixels and the two sides.

[0080] S3. In response to determining that the next pixel point of the target curb pixel point selected according to the skeleton information of the curb exists, form a new pair of target curb pixel points with the target curb pixel point and the next pixel point, and continue to execute the curb determination step.

[0081] In these implementation manners, in response to determining that the next pixel point of the target curb pixel point selected according to the skeleton information of the curb exists, the above-mentioned execution entity may form a new pair of target curb pixel points with the target curb pixel point and the next pixel point, and continue to execute the curb determination step. That is, the above-mentioned execution entity may move on the skeleton indicated by the skeleton information of the above-mentioned curb, and provide a reference for the determination of the next inner edge pixel point of the curb by recalculating the gradient formed by the pair of target curb pixel points selected according to the skeleton information.

[0082] S4. In response to determining that the next pixel point of the target curb pixel point selected according to the skeleton information of the curb does not exist, generate vector information of the curb according to the generated inner edge pixel points of the curb.

[0083] In these implementation manners, in response to determining that the next pixel point of the target curb pixel point selected according to the skeleton information of the curb does not exist (that is, it has moved to the other end of the skeleton indicated by the skeleton information of the curb), the above-mentioned execution entity may generate vector information of the curb according to the generated inner edge pixel points of the curb. Through cyclic iteration, the above-mentioned execution entity may determine the curb edge formed by the inner edge pixel points of the curb that match the skeleton indicated by the skeleton information of the above-mentioned curb, so as to generate vector information of the curb.

[0084] Based on the above optional implementation manners, the present solution innovatively utilizes the principle that the gradient direction of the curb in the grid map is exactly perpendicular to the vector direction of the road curb. By screening the gradient direction and moving the points on the curb skeleton, the boundary of the curb is recognized according to the gradient information on the density grid map, thereby improving the recognition accuracy and stability of the boundary of the curb.

[0085] Optionally, based on the above optional implementation manners, according to the determined gradient value, the above-mentioned execution entity may also select pixel points from both sides of the target pixel point as target candidate pixel points according to the following steps:

[0086] S231. In response to determining that the maximum value in the determined gradient values is greater than a preset gradient threshold, determine the pixel point corresponding to the maximum value in the determined gradient values as one of the target candidate pixel points.

[0087] In these implementation manners, in response to determining that the maximum value in the determined gradient values is greater than a preset gradient threshold, it is proved that there are pixel points representing the curb around the above-mentioned target pixel point.

[0088] S232. Screen out the pixels whose distance from the determined target candidate pixel points is less than a preset first distance threshold.

[0089] In these implementation manners, as an example, the above-mentioned preset first distance may be 2 pixels, that is, screen out the pixels that are 1 pixel away from one of the above-mentioned determined target candidate points.

[0090] S233. Select the pixel corresponding to the maximum gradient value from the pixels remaining after screening as another target candidate pixel point.

[0091] In these implementation manners, since the above step S232 screens out the pixels whose distance from the determined target candidate pixel points is less than the preset first distance threshold, the pixels in the above-mentioned target curb pixel pair usually lie between the target candidate pixel points determined in the above step S231 and another target candidate pixel point determined in the above step S233.

[0092] Optionally, if the target candidate pixel point determined in the above step S231 and another target candidate pixel point determined in the above step S233 fall on the same side of the pixels in the above-mentioned target curb pixel pair, it is usually prompted that the above-mentioned preset first distance should be set larger.

[0093] Based on the above optional implementation manners, the present solution provides a method for determining the boundaries on both sides of the curb according to the direction indicated by the gradient of the pixels on the density grid map, thereby improving the determination accuracy of the curb edge.

[0094] Continue to refer to Figure 3 , Figure 3 is a schematic diagram of an application scenario of a method for detecting a curb according to an embodiment of the present disclosure. In Figure 3 's application scenario, the point cloud data acquisition vehicle 301 can send the three-dimensional point cloud 302 containing the point cloud data of the curb to the server 303. The server 303 can extract the three-dimensional ground point cloud from the three-dimensional point cloud 302 and rasterize it to generate a target grid map (such as a density grid map) 304. Then, the server 303 slices the target grid map 304 according to the trajectory points of the point cloud data acquisition vehicle 301 corresponding to the three-dimensional ground point cloud to generate a target grid map slice image (such as a density grid map slice image) 305. Next, the server 303 extracts the skeleton information 306 of the curb from the target grid map slice image 305. Finally, the server 303 post-processes the extracted skeleton information 306 of the curb to generate the vector information 307 of the curb.

[0095] Currently, one of the existing technologies usually only detects curb elements based on the curvature of point cloud data or selects the entire curb elements by combining deep learning, resulting in the segmentation effect being severely affected by environmental factors. Therefore, the detection effect of the curb is not stable and not detailed enough. However, the method provided in the above embodiments of the present disclosure generates slice images by rasterizing the ground point cloud, then extracts the skeleton of the curb according to the slice images, and post-processes the skeleton of the curb to finally generate the vector information of the curb. Thus, a curb detection method different from the existing technology is provided. Moreover, through the combination of point cloud and image detection, and the process of generating the skeleton first and then post-processing, a more refined curb edge can be generated, which helps to improve the effectiveness of the curb detection method.

[0096] Further referring to Figure 4 , which shows the flow 400 of another embodiment of the method for detecting curbs. The flow 400 of the method for detecting curbs includes the following steps:

[0097] Step 401, obtaining a three-dimensional point cloud dataset.

[0098] In this embodiment, the execution subject of the method for detecting curbs (such as Figure 1 the server 103 shown) can obtain the three-dimensional point cloud dataset in various ways. Among them, the above three-dimensional point cloud dataset may include point cloud data for characterizing curbs. The specific obtaining method of the above three-dimensional point cloud dataset can refer to the relevant description of step 201 in the foregoing embodiments, which will not be elaborated here.

[0099] Step 402, using morphological filtering to separate the ground from the three-dimensional point cloud to obtain a first quasi-ground three-dimensional point cloud.

[0100] In this embodiment, the above execution subject may first use morphological filtering to separate the ground from the three-dimensional point cloud to obtain a first quasi-ground three-dimensional point cloud. As an example, the above execution subject may first select the data points with height information less than the height threshold from the data points in the three-dimensional point cloud dataset obtained in step 401 to form the first quasi-ground three-dimensional point cloud, so as to roughly distinguish the three-dimensional ground point cloud and the non-ground point cloud.

[0101] Step 403, filtering out the point cloud data with a distance exceeding a preset second distance threshold according to the position indicated by the point cloud data in the first quasi-ground three-dimensional point cloud and the distance from the trajectory points corresponding to the three-dimensional point cloud, to obtain the second quasi-ground three-dimensional point cloud data.

[0102] In this embodiment, according to the positions indicated by the point cloud data in the first quasi-ground three-dimensional point cloud and the distances from the trajectory points corresponding to the three-dimensional point cloud, the execution entity can filter out the point cloud data with distances exceeding a preset second distance threshold in various ways to obtain the second quasi-ground three-dimensional point cloud data.

[0103] As an example, the execution entity can first calculate the distance values in the X-Y plane between each frame of point cloud in the first quasi-ground three-dimensional point cloud and the trajectory point corresponding to that frame. Then, the execution entity can filter out the points with distances exceeding the preset second distance threshold (such as 20 meters) from the corresponding trajectory points. Thus, this solution can remove the laser points that are far from the lidar position at that moment during acquisition, thereby reducing the influence of sparse data.

[0104] Optionally, the preset second distance threshold is related to the effective width of the road, for example, it is the same as the effective width of the road. Thus, this solution can filter out the discrete points with poor reliability due to long distances, providing a basis for further processing of the subsequent point cloud data.

[0105] Step 404: Perform morphological filtering with a reduced window area on the second quasi-ground three-dimensional point cloud data to obtain the third quasi-ground three-dimensional point cloud data.

[0106] In this embodiment, the execution entity can perform morphological filtering with a reduced window area on the second quasi-ground three-dimensional point cloud data obtained in step 403 in various ways to obtain the third quasi-ground three-dimensional point cloud data. Thus, the ground points can be screened out more precisely.

[0107] Step 405: Perform statistical filtering on the third quasi-ground three-dimensional point cloud data to generate the ground point cloud.

[0108] In this embodiment, the execution entity can perform statistical filtering on the third quasi-ground three-dimensional point cloud data obtained in step 404 in various ways to generate the ground point cloud. As an example, the execution entity can calculate the average distance values between the data points in the third quasi-ground three-dimensional point cloud data and the preset number (such as 50) of neighboring point clouds. Then, the execution entity can remove the outlier points according to the calculated average distance values, and thus form the ground point cloud with the remaining data points. As an example, the outlier points can be, for example, the points with average distance values greater than the preset distance threshold. As another example, the outlier points can also be the data points with average distance values outside the standard range. The standard range can be, for example, a Gaussian distribution determined according to the average distance calculated from the third quasi-ground three-dimensional point cloud data, and its shape is determined by the mean and standard deviation.

[0109] Step 406: Perform rasterization on the ground point cloud to generate at least one of the following: a height raster map, a density raster map as the target raster map.

[0110] In this embodiment, the above-mentioned execution entity can rasterize the ground point cloud generated in step 405 in various ways. For example, reference can be made to the relevant description of step 201 in the foregoing embodiment, which will not be elaborated here.

[0111] Step 407: Slice the target raster map according to the trajectory points corresponding to the three-dimensional ground point cloud to generate a target raster map slice image;

[0112] Step 408: Extract the skeleton information of the curb according to the target raster map slice image.

[0113] Step 409: Post-process the skeleton information of the curb to generate the vector information of the curb.

[0114] The above steps 407, 408, and 409 are respectively consistent with steps 202, 203, and 204 in the foregoing embodiment and their optional implementation manners. The descriptions of steps 202, 203, and 204 and their optional implementation manners above also apply to steps 407, 408, and 409, which will not be elaborated here.

[0115] From Figure 4 it can be seen that the process 400 of the method for detecting curbs in this embodiment embodies the organic combination of a series of methods such as morphological filtering, filtering out points far from the trajectory points, morphological filtering with a reduced window area, and statistical filtering, realizing the separation of the ground point cloud data, thereby improving the separation effect of the ground point cloud. Furthermore, it makes the ground raster data cleaner, which has a positive impact on the training and application of deep learning models and data post-processing, etc.

[0116] Further referring to Figure 5 , as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a device for detecting curbs. This device embodiment corresponds to Figure 2 or Figure 4 the method embodiments shown, and this device can be specifically applied to various electronic devices.

[0117] Such as Figure 5As shown in the figure, the device 500 for detecting a curb provided in this embodiment includes a rasterization unit 501, a slicing unit 502, an extraction unit 503, and a post-processing unit 504. Among them, the rasterization unit 501 is configured to rasterize the pre-acquired three-dimensional ground point cloud to generate a target raster map. Among them, the three-dimensional ground point cloud includes point cloud data for characterizing the curb, and the target raster map includes at least one of the following: a height raster map, a density raster map; the slicing unit 502 is configured to slice the target raster map according to the trajectory points corresponding to the three-dimensional ground point cloud to generate a target raster map slice image; the extraction unit 503 is configured to extract the skeleton information of the curb according to the target raster map slice image; the post-processing unit 504 is configured to post-process the skeleton information of the curb to generate vector information of the curb, where the vector information of the curb is used to indicate the point cloud data characterizing the curb.

[0118] In this embodiment, in the device 500 for detecting a curb: the specific processing of the rasterization unit 501, the slicing unit 502, the extraction unit 503, and the post-processing unit 504 and the technical effects brought by them can be respectively referred to Figure 2 the relevant descriptions of steps 201, step 202, step 203, and step 204 in the corresponding embodiments, which will not be elaborated here.

[0119] In some alternative implementation manners of this embodiment, the above target raster map may further include an intensity raster map. The above extraction unit 503 may be further configured to: fuse the slice images corresponding to the target raster map to form a target raster map slice image with a first number of channels; input the target raster map slice image into a pre-trained deep learning model to generate a slice mask map of the curb; splice the generated slice mask map of the curb according to the corresponding coordinates to form a mask map of the curb; find connected regions according to the mask map of the curb and perform erosion operations on the connected regions to extract the skeleton information of the curb.

[0120] In some alternative implementation manners of this embodiment, the above target grid map may include a height grid map and a density grid map. The above post-processing unit 504 may include: a determination module (not shown in the figure), configured to determine a pair of target curb pixel points from the skeleton information of the curb according to the curb end indicated by the skeleton information of the curb and the edge detected based on the Canny operator; an execution module (not shown in the figure), configured to execute the following curb determination steps: determine the gradient direction of the curb according to the position of the determined pair of target curb pixel points on the density grid map; determine the gradient values corresponding to the second number of candidate pixel points selected along the gradient direction of the curb from the target pixel point according to the pixel values around the pixel point, where the target pixel point may be one of the pixel points in the pair of target curb pixel points; select pixel points from both sides of the target pixel point as target candidate pixel points according to the determined gradient values; select the target candidate pixel point corresponding to the target pixel point as the inner edge pixel point of the curb according to the position of the target candidate pixel point on the height grid map; in response to determining that there is a next pixel point of the target curb pixel point selected along the skeleton information of the curb, form a new pair of target curb pixel points with the target curb pixel point and the next pixel point, and continue to execute the above curb determination steps; a generation module (not shown in the figure), configured to generate vector information of the curb according to the generated inner edge pixel points of the curb in response to determining that there is no next pixel point of the target curb pixel point selected along the skeleton information of the curb.

[0121] In some alternative implementation manners of this embodiment, the above execution module may be further configured to: in response to determining that the maximum value among the determined gradient values is greater than a preset gradient threshold, determine the pixel point corresponding to the maximum value among the determined gradient values as one of the target candidate pixel points; filter out the pixel points whose distance from the determined target candidate pixel point is less than a preset first distance threshold; select the pixel point corresponding to the maximum value of the gradient value from the remaining pixel points after filtering as another target candidate pixel point.

[0122] In some alternative implementation manners of this embodiment, the above rasterization unit 501 may be further configured to: obtain a three-dimensional point cloud dataset, where the three-dimensional point cloud dataset may include point cloud data for characterizing the curb; use morphological filtering to separate the ground from the three-dimensional point cloud to obtain a first quasi-ground three-dimensional point cloud; filter out the point cloud data whose distance exceeds a preset second distance threshold according to the position indicated by the point cloud data in the first quasi-ground three-dimensional point cloud and the distance from the trajectory point corresponding to the three-dimensional point cloud, to obtain second quasi-ground three-dimensional point cloud data; perform morphological filtering with a reduced window area on the second quasi-ground three-dimensional point cloud data to obtain third quasi-ground three-dimensional point cloud data; perform statistical filtering on the third quasi-ground three-dimensional point cloud data to generate ground point cloud; rasterize the ground point cloud to generate at least one of the following as the target grid map: a height grid map, a density grid map.

[0123] The device provided by the above embodiments of the present disclosure rasterizes the ground point cloud through the rasterization unit 501 and generates a sliced image through the slicing unit 502. Then, the extraction unit 503 extracts the skeleton of the curb according to the sliced image, and the post-processing unit 504 performs post-processing on the skeleton of the curb, and finally generates the vector information of the curb. Thereby, a curb detection method different from the prior art is provided. Moreover, through the combination of point cloud and image detection, and the process of generating the skeleton first and then performing post-processing, a finer curb edge can be generated, which helps to improve the effectiveness of the curb detection method.

[0124] Next, refer to Figure 6 , which shows a schematic structural diagram of an electronic device (such as Figure 1 the server in Figure 6 The terminal device / server shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0125] As Figure 6 shown, the electronic device 600 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which may perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage device 608 into the random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. The input / output (I / O) interface 605 is also connected to the bus 604.

[0126] Generally, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 6 the electronic device 600 with various devices is shown, it should be understood that it is not required to implement or include all the shown devices. Instead, more or fewer devices may be implemented or included. Figure 6 Each block shown in

[0127] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above functions defined in the method of the embodiment of the present application are executed.

[0128] It should be noted that the computer-readable medium described in the embodiments of the present disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the embodiments of the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device. In the embodiments of the present disclosure, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable signal medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0129] The above computer-readable medium may be included in the above electronic device; or it may exist separately without being assembled into the server. The above computer-readable medium carries one or more programs, and when the one or more programs are executed by the server, the server is caused to: rasterize a pre-acquired three-dimensional ground point cloud to generate a target raster map, where the three-dimensional ground point cloud includes point cloud data for characterizing a curb, and the target raster map includes at least one of the following: a height raster map, a density raster map; slice the target raster map according to trajectory points corresponding to the three-dimensional ground point cloud to generate a target raster map slice image; extract the skeleton information of the curb from the target raster map slice image; post-process the skeleton information of the curb to generate vector information of the curb, where the vector information of the curb is used to indicate the point cloud data characterizing the curb.

[0130] Computer program code for performing the operations of the embodiments of the present disclosure may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as "C", the Python language, or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it may be connected to an external computer (for example, by connecting through an Internet service provider using the Internet).

[0131] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0132] The units involved in the embodiments described in this disclosure can be implemented in software or in hardware. The described units can also be provided in a processor. For example, it can be described as: a processor including a rasterization unit, a slicing unit, an extraction unit, and a post-processing unit. Among them, the names of these units do not constitute a limitation to the unit itself in some cases. For example, the rasterization unit can also be described as "a unit that rasterizes a pre-acquired three-dimensional ground point cloud to generate a target raster map, where the three-dimensional ground point cloud includes point cloud data for characterizing a curb, and the target raster map includes at least one of the following: a height raster map, a density raster map".

[0133] The above description is only for the preferred embodiments of this disclosure and the explanation of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solutions formed by mutually replacing the above features with the (but not limited to) technical features having similar functions disclosed in the embodiments of this disclosure.

Claims

1. A method for detecting a curb, comprising: Rasterize the pre-acquired three-dimensional ground point cloud to generate a target raster map, where the three-dimensional ground point cloud includes point cloud data for characterizing a curb, and the target raster map includes a height raster map and a density raster map; Slice the target raster map according to the trajectory points corresponding to the three-dimensional ground point cloud to generate a target raster map slice image; Extract the skeleton information of the curb according to the target raster map slice image; Determine a pair of target curb pixel points from the skeleton information of the curb according to the curb end indicated by the skeleton information of the curb and the edge detected based on the Canny operator; and, perform the following curb determination steps: Determine the gradient direction of the curb according to the position of the determined pair of target curb pixel points on the density raster map; Determine the gradient values corresponding to the second number of candidate pixel points selected along the gradient direction of the curb according to the pixel values around the pixel points, where the target pixel point is one of the pixel points in the pair of target curb pixel points; Select pixel points as target candidate pixel points from both sides of the target pixel point according to the determined gradient values; Select the target candidate pixel point corresponding to the target pixel point as the inner edge pixel point of the curb according to the position of the target candidate pixel point on the height raster map; In response to determining that the next pixel point of the target curb pixel point selected along the skeleton information of the curb does not exist, generate the vector information of the curb according to the generated inner edge pixel points of the curb; where the vector information of the curb is used to indicate the point cloud data characterizing the curb.

2. The method according to claim 1, wherein, The target raster map further includes an intensity raster map; And The extracting the skeleton information of the curb according to the target raster map slice image includes: Fuse the slice images corresponding to the target raster map to form a target raster map slice image with the first number of channels; Input the target raster map slice image into a pre-trained deep learning model to generate a slice mask map of the curb; Stitch the generated slice mask maps of the curb according to the corresponding coordinates to form a mask map of the curb; Find connected regions according to the mask map of the curb and perform erosion operations on the connected regions to extract the skeleton information of the curb.

3. The method according to claim 1 or 2, the method further comprising: In response to determining that the next pixel point of the target curb pixel point selected along the skeleton information of the curb exists, form a new pair of target curb pixel points with the target curb pixel point and the next pixel point, and continue to perform the curb determination steps.

4. The method according to claim 3, wherein, The selecting pixel points as target candidate pixel points from both sides of the target pixel point according to the determined gradient values includes: In response to determining that the maximum value among the determined gradient values is greater than a preset gradient threshold, determine the pixel point corresponding to the maximum value among the determined gradient values as one of the target candidate pixel points; Filter out pixel points whose distance from the determined target candidate pixel point is less than a preset first distance threshold; Select the pixel point corresponding to the maximum value of the gradient value from the remaining pixel points after filtering as the other target candidate pixel point.

5. The method according to claim 1, wherein, The rasterizing the pre-acquired three-dimensional ground point cloud to generate a target raster map includes: Obtain a three-dimensional point cloud dataset, where the three-dimensional point cloud dataset includes point cloud data for characterizing a curbstone; Use morphological filtering to separate the ground from the three-dimensional point cloud to obtain a first quasi-ground three-dimensional point cloud; Filter out point cloud data with a distance exceeding a preset second distance threshold according to the positions indicated by the point cloud data in the first quasi-ground three-dimensional point cloud and the distance from the trajectory points corresponding to the three-dimensional point cloud, to obtain second quasi-ground three-dimensional point cloud data; Perform morphological filtering with a reduced window area on the second quasi-ground three-dimensional point cloud data to obtain third quasi-ground three-dimensional point cloud data; Perform statistical filtering on the third quasi-ground three-dimensional point cloud data to generate a ground point cloud; Perform rasterization on the ground point cloud to generate at least one of the following: a height raster map, a density raster map as a target raster map.

6. A device for detecting a curb, comprising: A rasterization unit, configured to perform rasterization on a pre-acquired three-dimensional ground point cloud to generate a target raster map, where the three-dimensional ground point cloud includes point cloud data for characterizing a curbstone, and the target raster map includes a height raster map and a density raster map; A slicing unit, configured to slice the target raster map according to the trajectory points corresponding to the three-dimensional ground point cloud to generate a target raster map slice image; An extraction unit, configured to extract the skeleton information of the curbstone according to the target raster map slice image; A post-processing unit, including: A determination module, configured to determine a pair of target curbstone pixel points from the skeleton information of the curbstone according to the curbstone end indicated by the skeleton information of the curbstone and the edge detected based on the Canny operator; and An execution module, configured to execute the following curbstone determination steps: determine the gradient direction of the curbstone according to the positions of the determined pair of target curbstone pixel points on the density raster map; determine the gradient values corresponding to the second number of candidate pixel points selected along the gradient direction of the curbstone from the target pixel point according to the pixel values around the pixel point, where the target pixel point is one of the pixel points in the pair of target curbstone pixel points; select pixel points from both sides of the target pixel point as target candidate pixel points according to the determined gradient values; select the target candidate pixel points corresponding to the target pixel point as the inner edge pixel points of the curbstone according to the positions of the target candidate pixel points on the height raster map; A generation module, configured to generate the vector information of the curbstone according to the generated inner edge pixel points of the curbstone in response to determining that there is no next pixel point of the target curbstone pixel points selected along the skeleton information of the curbstone; where the vector information of the curbstone is used to indicate the point cloud data characterizing the curbstone.

7. The device according to claim 6, wherein, The target raster map further includes an intensity raster map; and The extraction unit is further configured to: Fuse the slice images corresponding to the target raster map to form a target raster map slice image with a first number of channels; Input the target raster map slice image into a pre-trained deep learning model to generate a slice mask map of the curbstone; Stitch the generated slice mask maps of the curbstone according to the corresponding coordinates to form a mask map of the curbstone; Find the connected regions according to the mask image of the curb and perform erosion operations on the connected regions to extract the skeleton information of the curb.

8. The device according to claim 6 or 7, wherein, The execution module is further configured to, in response to determining the existence of the next pixel point of the target curb pixel point selected along the skeleton information of the curb, form a new pair of target curb pixel points by the target curb pixel point and the next pixel point, and continue to execute the curb determination step.

9. An electronic device, comprising: One or more processors; A storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-5.

10. A computer-readable medium having a computer program stored thereon, wherein, When the program is executed by the processor, the method according to any one of claims 1-5 is implemented.

Images