Road edge detection method and device, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention provides a road edge detection method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: segmenting original point cloud data collected by a laser radar according to a preset segmentation threshold value, and obtaining local point cloud data; performing sampling processing on the local point cloud data according to a preset three-dimensional voxel grid size to obtain sampling point cloud data; classifying three-dimensional points in the sampling point cloud data through a point cloud convolutional neural network model, and determining to obtain curb three-dimensional points; according to the curb three-dimensional points, curb marker facades are obtained; and determining curb position information according to the facade of the curb marker. In this way, the original point cloud data is segmented and sampled to obtain the sampling point cloud data, the point cloud convolutional neural network model is used to determine the facade of the road edge marker for the sampling point cloud data, and the position information of the road edge is determined according to the position information of the road edge marker, so that the stability and precision of the road edge detection result can be improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a road edge detection method, device, electronic device, and computer-readable storage medium. Background Art

[0002] With the continuous development and advancement of artificial intelligence and modern sensor technologies, autonomous driving technology based on environmental perception has emerged. Accurate road edge detection is a crucial technology for achieving safe and autonomous driving. When a vehicle deviates from its lane for reasons such as parking or obstacle avoidance, the detected road edge position information is a crucial basis for the autonomous driving system to plan its path.

[0003] The main existing technical approach to detecting road edges is as follows: first, the coordinates of the road surface area are detected from a 3D point cloud or RGB image, and then the road edge is located by extracting the boundary coordinates of the road surface area. This approach works well in open areas with minimal interference, but in complex road scenes, it is susceptible to interference from other objects such as vehicles, resulting in poor road edge detection. Summary of the Invention

[0004] In order to solve the above technical problems, embodiments of the present application provide a road edge detection method, device, electronic device, and computer-readable storage medium.

[0005] In a first aspect, an embodiment of the present application provides a road edge detection method, the method comprising:

[0006] Segment the original point cloud data collected by the lidar according to the preset segmentation threshold to obtain local point cloud data;

[0007] Sampling the local point cloud data according to a preset voxel grid size to obtain sampled point cloud data;

[0008] Classifying each three-dimensional point in the sampled point cloud data using a point cloud convolutional neural network model to determine a three-dimensional curb point;

[0009] Acquire the facade of the curb marker according to the curb three-dimensional point;

[0010] The curb position information is determined according to the curb marker elevation.

[0011] In a second aspect, an embodiment of the present application provides a road edge detection device, the device comprising:

[0012] The segmentation module is used to segment the original point cloud data collected by the lidar according to a preset segmentation threshold to obtain local point cloud data;

[0013] a sampling module, configured to perform sampling processing on the local point cloud data according to a preset three-dimensional voxel grid size to obtain sampled point cloud data;

[0014] A classification module is used to classify each 3D point in the sampled point cloud data through a point cloud convolutional neural network model to determine the 3D curb point;

[0015] An acquisition module, configured to acquire a curb marker elevation based on the curb three-dimensional points;

[0016] A determination module is used to determine the curb position information according to the facade of the curb marker.

[0017] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is run by the processor, the road edge detection method provided in the first aspect is executed.

[0018] In a fourth aspect, an embodiment of the present application provides a vehicle comprising a vehicle body, a laser radar mounted on the vehicle body, and the electronic device provided in the third aspect; the laser radar is used to obtain original point cloud data.

[0019] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which executes the road edge detection method provided in the first aspect when running on a processor.

[0020] The road edge detection method, device, electronic device, and computer-readable storage medium provided by the present application segment the raw point cloud data collected by the laser radar according to a preset segmentation threshold to obtain local point cloud data; sample and process the local point cloud data according to a preset voxel grid size to obtain sampled point cloud data; classify each three-dimensional point in the sampled point cloud data using a point cloud convolutional neural network model to determine a three-dimensional curb point; obtain the elevation of the curb marker based on the three-dimensional curb point; and determine the curb position information based on the elevation of the curb marker. In this way, by segmenting and sampling the raw point cloud data to obtain sampled point cloud data, determining the elevation of the curb marker from the sampled point cloud data using a point cloud convolutional neural network model, and determining the position information of the road edge based on the position information of the road edge marker, the situation where the intersection line between the road surface and the moving vehicle is regarded as the road edge can be significantly avoided, thereby improving the stability and accuracy of the curb detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solution of this application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of this application and should not be regarded as limiting the scope of protection of this application. In each of the drawings, similar components are numbered similarly.

[0022] Figure 1 A schematic diagram of a process flow of a road edge detection method provided by an embodiment of the present application is shown;

[0023] Figure 2 A schematic diagram of original three-dimensional point cloud data provided by an embodiment of the present application is shown;

[0024] Figure 3 A schematic diagram of local three-dimensional point cloud data provided by an embodiment of the present application is shown;

[0025] Figure 4 A schematic diagram of sampling point cloud data provided by an embodiment of the present application is shown;

[0026] Figure 5 A schematic diagram of a road edge detection effect provided by an embodiment of the present application is shown;

[0027] Figure 6 A schematic structural diagram of a road edge detection method and device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0029] The components of the embodiments of the present application generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but rather merely represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0030] Hereinafter, the terms "including", "having" and their cognates, which may be used in various embodiments of the present application, are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or the possibility of adding one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items.

[0031] Furthermore, the terms “first,” “second,” “third,” etc., are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.

[0032] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art to which the various embodiments of the present application belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meaning as in the context of the relevant technical field and will not be interpreted as having an idealized meaning or an overly formal meaning unless clearly defined in the various embodiments of the present application.

[0033] Example 1

[0034] An embodiment of the present disclosure provides a road edge detection method.

[0035] For details, see Figure 1 ,Road edge detection methods include:

[0036] Step S101 : segmenting the original point cloud data collected by the lidar according to a preset segmentation threshold to obtain local point cloud data.

[0037] In this embodiment, the road edge detection method can be applied to an electronic device installed on a vehicle. The vehicle can also be equipped with a laser radar. The laser radar can obtain raw point cloud data. The raw point cloud data includes multiple three-dimensional points, each of which is used to represent position information in the three-dimensional coordinate system of the laser radar. The three-dimensional points of the raw point cloud data are represented by coordinate positions in a first direction, a second direction, and a third direction. For example, a three-dimensional coordinate system is established with the laser radar as the coordinate origin. In this coordinate system, for example, the axial directions of the three-dimensional coordinate system can be as follows: the X axis is parallel to the ground and points to the right of the radar, the Y axis is parallel to the ground and points to the front of the radar, and the Z axis passes through the radar center of mass and points upward. The axial directions of the three-dimensional coordinate system can also be other ways, which are not limited here. The three-dimensional points of the raw point cloud data can be represented by three-dimensional points (x, y, z), and the three-dimensional points (x, y, z) represent positions in the three-dimensional coordinate system of the laser radar.

[0038] See also Figure 2 , Figure 2 The figure shows a schematic diagram of the original point cloud data. The original point cloud data includes a large number of 3D points, which contain relatively rich ground information and road edge information. The following interference points may exist in the 3D points, and the interference points can be denoised.

[0039] It should be noted that the laser radar scans the surrounding environment to obtain raw three-dimensional point cloud data of the surrounding environment. The number of three-dimensional points in the raw point cloud data is relatively large and contains relatively rich environmental information. However, due to the large number of points, if road edge detection is performed directly based on the raw point cloud data, the raw point cloud data may contain a large number of interfering three-dimensional points, which will affect the accuracy of the calculation. In addition, the large number of three-dimensional points in the raw point cloud data will lead to a large amount of subsequent calculations and consume a lot of computing resources. In this embodiment, the raw point cloud data collected by the laser radar is segmented according to a preset segmentation threshold to obtain local point cloud data. The raw point cloud data can be pre-processed for denoising to reduce interference and the number of three-dimensional points, thereby saving computing resources and improving calculation accuracy.

[0040] It should be noted that, according to actual conditions, the effective scanning distance of the laser radar is limited due to the limitations of the configuration parameters. The distance that the laser radar can effectively scan can be called the maximum scanning distance. The distribution range of valid three-dimensional points in the original point cloud data in each coordinate axis direction can be estimated based on the maximum scanning distance of the laser radar and the width of the road surface. For example, the first direction preset threshold and the third direction preset threshold can be set to filter the three-dimensional points of the original three-dimensional points, remove the interfering three-dimensional points, and form the remaining valid three-dimensional points into local point cloud data. If the axes of the three-dimensional coordinate system of the laser radar are: the X-axis is parallel to the ground and points to the right side of the radar, the Y-axis is parallel to the ground and points to the front of the radar, and the Z-axis passes through the center of mass of the radar and points upward, the first direction preset threshold can be the Z-axis direction preset threshold, and the third direction preset threshold can be the X-axis direction preset threshold.

[0041] In this example, a 3D point set containing both road surface and curb information is extracted from the original point cloud and used as input data for subsequent processing. By filtering the original point cloud data, the amount of data required by the subsequent point cloud convolutional neural network model is reduced, improving overall detection speed.

[0042] In one embodiment, step S101 includes the following steps:

[0043] Determine a first target three-dimensional point in the original point cloud data, the absolute value of which in the first direction is less than or equal to a preset threshold in the first direction;

[0044] Determine a second target three-dimensional point in the original point cloud data, the value of the third direction being greater than or equal to a preset threshold value of the third direction;

[0045] The first target three-dimensional point and the second target three-dimensional point are deleted from the original point cloud data to obtain the local point cloud data.

[0046] See also Figure 3 ,Figure 3 The figure shows a schematic diagram of local point cloud data, which is the remaining point cloud data after removing ground point cloud data and invalid interference points from the original point cloud data.

[0047] In one embodiment, the first direction preset threshold is determined based on the maximum number of lanes on the road, the maximum width of the road, and a preset detection range safety factor;

[0048] The preset threshold value of the third direction is determined based on the distance between the laser radar and the ground, the farthest scanning distance of the laser radar, and the maximum longitudinal slope of the road.

[0049] In one embodiment, if the first direction preset threshold is the X-axis direction preset threshold, and the third direction preset threshold is the Z-axis direction preset threshold, the X-axis direction preset threshold can be calculated according to the following formula 1:

[0050] Formula 1: WX = (NW × WL × A) / 2;

[0051] Where WX represents the preset threshold in the X-axis direction, NW represents the maximum number of lanes on the road, WL represents the maximum lane width, and A represents the detection range safety factor, which ranges from [1.5, 4].

[0052] In addition, the preset threshold in the Z-axis direction can be calculated according to the following formula 2:

[0053] Formula 2: ZH=ZL+YL×C;

[0054] Among them, ZH represents the preset threshold in the Z-axis direction, ZL represents the distance between the center of mass of the lidar and the ground, YL represents the farthest scanning distance of the lidar in the Y-axis direction, and C represents the maximum longitudinal slope of the road.

[0055] For example, if the preset threshold in the X-axis direction is WX, then the step of determining the second target three-dimensional point whose absolute value of the second direction value in the original point cloud data is less than or equal to the preset threshold in the third direction can be understood as: determining the second target three-dimensional point whose second direction value in the original point cloud data is greater than or equal to -WX and less than or equal to WX, that is, the X-axis element value of the second target three-dimensional point is in the interval [-WX, WX].

[0056] Step S102 : Sampling the local point cloud data according to a preset voxel grid size to obtain sampled point cloud data.

[0057] In one embodiment, step S102 may include the following steps:

[0058] Segmenting the local point cloud data according to a preset three-dimensional voxel grid size to obtain a plurality of three-dimensional voxel grids;

[0059] A preset number of sampling three-dimensional points are randomly sampled from all three-dimensional points of each three-dimensional voxel grid, and the sampling three-dimensional points of the plurality of the three-dimensional voxel grids are used as the sampling point cloud data.

[0060] See also Figure 4 , Figure 4 The figure shows a schematic diagram of sampling point cloud data. The acquisition process of the sampling point cloud data is as follows: Figure 4 The local point cloud data shown is divided into a plurality of three-dimensional voxel grids; N sampling three-dimensional points are randomly sampled from all three-dimensional points of each three-dimensional voxel grid, and all the sampled three-dimensional points are used to form sampling point cloud data. Figure 5 The number of 3D points of the sampling point cloud data is less than Figure 4 In this way, the amount of calculation can be further reduced and computing resources can be saved.

[0061] In one embodiment, the preset voxel grid size may be determined based on a maximum edge detection error in the first direction, a maximum edge detection error in the second direction, and a minimum curb height in the third direction.

[0062] For example, a preset voxel grid size is [dx / 2, dy / 2, dz / 4], where dx is the maximum allowable edge detection error of the road edge in the use scenario along the X-axis, dy is the maximum allowable edge detection error of the road edge in the use scenario along the Y-axis, and dz is the lowest curb height recognizable by the lidar in the use scenario, which must be no less than twice the distance accuracy of the lidar. In one embodiment, the local point cloud data is divided into multiple voxel grids according to the [dx / 2, dy / 2, dz / 4] voxel grid size, and a corresponding 3D point is determined for each voxel grid.

[0063] In one embodiment, the position information of the center of gravity of each stereo voxel grid can be calculated based on the multiple three-dimensional points of each stereo voxel grid, the center of gravity point can be used as an adjusted three-dimensional point of each stereo voxel grid, and the multiple adjusted three-dimensional points can be used to form sampling point cloud data.

[0064] In the embodiment, the point cloud convolutional neural network model can automatically generate a preset three-dimensional voxel grid size according to a self-defined edge detection accuracy, and perform down-sampling on the input sampling point cloud data based on the preset three-dimensional voxel grid size to reduce the density of the point cloud data. The higher the self-defined edge detection accuracy, the smaller the preset three-dimensional voxel grid size automatically generated by the point cloud convolutional neural network model, the greater the point cloud density, and the more accurate the position information of the detected road edge, but the greater the required storage space and the greater the subsequent calculation amount of the point cloud convolutional neural network model. When multiple frames of sampling point cloud data are detected simultaneously, after the multiple frames of sampling point cloud data are compressed according to the preset three-dimensional voxel grid size, the compressed point cloud data is also randomly sampled at a fixed number to ensure that the number of points contained in each frame of point cloud is the same. For example, the compressed point cloud data is randomly sampled according to a sampling point N, where N is an integer multiple of 1024.

[0065] In this way, the detection accuracy and detection speed of the road edge detection scheme can be effectively balanced, the point cloud density can be reduced as much as possible while ensuring the required detection accuracy, and the detection speed can be improved.

[0066] Step S103, classifying each three-dimensional point in the sampling point cloud data by the point cloud convolutional neural network model to determine the curb three-dimensional point.

[0067] It should be noted that the point cloud convolutional neural network model is trained according to the classes of points in the historical collected point cloud and the manually labeled point cloud, and can improve the accuracy of classification. The point cloud convolutional neural network model includes a single-frame detection mode and a multi-frame detection mode.

[0068] In an embodiment, step S103 includes the following steps:

[0069] The point cloud convolutional neural network model determines the curb point probability, the road surface point probability, and the other point probability of each three-dimensional point in the sampling point cloud data.

[0070] The maximum probability value of the road surface point probability, the curb point probability, and the other point probability of each three-dimensional point is determined, and the three-dimensional point with the maximum curb point probability is determined as the curb three-dimensional point.

[0071] For example, the detection result of three-dimensional point A is [70%, 20%, 10%], the probability of three-dimensional point A being a curb point is 70%, the probability of three-dimensional point A being a road surface is 20%, and the probability of three-dimensional point A being other areas is 10%. Therefore, three-dimensional point A is classified as a curb point category.

[0072] In one embodiment, determining the curb point probability, road surface point probability, and other point probabilities of each three-dimensional point of the sampled point cloud data using the point cloud convolutional neural network model may include the following steps:

[0073] When the point cloud convolutional neural network model is in a single-frame detection mode, converting the single-frame sampled point cloud data into a first matrix, the first matrix including N rows, each row of the first matrix representing coordinate information of each three-dimensional point of the sampled point cloud data;

[0074] The first matrix is ​​input into the point cloud convolutional neural network model, and the first matrix is ​​calculated by the point cloud convolutional neural network model to obtain a second matrix, where the second matrix includes N rows, and each row of the second matrix represents the category probability of each three-dimensional point of the sampled point cloud data, where the category probability includes road surface point probability, curb point probability, and other point probabilities.

[0075] Specifically, the point cloud data processing process of the single-frame detection mode is as follows: the sampled point cloud data is formatted into a first matrix with N rows and 3 columns, where N is the number of 3D points contained in the sampled point cloud data. The i-th row of the first matrix represents the coordinates of the i-th input 3D point [xi, yi, zi]. This first matrix is ​​input into the point cloud convolutional neural network model. After the point cloud convolutional neural network model calculates the input sampled point cloud data, a second matrix with N rows and 3 columns is obtained. The i-th row of the second matrix represents the detection result of the i-th input 3D point [Pi0, Pi1, Pi2], where Pi0 is the probability that the i-th input 3D point belongs to the road surface point, Pi1 is the probability that the i-th input 3D point belongs to the curb point, and Pi2 is the probability that the i-th input 3D point belongs to other points in other areas. The category with the highest probability in the detection result corresponding to each point is the category to which the point belongs.

[0076] In another embodiment, determining the curb point probability, road surface point probability, and other point probabilities of each three-dimensional point of the sampled point cloud data using the point cloud convolutional neural network model may include the following steps:

[0077] When the point cloud convolutional neural network model is in a multi-frame detection mode, converting the multi-frame sampling point cloud data into a third matrix, the third matrix including a plurality of sub-matrices, each row of each sub-matrix of the third matrix representing the coordinate information of each three-dimensional point of each frame of sampling point cloud data;

[0078] input the third matrix into the point cloud convolutional neural network model, and calculate the third matrix through the point cloud convolutional neural network model to obtain a fourth matrix, the fourth matrix comprising a plurality of sub-matrices, each row of each sub-matrix of the fourth matrix representing a class probability of each three-dimensional point of each frame of sampled point cloud data, the class probability comprising a road surface point probability, a curb point probability and an other point probability.

[0079] Specifically, the processing procedure of the multi-frame detection sampled point cloud data is as follows: the multi-frame sampled point cloud data to be identified is formatted into a third matrix of B*N*3, the third matrix of B*N*3 is equivalent to B sub-matrices of N*3, the formatted third matrix of B*N*3 is input into the point cloud convolutional neural network model, and the point cloud convolutional neural network model calculates the input third matrix of B*N*3 to output a fourth matrix of B*N*3, the fourth matrix of B*N*3 is equivalent to B sub-matrices of N*3, wherein B is the number of frames of standard power data input into the point cloud convolutional neural network model, and N is the number of three-dimensional points contained in one frame of sampled point cloud data. When B=j, j=1, 2, 3,..., the formatted j*N*3 sub-matrix is the sampled point cloud data corresponding to the jth frame of sampled point cloud data, and the output j*N*3 matrix is the classification result corresponding to the jth frame of sampled point cloud data.

[0080] In step S104, a curb marker facade is obtained according to the curb three-dimensional point.

[0081] In the embodiment, step S104 can include the following steps:

[0082] generating at least one geometric plane according to the curb three-dimensional point;

[0083] when there is a target geometric plane in at least one of the geometric planes, the target geometric plane is determined as the curb marker facade.

[0084] In an embodiment, there are road separation guardrails, road stakes, road curbs and other protrusions in reference curbs, these protrusions can be used as markers, the facade form of the curb marker is obtained in advance, and after classification of each three-dimensional point, it can be determined that each three-dimensional point belongs to a curb point, a road surface point or an other point. Therefore, a plurality of geometric planes are generated according to all curb points, a target geometric plane matching a preset marker facade is determined as a curb marker facade, and the curb marker facade can be the facade of the curb protrusion. Therefore, the detection of the curb marker facade can determine that the curb is detected.

[0085] Please refer to Figure 5 , Figure 5 Fig. 2 shows a curb detection effect schematic diagram, and Figure 5The roadside marker facade 501 is included, and the roadside marker facade 501 can be a road dividing guardrail, a road bollard, a curb or other protruding object facade.

[0086] Step S105: determining the curb position information according to the curb marker elevation.

[0087] In this embodiment, step S105 may include the following steps:

[0088] Determine the real position information of the curb marker facade according to the three-dimensional points corresponding to the curb marker facade;

[0089] The actual position information of the facade of the curb marker is used as the curb position information.

[0090] In this embodiment, since the curb marker's elevation is the elevation of a raised feature on the curb, it can be determined that the curb has been detected. Based on the three-dimensional point corresponding to the curb marker's elevation in the LiDAR coordinate system, the real-world location of the curb marker's elevation can be converted and used to determine the curb's position.

[0091] The road edge detection method provided in this embodiment segments the raw point cloud data collected by the laser radar according to a preset segmentation threshold to obtain local point cloud data; samples and processes the local point cloud data according to a preset voxel grid size to obtain sampled point cloud data; classifies each three-dimensional point in the sampled point cloud data using a point cloud convolutional neural network model to determine the three-dimensional road edge points; obtains the elevation of the road edge marker based on the three-dimensional road edge points; and determines the road edge position information based on the elevation of the road edge marker. In this way, by segmenting and sampling the raw point cloud data to obtain sampled point cloud data, determining the elevation of the road edge marker from the sampled point cloud data using a point cloud convolutional neural network model, and determining the position information of the road edge based on the position information of the road edge marker, the method can significantly avoid mistaking the intersection of the road surface and the moving vehicle as the road edge, thereby improving the stability and accuracy of the road edge detection results.

[0092] Example 2

[0093] In addition, an embodiment of the present disclosure provides a road edge detection device.

[0094] Specifically, such as Figure 6 As shown, the road edge detection device 600 includes:

[0095] The segmentation module 601 is used to segment the original point cloud data collected by the laser radar according to a preset segmentation threshold to obtain local point cloud data;

[0096] The sampling module 602 is configured to sample the local point cloud data according to a preset three-dimensional voxel grid size, to obtain sampled point cloud data.

[0097] The classification module 603 is configured to classify each three-dimensional point in the sampled point cloud data by using a point cloud convolutional neural network model, to determine curb three-dimensional points.

[0098] The acquisition module 604 is configured to acquire a curb marker facade according to the curb three-dimensional points.

[0099] The determination module 605 is configured to determine curb position information according to the curb marker facade.

[0100] In this embodiment, the segmentation module 601 is further configured to determine first target three-dimensional points in the original point cloud data, whose absolute values of first direction values are less than or equal to a first direction preset threshold value.

[0101] The determination module 605 is configured to determine second target three-dimensional points in the original point cloud data, whose third direction values are greater than or equal to a third direction preset threshold value.

[0102] In the original point cloud data, the first target three-dimensional points and the second target three-dimensional points are deleted, to obtain the local point cloud data.

[0103] In this embodiment, the first direction preset threshold value is determined according to a maximum lane number of a road, a maximum width of the road, and a preset detection range safety coefficient; and the third direction preset threshold value is determined according to a distance between the laser radar and the ground, a farthest scanning distance of the laser radar, and a maximum longitudinal slope of the road.

[0104] In this embodiment, the sampling module 602 is further configured to segment the local point cloud data according to a preset three-dimensional voxel grid size, to obtain a plurality of three-dimensional voxel grids.

[0105] A preset number of sampling three-dimensional points are randomly sampled from all three-dimensional points in each three-dimensional voxel grid, and the sampling three-dimensional points of the plurality of three-dimensional voxel grids are taken as the sampled point cloud data.

[0106] In this embodiment, the classification module 603 is further configured to determine, by using the point cloud convolutional neural network model, a curb point probability, a road surface point probability, and other point probabilities of each three-dimensional point in the sampled point cloud data.

[0107] The maximum probability value of the road surface point probability, the curb point probability, and the other point probabilities of each three-dimensional point is determined, and a three-dimensional point with the maximum probability value as the curb point probability is determined as a curb three-dimensional point.

[0108] In this embodiment, the classification module 603 is further configured to, when the point cloud convolutional neural network model is in a single-frame detection mode, convert the single-frame sampled point cloud data into a first matrix, where the first matrix includes N rows, and each row of the first matrix represents coordinate information of each three-dimensional point of the sampled point cloud data;

[0109] The first matrix is ​​input into the point cloud convolutional neural network model, and the first matrix is ​​calculated by the point cloud convolutional neural network model to obtain a second matrix, where the second matrix includes N rows, and each row of the second matrix represents the category probability of each three-dimensional point of the sampled point cloud data, where the category probability includes road surface point probability, curb point probability, and other point probabilities.

[0110] In this embodiment, the classification module 603 is further configured to, when the point cloud convolutional neural network model is in a multi-frame detection mode, convert the multi-frame sampled point cloud data into a third matrix, wherein the third matrix includes a plurality of sub-matrices, and each row of each sub-matrix of the third matrix represents the coordinate information of each three-dimensional point of each frame of the sampled point cloud data;

[0111] The third matrix is ​​input into the point cloud convolutional neural network model, and the third matrix is ​​calculated by the point cloud convolutional neural network model to obtain a fourth matrix, wherein the fourth matrix includes multiple sub-matrices, and each row of each sub-matrix of the fourth matrix represents the category probability of each three-dimensional point of each frame of sampled point cloud data, and the category probability includes road surface point probability, curb point probability and other point probabilities.

[0112] In this embodiment, the acquisition module 604 is further configured to generate at least one geometric plane according to the three-dimensional points of the curb;

[0113] When there is a target geometric plane whose shape matches the preset marker facade in at least one of the geometric planes, the target geometric plane is determined to be the curb marker facade.

[0114] In this embodiment, the determination module 605 is further configured to determine the real position information of the curb marker facade according to the three-dimensional points corresponding to the curb marker facade;

[0115] The actual position information of the facade of the curb marker is used as the curb position information.

[0116] The road edge detection device provided in this embodiment segments the raw point cloud data collected by the laser radar according to a preset segmentation threshold to obtain local point cloud data; samples and processes the local point cloud data according to a preset voxel grid size to obtain sampled point cloud data; classifies each three-dimensional point in the sampled point cloud data using a point cloud convolutional neural network model to determine a three-dimensional curb point; obtains the elevation of a curb marker based on the three-dimensional curb point; and determines curb position information based on the elevation of the curb marker. In this way, by segmenting and sampling the raw point cloud data to obtain sampled point cloud data, determining the elevation of a curb marker based on the sampled point cloud data using a point cloud convolutional neural network model, and determining the position information of the road edge based on the position information of the road edge marker, the device can significantly avoid mistaking the intersection of the road surface and the moving vehicle as the road edge, thereby improving the stability and accuracy of curb detection results.

[0117] Example 3

[0118] In addition, an embodiment of the present disclosure provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the road edge detection method provided in Example 1 is executed.

[0119] The electronic device provided in this embodiment can execute the road edge detection method provided in Example 1, and will not be described again here to avoid repetition.

[0120] Example 4

[0121] In addition, an embodiment of the present disclosure provides a vehicle, including a vehicle body, a laser radar installed on the vehicle body, and the electronic device provided in Example 3; the laser radar is used to obtain original point cloud data.

[0122] The vehicle provided in this embodiment can execute the road edge detection method provided in Example 1, and to avoid repetition, it will not be described here.

[0123] Example 5

[0124] In addition, an embodiment of the present disclosure provides a computer-readable storage medium storing a computer program. When the computer program is run on a processor, the road edge detection method provided in embodiment 1 is executed.

[0125] The computer-readable storage medium of this embodiment can execute the road edge detection method provided in Example 1, which will not be described again here to avoid repetition.

[0126] In this embodiment, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0127] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or terminal comprising the element.

[0128] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0129] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A road edge detection method, characterized in that: The method comprises: Segment the original point cloud data collected by the lidar according to the preset segmentation threshold to obtain local point cloud data; Sampling the local point cloud data according to a preset voxel grid size to obtain sampled point cloud data; Classifying each three-dimensional point in the sampled point cloud data using a point cloud convolutional neural network model to determine a three-dimensional curb point; Acquire the facade of the curb marker according to the curb three-dimensional point; The curb position information is determined according to the curb marker elevation.

2. The method according to claim 1, characterized in that The sampling and processing of the local point cloud data according to the preset voxel grid size to obtain the sampled point cloud data includes: Segmenting the local point cloud data according to a preset three-dimensional voxel grid size to obtain a plurality of three-dimensional voxel grids; A preset number of sampling three-dimensional points are randomly sampled from all three-dimensional points of each three-dimensional voxel grid, and the sampling three-dimensional points of the plurality of the three-dimensional voxel grids are used as the sampling point cloud data.

3. The method according to claim 1 or 2, characterized in that The method of classifying the three-dimensional points in the sampled point cloud data using a point cloud convolutional neural network model to determine the three-dimensional curb points includes: Determine the curb point probability, road surface point probability, and other point probabilities of each three-dimensional point of the sampled point cloud data through the point cloud convolutional neural network model; The maximum probability value of the road surface point probability, the curb point probability and the other point probabilities among the three-dimensional points is determined, and the three-dimensional point with the maximum probability value of the curb point probability is determined as the curb three-dimensional point.

4. The method according to claim 3, characterized in that The determining of the curb point probability, road surface point probability, and other point probabilities of each three-dimensional point of the sampled point cloud data by the point cloud convolutional neural network model includes: When the point cloud convolutional neural network model is in a single-frame detection mode, converting the single-frame sampled point cloud data into a first matrix, the first matrix including N rows, each row of the first matrix representing coordinate information of each three-dimensional point of the sampled point cloud data; The first matrix is ​​input into the point cloud convolutional neural network model, and the first matrix is ​​calculated by the point cloud convolutional neural network model to obtain a second matrix, where the second matrix includes N rows, and each row of the second matrix represents the category probability of each three-dimensional point of the sampled point cloud data, where the category probability includes road surface point probability, curb point probability, and other point probabilities.

5. The method according to claim 3, characterized in that The determining of the curb point probability, road surface point probability, and other point probabilities of each three-dimensional point of the sampled point cloud data by the point cloud convolutional neural network model includes: When the point cloud convolutional neural network model is in a multi-frame detection mode, converting the multi-frame sampling point cloud data into a third matrix, the third matrix including a plurality of sub-matrices, each row of each sub-matrix of the third matrix representing the coordinate information of each three-dimensional point of each frame of sampling point cloud data; The third matrix is ​​input into the point cloud convolutional neural network model, and the third matrix is ​​calculated by the point cloud convolutional neural network model to obtain a fourth matrix, wherein the fourth matrix includes multiple sub-matrices, and each row of each sub-matrix of the fourth matrix represents the category probability of each three-dimensional point of each frame of sampled point cloud data, and the category probability includes road surface point probability, curb point probability and other point probabilities.

6. The method according to claim 1, 2, 4 or 5, characterized in that Acquiring a curb marker elevation according to the curb three-dimensional point includes: generating at least one geometric plane according to the three-dimensional points of the curb; When there is a target geometric plane whose shape matches the preset marker facade in at least one of the geometric planes, the target geometric plane is determined to be the curb marker facade.

7. The method according to claim 1, 2, 4 or 5, characterized in that The step of determining the curb position information according to the curb marker elevation includes: Determine the real position information of the curb marker facade according to the three-dimensional points corresponding to the curb marker facade; The actual position information of the facade of the curb marker is used as the curb position information.

8. A road edge detection device, characterized in that: The device comprises: The segmentation module is used to segment the original point cloud data collected by the lidar according to a preset segmentation threshold to obtain local point cloud data; a sampling module, configured to perform sampling processing on the local point cloud data according to a preset three-dimensional voxel grid size to obtain sampled point cloud data; A classification module is used to classify each 3D point in the sampled point cloud data through a point cloud convolutional neural network model to determine the 3D curb point; An acquisition module, configured to acquire a curb marker elevation based on the curb three-dimensional points; A determination module is used to determine the curb position information according to the facade of the curb marker.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is run on the processor, the road edge detection method according to any one of claims 1 to 7 is executed.

10. A computer-readable storage medium, characterized in that The device stores a computer program, which executes the road edge detection method according to any one of claims 1 to 7 when running on a processor.

Images