Road edge detection method, device, electronic device, vehicle and computer-readable storage medium

Abstract

The present application provides a road edge detection method, apparatus, electronic device, vehicle, and computer-readable storage medium, wherein the method comprises: segmenting raw point cloud data collected by a laser radar according to a preset segmentation threshold to obtain local point cloud data; sampling and processing the local point cloud data according to a preset voxel grid size to obtain sampled point cloud data; classifying each three-dimensional point in the sampled point cloud data using a point cloud convolutional neural network model to determine a three-dimensional curb point; obtaining a curb marker elevation based on the three-dimensional curb point; and determining curb position information based on the curb marker elevation. Thus, by segmenting and sampling the raw point cloud data to obtain sampled point cloud data, determining the curb marker elevation based on the sampled point cloud data using a point cloud convolutional neural network model, and determining the road edge position information based on the road edge marker position information, the stability and accuracy of curb detection results can be improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a road edge detection method, device, electronic device, vehicle, and computer-readable storage medium. Background Art

[0002] With the continuous development and advancement of artificial intelligence and modern sensor technologies, autonomous driving technology based on environmental perception has emerged. Accurate road edge detection is a crucial technology for achieving safe and autonomous driving. When a vehicle deviates from its lane for reasons such as parking or obstacle avoidance, the detected road edge position information is a crucial basis for the autonomous driving system to plan its path.

[0003] The main existing technical approach to detecting road edges is as follows: first, the coordinates of the road surface area are detected from a 3D point cloud or RGB image, and then the road edge is located by extracting the boundary coordinates of the road surface area. This approach works well in open areas with minimal interference, but in complex road scenes, it is susceptible to interference from other objects such as vehicles, resulting in poor road edge detection. Summary of the Invention

[0004] In order to solve the above technical problems, embodiments of the present application provide a road edge detection method, device, electronic device, vehicle and computer-readable storage medium.

[0005] In a first aspect, an embodiment of the present application provides a road edge detection method, the method comprising:

[0006] Segment the original point cloud data collected by the lidar according to the preset segmentation threshold to obtain local point cloud data;

[0007] Sampling the local point cloud data according to a preset voxel grid size to obtain sampled point cloud data;

[0008] Classifying each three-dimensional point in the sampled point cloud data using a point cloud convolutional neural network model to determine a three-dimensional curb point;

[0009] Acquire the facade of the curb marker according to the curb three-dimensional point;

[0010] The curb position information is determined according to the curb marker elevation.

[0011] In a second aspect, an embodiment of the present application provides a road edge detection device, the device comprising:

[0012] The segmentation module is used to segment the original point cloud data collected by the lidar according to a preset segmentation threshold to obtain local point cloud data;

[0013] a sampling module, configured to perform sampling processing on the local point cloud data according to a preset three-dimensional voxel grid size to obtain sampled point cloud data;

[0014] A classification module is used to classify each 3D point in the sampled point cloud data through a point cloud convolutional neural network model to determine the 3D curb point;

[0015] An acquisition module, configured to acquire a curb marker elevation based on the curb three-dimensional points;

[0016] A determination module is used to determine the curb position information according to the facade of the curb marker.

[0017] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is run by the processor, the road edge detection method provided in the first aspect is executed.

[0018] In a fourth aspect, an embodiment of the present application provides a vehicle comprising a vehicle body, a laser radar mounted on the vehicle body, and the electronic device provided in the third aspect; the laser radar is used to obtain original point cloud data.

[0019] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which executes the road edge detection method provided in the first aspect when running on a processor.

[0020] The road edge detection method, device, electronic device, vehicle, and computer-readable storage medium provided by the present application segment the raw point cloud data collected by the laser radar according to a preset segmentation threshold to obtain local point cloud data; sample and process the local point cloud data according to a preset voxel grid size to obtain sampled point cloud data; classify each three-dimensional point in the sampled point cloud data using a point cloud convolutional neural network model to determine a three-dimensional curb point; obtain the elevation of the curb marker based on the three-dimensional curb point; and determine the curb position information based on the curb marker elevation. In this way, by segmenting and sampling the raw point cloud data to obtain sampled point cloud data, determining the elevation of the curb marker from the sampled point cloud data using a point cloud convolutional neural network model, and determining the position information of the road edge based on the position information of the road edge marker, the situation where the intersection line between the road surface and the moving vehicle is regarded as the road edge can be significantly avoided, thereby improving the stability and accuracy of the curb detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solution of this application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of this application and should not be regarded as limiting the scope of protection of this application. In each of the drawings, similar components are numbered similarly.

[0022] Figure 1 A schematic diagram of a process flow of a road edge detection method provided by an embodiment of the present application is shown;

[0023] Figure 2 A schematic diagram of the original point cloud data provided by an embodiment of the present application is shown;

[0024] Figure 3 A schematic diagram of local three-dimensional point cloud data provided by an embodiment of the present application is shown;

[0025] Figure 4 A schematic diagram of sampling point cloud data provided by an embodiment of the present application is shown;

[0026] Figure 5 A schematic diagram of a road edge detection effect provided by an embodiment of the present application is shown;

[0027] Figure 6 A schematic structural diagram of a road edge detection method and device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0029] The components of the embodiments of the present application generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but rather merely represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0030] Hereinafter, the terms "including", "having" and their cognates, which may be used in various embodiments of the present application, are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or the possibility of adding one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items.

[0031] Furthermore, the terms “first,” “second,” “third,” etc., are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.

[0032] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art to which the various embodiments of the present application belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meaning as in the context of the relevant technical field and will not be interpreted as having an idealized meaning or an overly formal meaning unless clearly defined in the various embodiments of the present application.

[0033] Example 1

[0034] An embodiment of the present disclosure provides a road edge detection method.

[0035] For details, see Figure 1 ,Road edge detection methods include:

[0036] Step S101 : segmenting the original point cloud data collected by the lidar according to a preset segmentation threshold to obtain local point cloud data.

[0037] In this embodiment, the road edge detection method can be applied to an electronic device mounted on a vehicle. The vehicle can also be equipped with a laser radar (LiDAR). The LiDAR can acquire raw point cloud data. The raw point cloud data includes multiple three-dimensional points, each representing position information in the three-dimensional coordinate system of the LiDAR. The three-dimensional points in the raw point cloud data are characterized by coordinate positions in a first direction, a second direction, and a third direction. For example, a three-dimensional coordinate system is established with the LiDAR as the coordinate origin. In this coordinate system, for example, the axes of the three-dimensional coordinate system can be as follows: the X-axis is parallel to the ground and points to the right of the LiDAR; the Y-axis is parallel to the ground and points forward of the LiDAR; and the Z-axis passes through the center of mass of the LiDAR and points upward. The axes of the three-dimensional coordinate system can also be other axes, which are not limited here. The three-dimensional points in the raw point cloud data can be represented by three-dimensional points (x, y, z), each of which represents a position in the three-dimensional coordinate system of the LiDAR.

[0038] See also Figure 2 , Figure 2 The figure shows a schematic diagram of the original point cloud data. The original point cloud data includes a large number of 3D points, which contain relatively rich ground information and road edge information. The following interference points may exist in the 3D points, and the interference points can be denoised.

[0039] It should be noted that the laser radar scans the surrounding environment and obtains raw point cloud data of the surrounding environment. The number of three-dimensional points in the raw point cloud data is relatively large and contains relatively rich environmental information. However, due to the large number of points, if road edge detection is performed directly based on the raw point cloud data, the raw point cloud data may contain a large number of interfering three-dimensional points, which will affect the calculation accuracy. In addition, the large number of three-dimensional points in the raw point cloud data will lead to a large amount of subsequent calculations and consume a lot of computing resources. In this embodiment, the raw point cloud data collected by the laser radar is segmented according to a preset segmentation threshold to obtain local point cloud data. The raw point cloud data can be pre-processed for denoising to reduce interference and the number of three-dimensional points, thereby saving computing resources and improving calculation accuracy.

[0040] It should be noted that, according to actual conditions, the effective scanning distance of the laser radar is limited due to the limitations of the configuration parameters. The distance that the laser radar can effectively scan can be called the maximum scanning distance. The distribution range of valid three-dimensional points in the original point cloud data in each coordinate axis direction can be estimated based on the maximum scanning distance of the laser radar and the width of the road surface. For example, the first direction preset threshold and the third direction preset threshold can be set to filter the three-dimensional points of the original point cloud data, remove the interfering three-dimensional points, and form the remaining valid three-dimensional points into local point cloud data. If the axes of the three-dimensional coordinate system of the laser radar are: the X-axis is parallel to the ground and points to the right side of the radar, the Y-axis is parallel to the ground and points to the front of the radar, and the Z-axis passes through the center of mass of the radar and points upward, the first direction preset threshold can be the Z-axis direction preset threshold, and the third direction preset threshold can be the X-axis direction preset threshold.

[0041] In this example, a 3D point set containing both road surface and curb information is extracted from the original point cloud and used as input data for subsequent processing. By filtering the original point cloud data, the amount of data required by the subsequent point cloud convolutional neural network model is reduced, improving overall detection speed.

[0042] In one embodiment, step S101 includes the following steps:

[0043] Determine a first target three-dimensional point in the original point cloud data, the absolute value of which in the first direction is less than or equal to a preset threshold in the first direction;

[0044] Determine a second target three-dimensional point in the original point cloud data, the value of the third direction being greater than or equal to a preset threshold value of the third direction;

[0045] The first target three-dimensional point and the second target three-dimensional point are deleted from the original point cloud data to obtain the local point cloud data.

[0046] See also Figure 3 , Figure 3 The figure shows a schematic diagram of local point cloud data, which is the remaining point cloud data after removing ground point cloud data and invalid interference points from the original point cloud data.

[0047] In one embodiment, the first direction preset threshold is determined based on the maximum number of lanes on the road, the maximum width of the road, and a preset detection range safety factor;

[0048] The preset threshold value of the third direction is determined based on the distance between the laser radar and the ground, the farthest scanning distance of the laser radar, and the maximum longitudinal slope of the road.

[0049] In one embodiment, if the first direction preset threshold is the X-axis direction preset threshold, and the third direction preset threshold is the Z-axis direction preset threshold, the X-axis direction preset threshold can be calculated according to the following formula 1:

[0050] Formula 1: ;

[0051] Where WX represents the preset threshold in the X-axis direction, NW represents the maximum number of lanes on the road, WL represents the maximum lane width, and A represents the detection range safety factor, which ranges from [1.5, 4].

[0052] In addition, the preset threshold in the Z-axis direction can be calculated according to the following formula 2:

[0053] Formula 2: ;

[0054] Among them, ZH represents the preset threshold in the Z-axis direction, ZL represents the distance between the center of mass of the lidar and the ground, YL represents the farthest scanning distance of the lidar in the Y-axis direction, and C represents the maximum longitudinal slope of the road.

[0055] For example, if the preset threshold in the X-axis direction is WX, then the step of determining the second target three-dimensional point whose absolute value of the second direction value in the original point cloud data is less than or equal to the preset threshold in the third direction can be understood as: determining the second target three-dimensional point whose second direction value in the original point cloud data is greater than or equal to -WX and less than or equal to WX, that is, the X-axis element value of the second target three-dimensional point is in the interval [-WX, WX].

[0056] Step S102 : Sampling the local point cloud data according to a preset voxel grid size to obtain sampled point cloud data.

[0057] In one embodiment, step S102 may include the following steps:

[0058] Segmenting the local point cloud data according to a preset three-dimensional voxel grid size to obtain a plurality of three-dimensional voxel grids;

[0059] A preset number of sampling three-dimensional points are randomly sampled from all three-dimensional points of each three-dimensional voxel grid, and the sampling three-dimensional points of the plurality of the three-dimensional voxel grids are used as the sampling point cloud data.

[0060] See also Figure 4 , Figure 4 The figure shows a schematic diagram of sampling point cloud data. The acquisition process of the sampling point cloud data is as follows: Figure 4 The local point cloud data shown is divided into a plurality of three-dimensional voxel grids; N sampling three-dimensional points are randomly sampled from all three-dimensional points of each three-dimensional voxel grid, and all the sampled three-dimensional points are used to form sampling point cloud data. Figure 5 The number of 3D points of the sampling point cloud data is less than Figure 4 In this way, the amount of calculation can be further reduced and computing resources can be saved.

[0061] In one embodiment, the preset voxel grid size may be determined based on a maximum edge detection error in the first direction, a maximum edge detection error in the second direction, and a minimum curb height in the third direction.

[0062] For example, a preset voxel grid size is [dx / 2, dy / 2, dz / 4], where dx is the maximum allowable edge detection error of the road edge in the use scenario along the X-axis, dy is the maximum allowable edge detection error of the road edge in the use scenario along the Y-axis, and dz is the lowest curb height recognizable by the lidar in the use scenario, which must be no less than twice the distance accuracy of the lidar. In one embodiment, the local point cloud data is divided into multiple voxel grids according to the [dx / 2, dy / 2, dz / 4] voxel grid size, and a corresponding 3D point is determined for each voxel grid.

[0063] In one embodiment, the position information of the center of gravity of each stereo voxel grid can be calculated based on the multiple three-dimensional points of each stereo voxel grid, the center of gravity point can be used as an adjusted three-dimensional point of each stereo voxel grid, and the multiple adjusted three-dimensional points can be used to form sampling point cloud data.

[0064] In this embodiment, the point cloud convolutional neural network model automatically generates a corresponding preset voxel grid size based on a custom edge detection accuracy setting, and downsamples the input sampled point cloud data based on this preset voxel grid size to reduce the point cloud data density. The higher the custom edge detection accuracy setting, the smaller the preset voxel grid size automatically generated by the point cloud convolutional neural network model, the greater the point cloud density, and the more accurate the detected road edge location information. However, this requires more storage space and increases the subsequent computational workload of the point cloud convolutional neural network model. When simultaneously detecting multiple frames of sampled point cloud data, after compressing the multiple frames according to the preset voxel grid size, the compressed point cloud data is randomly sampled at a fixed number of points to ensure that each frame contains the same number of points. For example, the compressed point cloud data is randomly sampled at sampling points N, where N is an integer multiple of 1024.

[0065] In this way, the detection accuracy and detection speed of the road edge detection solution can be effectively balanced. While ensuring the required detection accuracy, the point cloud density can be reduced as much as possible, thereby improving the detection speed.

[0066] Step S103: Classify each 3D point in the sampled point cloud data using a point cloud convolutional neural network model to determine a curb 3D point.

[0067] It should be noted that the point cloud convolutional neural network model is trained based on the category labels of each point in the historically collected point cloud and manually annotated point cloud, which can improve the accuracy of classification. The point cloud convolutional neural network model includes single-frame detection mode and multi-frame detection mode.

[0068] In one embodiment, step S103 includes the following steps:

[0069] Determine the curb point probability, road surface point probability, and other point probabilities of each three-dimensional point of the sampled point cloud data through the point cloud convolutional neural network model;

[0070] The maximum probability value of the road surface point probability, the curb point probability and the other point probabilities among the three-dimensional points is determined, and the three-dimensional point with the maximum probability value of the curb point probability is determined as the curb three-dimensional point.

[0071] For example, the detection result of 3D point A is [70%, 20%, 10%]. The probability that 3D point A is a curb point is 70%, the probability that 3D point A is on the road surface is 20%, and the probability that 3D point A is in other areas is 10%. Therefore, 3D point A is classified as a curb point.

[0072] In one embodiment, determining the curb point probability, road surface point probability, and other point probabilities of each three-dimensional point of the sampled point cloud data using the point cloud convolutional neural network model may include the following steps:

[0073] When the point cloud convolutional neural network model is in a single-frame detection mode, converting the single-frame sampled point cloud data into a first matrix, the first matrix including N rows, each row of the first matrix representing coordinate information of each three-dimensional point of the sampled point cloud data;

[0074] The first matrix is ​​input into the point cloud convolutional neural network model, and the first matrix is ​​calculated by the point cloud convolutional neural network model to obtain a second matrix, where the second matrix includes N rows, and each row of the second matrix represents the category probability of each three-dimensional point of the sampled point cloud data, where the category probability includes road surface point probability, curb point probability, and other point probabilities.

[0075] Specifically, the point cloud data processing process of the single-frame detection mode is as follows: the sampled point cloud data is formatted into a first matrix with N rows and 3 columns, where N is the number of 3D points contained in the sampled point cloud data. The i-th row of the first matrix represents the coordinates of the i-th input 3D point [xi, yi, zi]. This first matrix is ​​input into the point cloud convolutional neural network model. After the point cloud convolutional neural network model calculates the input sampled point cloud data, a second matrix with N rows and 3 columns is obtained. The i-th row of the second matrix represents the detection result of the i-th input 3D point [Pi0, Pi1, Pi2], where Pi0 is the probability that the i-th input 3D point belongs to the road surface point, Pi1 is the probability that the i-th input 3D point belongs to the curb point, and Pi2 is the probability that the i-th input 3D point belongs to other points in other areas. The category with the highest probability in the detection result corresponding to each point is the category to which the point belongs.

[0076] In another embodiment, determining the curb point probability, road surface point probability, and other point probabilities of each three-dimensional point of the sampled point cloud data using the point cloud convolutional neural network model may include the following steps:

[0077] When the point cloud convolutional neural network model is in a multi-frame detection mode, converting the multi-frame sampling point cloud data into a third matrix, the third matrix including a plurality of sub-matrices, each row of each sub-matrix of the third matrix representing the coordinate information of each three-dimensional point of each frame of sampling point cloud data;

[0078] The third matrix is ​​input into the point cloud convolutional neural network model, and the third matrix is ​​calculated by the point cloud convolutional neural network model to obtain a fourth matrix, wherein the fourth matrix includes multiple sub-matrices, and each row of each sub-matrix of the fourth matrix represents the category probability of each three-dimensional point of each frame of sampled point cloud data, and the category probability includes road surface point probability, curb point probability and other point probabilities.

[0079] Specifically, the processing of sampled point cloud data for multi-frame detection is as follows: the multi-frame sampled point cloud data to be identified is formatted into a B*N*3 third matrix, which is equivalent to having B N*3 sub-matrices. The formatted B*N*3 third matrix is ​​input into the point cloud convolutional neural network model. After the point cloud convolutional neural network model calculates the input B*N*3 third matrix, it outputs a B*N*3 fourth matrix, which is equivalent to having B N*3 sub-matrices, where B is the number of frames of standard power data input to the point cloud convolutional neural network model, and N is the number of three-dimensional points contained in a frame of sampled point cloud data. When B = j, j = 1, 2, 3, etc., the formatted j*N*3 sub-matrix is ​​the sampled point cloud data corresponding to the j-th frame of sampled point cloud data, and the output j*N*3 matrix is ​​the classification result corresponding to the j-th frame of sampled point cloud data.

[0080] Step S104: Acquire the facade of the curb marker according to the curb three-dimensional points.

[0081] In this embodiment, step S104 may include the following steps:

[0082] generating at least one geometric plane according to the three-dimensional points of the curb;

[0083] When there is a target geometric plane whose shape matches the preset marker facade in at least one of the geometric planes, the target geometric plane is determined to be the curb marker facade.

[0084] In one embodiment, there are protrusions such as road dividing guardrails, road posts, and curbs on the reference curb. These protrusions can be used as markers, and the facade morphology of the curb marker can be obtained in advance. After each three-dimensional point is classified, it can be clear whether each three-dimensional point belongs to a curb point, a road surface point, or other point. In this way, multiple geometric planes are generated based on all curb points, and the target geometric plane that matches the preset marker facade is determined as the curb marker facade. The curb marker facade can be the facade of the curb protrusion, so by detecting the curb marker facade, it can be determined that the curb has been detected.

[0085] See also Figure 5 , Figure 5 The following is a schematic diagram of the curb detection effect. Figure 5The roadside marker facade 501 is included, and the roadside marker facade 501 can be a road dividing guardrail, a road bollard, a curb or other protruding object facade.

[0086] Step S105: determining the curb position information according to the curb marker elevation.

[0087] In this embodiment, step S105 may include the following steps:

[0088] Determine the real position information of the curb marker facade according to the three-dimensional points corresponding to the curb marker facade;

[0089] The actual position information of the facade of the curb marker is used as the curb position information.

[0090] In this embodiment, since the curb marker's elevation is the elevation of a raised feature on the curb, it can be determined that the curb has been detected. Based on the three-dimensional point corresponding to the curb marker's elevation in the LiDAR coordinate system, the real-world location of the curb marker's elevation can be converted and used to determine the curb's position.

[0091] The road edge detection method provided in this embodiment segments the raw point cloud data collected by the laser radar according to a preset segmentation threshold to obtain local point cloud data; samples and processes the local point cloud data according to a preset voxel grid size to obtain sampled point cloud data; classifies each three-dimensional point in the sampled point cloud data using a point cloud convolutional neural network model to determine the three-dimensional road edge points; obtains the elevation of the road edge marker based on the three-dimensional road edge points; and determines the road edge position information based on the elevation of the road edge marker. In this way, by segmenting and sampling the raw point cloud data to obtain sampled point cloud data, determining the elevation of the road edge marker from the sampled point cloud data using a point cloud convolutional neural network model, and determining the position information of the road edge based on the position information of the road edge marker, the method can significantly avoid mistaking the intersection of the road surface and the moving vehicle as the road edge, thereby improving the stability and accuracy of the road edge detection results.

[0092] Example 2

[0093] In addition, an embodiment of the present disclosure provides a road edge detection device.

[0094] Specifically, such as Figure 6 As shown, the road edge detection device 600 includes:

[0095] The segmentation module 601 is used to segment the original point cloud data collected by the laser radar according to a preset segmentation threshold to obtain local point cloud data;

[0096] A sampling module 602 is configured to perform sampling processing on the local point cloud data according to a preset voxel grid size to obtain sampled point cloud data;

[0097] A classification module 603 is used to classify each 3D point in the sampled point cloud data using a point cloud convolutional neural network model to determine a 3D curb point;

[0098] An acquisition module 604 is configured to acquire a curb marker elevation based on the curb three-dimensional points;

[0099] The determination module 605 is configured to determine the curb position information according to the curb marker elevation.

[0100] In this embodiment, the segmentation module 601 is further configured to determine a first target three-dimensional point in the original point cloud data whose absolute value of a first direction value is less than or equal to a preset threshold value in the first direction;

[0101] Determine a second target three-dimensional point in the original point cloud data, the value of the third direction being greater than or equal to a preset threshold value of the third direction;

[0102] The first target three-dimensional point and the second target three-dimensional point are deleted from the original point cloud data to obtain the local point cloud data.

[0103] In this embodiment, the first direction preset threshold is determined based on the maximum number of lanes on the road, the maximum width of the road, and the preset detection range safety factor; the third direction preset threshold is determined based on the distance between the laser radar and the ground, the farthest scanning distance of the laser radar, and the maximum longitudinal slope of the road.

[0104] In this embodiment, the sampling module 602 is further configured to segment the local point cloud data according to a preset 3D voxel grid size to obtain a plurality of 3D voxel grids;

[0105] A preset number of sampling three-dimensional points are randomly sampled from all three-dimensional points of each three-dimensional voxel grid, and the sampling three-dimensional points of the plurality of the three-dimensional voxel grids are used as the sampling point cloud data.

[0106] In this embodiment, the classification module 603 is further configured to determine the curb point probability, road surface point probability, and other point probability of each three-dimensional point of the sampled point cloud data through the point cloud convolutional neural network model;

[0107] The maximum probability value of the road surface point probability, the curb point probability and the other point probabilities among the three-dimensional points is determined, and the three-dimensional point with the maximum probability value of the curb point probability is determined as the curb three-dimensional point.

[0108] In this embodiment, the classification module 603 is further configured to, when the point cloud convolutional neural network model is in a single-frame detection mode, convert the single-frame sampled point cloud data into a first matrix, where the first matrix includes N rows, and each row of the first matrix represents coordinate information of each three-dimensional point of the sampled point cloud data;

[0109] The first matrix is ​​input into the point cloud convolutional neural network model, and the first matrix is ​​calculated by the point cloud convolutional neural network model to obtain a second matrix, where the second matrix includes N rows, and each row of the second matrix represents the category probability of each three-dimensional point of the sampled point cloud data, where the category probability includes road surface point probability, curb point probability, and other point probabilities.

[0110] In this embodiment, the classification module 603 is further configured to, when the point cloud convolutional neural network model is in a multi-frame detection mode, convert the multi-frame sampled point cloud data into a third matrix, wherein the third matrix includes a plurality of sub-matrices, and each row of each sub-matrix of the third matrix represents the coordinate information of each three-dimensional point of each frame of the sampled point cloud data;

[0111] The third matrix is ​​input into the point cloud convolutional neural network model, and the third matrix is ​​calculated by the point cloud convolutional neural network model to obtain a fourth matrix, wherein the fourth matrix includes multiple sub-matrices, and each row of each sub-matrix of the fourth matrix represents the category probability of each three-dimensional point of each frame of sampled point cloud data, and the category probability includes road surface point probability, curb point probability and other point probabilities.

[0112] In this embodiment, the acquisition module 604 is further configured to generate at least one geometric plane according to the three-dimensional points of the curb;

[0113] When there is a target geometric plane whose shape matches the preset marker facade in at least one of the geometric planes, the target geometric plane is determined to be the curb marker facade.

[0114] In this embodiment, the determination module 605 is further configured to determine the real position information of the curb marker facade according to the three-dimensional points corresponding to the curb marker facade;

[0115] The actual position information of the facade of the curb marker is used as the curb position information.

[0116] The road edge detection device provided in this embodiment segments the raw point cloud data collected by the laser radar according to a preset segmentation threshold to obtain local point cloud data; samples and processes the local point cloud data according to a preset voxel grid size to obtain sampled point cloud data; classifies each three-dimensional point in the sampled point cloud data using a point cloud convolutional neural network model to determine a three-dimensional curb point; obtains the elevation of a curb marker based on the three-dimensional curb point; and determines curb position information based on the elevation of the curb marker. In this way, by segmenting and sampling the raw point cloud data to obtain sampled point cloud data, determining the elevation of a curb marker based on the sampled point cloud data using a point cloud convolutional neural network model, and determining the position information of the road edge based on the position information of the road edge marker, the device can significantly avoid mistaking the intersection of the road surface and the moving vehicle as the road edge, thereby improving the stability and accuracy of curb detection results.

[0117] Example 3

[0118] In addition, an embodiment of the present disclosure provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the road edge detection method provided in Example 1 is executed.

[0119] The electronic device provided in this embodiment can execute the road edge detection method provided in Example 1, and will not be described again here to avoid repetition.

[0120] Example 4

[0121] In addition, an embodiment of the present disclosure provides a vehicle, including a vehicle body, a laser radar installed on the vehicle body, and the electronic device provided in Example 3; the laser radar is used to obtain original point cloud data.

[0122] The vehicle provided in this embodiment can execute the road edge detection method provided in Example 1, and to avoid repetition, it will not be described here.

[0123] Example 5

[0124] In addition, an embodiment of the present disclosure provides a computer-readable storage medium storing a computer program. When the computer program is run on a processor, the road edge detection method provided in embodiment 1 is executed.

[0125] The computer-readable storage medium of this embodiment can execute the road edge detection method provided in Example 1, which will not be described again here to avoid repetition.

[0126] In this embodiment, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0127] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or terminal comprising the element.

[0128] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0129] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A road edge detection method, characterized in that: The method comprises: Segmenting the original point cloud data collected by the laser radar according to a preset segmentation threshold to obtain local point cloud data; the preset segmentation threshold is determined according to the maximum scanning distance of the laser radar and the road width, and the preset segmentation threshold includes a first direction preset threshold and a third direction preset threshold; Sampling the local point cloud data according to a preset voxel grid size to obtain sampled point cloud data; Classifying each three-dimensional point in the sampled point cloud data using a point cloud convolutional neural network model to determine a three-dimensional curb point; Acquire the facade of the curb marker according to the curb three-dimensional point; Determining curb position information based on the curb marker elevation; Segment the raw point cloud data collected by the lidar according to the preset segmentation threshold, including: Setting the first direction preset threshold and the third direction preset threshold according to the maximum scanning distance of the laser radar and the road width; The three-dimensional points of the original point cloud data are screened according to the first direction preset threshold and the third direction preset threshold, interfering three-dimensional points are removed, and the remaining valid three-dimensional points are used to form the local point cloud data.

2. The method according to claim 1, characterized in that The raw point cloud data collected by the laser radar is segmented according to a preset segmentation threshold to obtain local point cloud data, including: Determine a first target three-dimensional point in the original point cloud data, the absolute value of which in the first direction is less than or equal to a preset threshold in the first direction; Determine a second target three-dimensional point in the original point cloud data, the value of the third direction being greater than or equal to a preset threshold value of the third direction; The first target three-dimensional point and the second target three-dimensional point are deleted from the original point cloud data to obtain the local point cloud data.

3. The method according to claim 2, characterized in that The first direction preset threshold is determined based on the maximum number of lanes on the road, the maximum width of the road, and the preset detection range safety factor; the third direction preset threshold is determined based on the distance between the laser radar and the ground, the farthest scanning distance of the laser radar, and the maximum longitudinal slope of the road.

4. The method according to claim 1, wherein The sampling and processing of the local point cloud data according to the preset voxel grid size to obtain the sampled point cloud data includes: Segmenting the local point cloud data according to a preset three-dimensional voxel grid size to obtain a plurality of three-dimensional voxel grids; A preset number of sampling three-dimensional points are randomly sampled from all three-dimensional points of each three-dimensional voxel grid, and the sampling three-dimensional points of the plurality of the three-dimensional voxel grids are used as the sampling point cloud data.

5. The method according to claim 1, wherein The method of classifying the three-dimensional points in the sampled point cloud data using a point cloud convolutional neural network model to determine the three-dimensional curb points includes: Determine the curb point probability, road surface point probability, and other point probabilities of each three-dimensional point of the sampled point cloud data through the point cloud convolutional neural network model; The maximum probability value of the road surface point probability, the curb point probability and the other point probabilities among the three-dimensional points is determined, and the three-dimensional point with the maximum probability value of the curb point probability is determined as the curb three-dimensional point.

6. The method according to claim 5, characterized in that The determining of the curb point probability, road surface point probability, and other point probabilities of each three-dimensional point of the sampled point cloud data by the point cloud convolutional neural network model includes: When the point cloud convolutional neural network model is in a single-frame detection mode, converting the single-frame sampled point cloud data into a first matrix, the first matrix including N rows, each row of the first matrix representing coordinate information of each three-dimensional point of the sampled point cloud data; The first matrix is ​​input into the point cloud convolutional neural network model, and the first matrix is ​​calculated by the point cloud convolutional neural network model to obtain a second matrix, where the second matrix includes N rows, and each row of the second matrix represents the category probability of each three-dimensional point of the sampled point cloud data, where the category probability includes road surface point probability, curb point probability, and other point probabilities.

7. The method according to claim 5, characterized in that The determining of the curb point probability, road surface point probability, and other point probabilities of each three-dimensional point of the sampled point cloud data by the point cloud convolutional neural network model includes: When the point cloud convolutional neural network model is in a multi-frame detection mode, converting the multi-frame sampling point cloud data into a third matrix, the third matrix including a plurality of sub-matrices, each row of each sub-matrix of the third matrix representing the coordinate information of each three-dimensional point of each frame of sampling point cloud data; The third matrix is ​​input into the point cloud convolutional neural network model, and the third matrix is ​​calculated by the point cloud convolutional neural network model to obtain a fourth matrix, wherein the fourth matrix includes multiple sub-matrices, and each row of each sub-matrix of the fourth matrix represents the category probability of each three-dimensional point of each frame of sampled point cloud data, and the category probability includes road surface point probability, curb point probability and other point probabilities.

8. The method according to claim 1, characterized in that Acquiring a curb marker elevation according to the curb three-dimensional point includes: generating at least one geometric plane according to the three-dimensional points of the curb; When there is a target geometric plane whose shape matches the preset marker facade in at least one of the geometric planes, the target geometric plane is determined to be the curb marker facade.

9. The method according to claim 1, characterized in that The step of determining the curb position information according to the curb marker elevation includes: Determine the real position information of the curb marker facade according to the three-dimensional points corresponding to the curb marker facade; The actual position information of the facade of the curb marker is used as the curb position information.

10. A road edge detection device, characterized in that: The device comprises: a segmentation module, configured to segment the raw point cloud data collected by the laser radar according to a preset segmentation threshold to obtain local point cloud data; the preset segmentation threshold is determined based on the maximum scanning distance of the laser radar and the road width, and the preset segmentation threshold includes a first direction preset threshold and a third direction preset threshold; a sampling module, configured to perform sampling processing on the local point cloud data according to a preset three-dimensional voxel grid size to obtain sampled point cloud data; A classification module is used to classify each 3D point in the sampled point cloud data through a point cloud convolutional neural network model to determine the 3D curb point; An acquisition module, configured to acquire a curb marker elevation based on the curb three-dimensional points; A determination module, configured to determine the curb position information based on the curb marker elevation; The segmentation module is further configured to set the first direction preset threshold and the third direction preset threshold according to the maximum scanning distance of the laser radar and the road width; The three-dimensional points of the original point cloud data are screened according to the first direction preset threshold and the third direction preset threshold, interfering three-dimensional points are removed, and the remaining valid three-dimensional points are used to form the local point cloud data.

11. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is run on the processor, the road edge detection method according to any one of claims 1 to 9 is executed.

12. A vehicle comprising a vehicle body, characterized in that: It also includes a laser radar installed on the vehicle body and an electronic device as described in claim 11; the laser radar is used to obtain original point cloud data.

13. A computer-readable storage medium, characterized in that The device stores a computer program, which executes the road edge detection method according to any one of claims 1 to 9 when the computer program is run on a processor.

Images