Roadside detection method and device, vehicle and storage medium

Abstract

The application relates to a road edge detection method and device, a vehicle and a storage medium. The method comprises the following steps: acquiring original point cloud data, performing filtering processing and ROI (region of interest) division on the original point cloud data, and obtaining a point cloud ROI region; inputting the point cloud data in the point cloud ROI region into a preset road edge detection network model to obtain point cloud information of road edge key points, wherein the preset road edge detection network model is composed of a point cloud feature extraction structure network, a 2D backbone network, a Decoder network and a Head network; based on a preset clustering and polar coordinate system sorting method, the point cloud information of the road edge key points is processed to obtain a polyline of the road edge, and the shape of the road edge is fitted according to the polyline of the road edge. Thus, the problems of poor migration and poor effect in the related art are solved, and the migration and reusability of road edge detection are enhanced.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a curb detection method, device, vehicle, and storage medium. Background Technology

[0002] With the development of intelligent driving technology, more and more sensors are being used in autonomous vehicles, and LiDAR, with its stable and light-resistant characteristics, is being used more and more widely. As the scope of autonomous driving routes continues to expand, and high-precision maps have limited coverage areas due to their high production costs, curb detection is playing an increasingly important role in autonomous driving technology, especially in areas that are difficult to cover by maps. The detected curbs then become the boundary lines of the drivable area. Since point cloud information is less rich than image information, and roads are often winding in the distance, and the appearance of curbs is not standardized, extracting a recognizable curb from this disordered point cloud data has become a very challenging task.

[0003] Most related technologies perform curb detection based on the curvature characteristics of point cloud information changes. A set of parameters is applicable to a specific scenario and may achieve relatively good results in the current scenario.

[0004] However, the transferability of related technologies is not strong. Given the diversity of autonomous driving scenarios, it is difficult for a single set of parameters to meet all scenarios. Moreover, the curvature change of point clouds at a distance is not obvious, making it difficult to distinguish between road edges and outliers. Summary of the Invention

[0005] This application provides a curb detection method, device, vehicle, and storage medium to solve the problems of curb detection being easily affected, having poor portability, and having poor performance in related technologies. It can improve the performance of curb point cloud detection, enhance the portability and reusability of curb detection, reduce computing power requirements while meeting the needs of engineering deployment, and meet the needs of real-time detection.

[0006] The first aspect of this application provides a curb detection method, comprising the following steps: acquiring raw point cloud data, and performing filtering and region of interest (ROI) division on the raw point cloud data to obtain point cloud ROI regions; inputting the point cloud data within the point cloud ROI regions into a preset curb detection network model to obtain point cloud information of curb key points, wherein the preset curb detection network model consists of a point cloud feature extraction structure network, a 2D (two-dimensional) backbone network, a decoder network, and a head network; and processing the point cloud information of the curb key points based on a preset clustering and polar coordinate system sorting method to obtain the curb polyline, and fitting the curb shape based on the curb polyline.

[0007] Based on the above technical means, this application can solve the problems of roadside detection being easily affected, having poor portability, and having poor performance in related technologies. It can improve the performance of roadside point cloud detection, enhance the portability and reusability of roadside detection, reduce computing power requirements while meeting the needs of engineering deployment, and meet the needs of real-time detection.

[0008] Optionally, in some embodiments, the step of inputting the point cloud data within the point cloud ROI region into a preset roadside detection network model to obtain point cloud information of roadside key points includes: processing the point cloud data within the point cloud ROI region based on the point cloud feature extraction structure network to obtain 3D sparse convolution results; obtaining sparse features based on the 3D sparse convolution results, and projecting the sparse features onto the BEV (Bird's EyeView) plane based on the 2D backbone network, and performing feature extraction on the 2D plane through the Decoder network to obtain target point cloud features; and inputting the target point cloud features into the Head network to obtain the point cloud information of the roadside key points.

[0009] Based on the above-mentioned technical means, this application can improve the effect of curb point cloud detection, and it is applicable to multiple scenarios of autonomous driving with strong transferability.

[0010] Optionally, in some embodiments, the step of processing the point cloud data within the point cloud ROI region based on the point cloud feature extraction structure network to obtain a 3D sparse convolution result includes: performing gridding processing on the point cloud data within the point cloud ROI region based on a preset voxel grid to obtain multiple voxel grids; and downsampling or filling each voxel grid according to the maximum number of points in each voxel grid to obtain the 3D sparse convolution result.

[0011] Based on the above-mentioned technical means, this application can detect distant roadside point clouds, with a wide detection range, and has a good detection effect on roadside point clouds with large curvature or occlusion.

[0012] Optionally, in some embodiments, the Head network includes a classification loss function and a regression loss function. The step of inputting the target point cloud features into the Head network to obtain the point cloud information of the roadside key points includes: classifying the target point cloud features according to the classification loss function to obtain a point cloud classification result; regressing the target point cloud features according to the regression loss function to obtain the position of the roadside; and obtaining the point cloud information of the roadside key points based on the point cloud classification result and the position of the roadside.

[0013] Based on the above technical means, this application can reduce the computing power requirement for point cloud feature extraction, enable the deployment of embedded low-computing-power platforms and meet the requirements for real-time detection, thus fulfilling the requirements for practical application.

[0014] Optionally, in some embodiments, the 2D backbone network is a bevbackbone.

[0015] Based on the above technical means, this application can directly fit the line segment representation of the road edge through a simple post-processing method, which improves time efficiency and reduces computing power requirements.

[0016] A second aspect of this application provides a curb detection device, comprising: an acquisition module for acquiring raw point cloud data, and performing filtering and Region of Interest (ROI) segmentation on the raw point cloud data to obtain point cloud ROI regions; an input module for inputting the point cloud data within the point cloud ROI regions into a preset curb detection network model to obtain point cloud information of curb key points, wherein the preset curb detection network model consists of a point cloud feature extraction structure network, a 2D backbone network, a decoder network, and a head network; and a processing module for processing the point cloud information of the curb key points based on a preset clustering and polar coordinate system sorting method to obtain a curb polyline, and fitting the curb shape based on the curb polyline.

[0017] Optionally, in some embodiments, the input module further includes: a processing unit, configured to process point cloud data within the point cloud ROI region based on the point cloud feature extraction structure network to obtain a 3D sparse convolution result; an extraction unit, configured to obtain sparse features based on the 3D sparse convolution result, project the sparse features onto the BEV plane based on the 2D backbone network, and extract features on the 2D plane through the Decoder network to obtain target point cloud features; and a generation unit, configured to input the target point cloud features into the Head network to obtain the point cloud information of the roadside key points.

[0018] Optionally, in some embodiments, the processing unit is specifically used to: perform gridding processing on the point cloud data within the point cloud ROI region based on a preset voxel grid to obtain multiple voxel grids; and downsample or fill each voxel grid according to the maximum number of point clouds in each voxel grid to obtain the 3D sparse convolution result.

[0019] Optionally, in some embodiments, the Head network includes a classification loss function and a regression loss function. The step of inputting the target point cloud features into the Head network to obtain the point cloud information of the roadside key points includes: classifying the target point cloud features according to the classification loss function to obtain a point cloud classification result; regressing the target point cloud features according to the regression loss function to obtain the position of the roadside; and obtaining the point cloud information of the roadside key points based on the point cloud classification result and the position of the roadside.

[0020] Optionally, in some embodiments, the 2D backbone network is a bevbackbone.

[0021] A third aspect of this application provides a vehicle including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the curb detection method as described in the above embodiments.

[0022] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the curb detection method as described in the above embodiments.

[0023] Therefore, by acquiring raw point cloud data, processing it to obtain the point cloud ROI region, performing voxelized mesh point cloud feature extraction, obtaining 3D sparse convolution results, and projecting the sparse features onto the BEV (Browser Elevation Vehicle), point cloud features are extracted using a 2D backbone, and a decoder is used to detect key points on the roadside. The decoder-processed point cloud features are used as input to the detection head, which outputs key points, and post-processing outputs a smoothed curve. This solves the problems of susceptibility to interference, poor transferability, and unsatisfactory performance in related technologies, improving the performance of roadside point cloud detection, enhancing the transferability and reusability of roadside detection, meeting the requirements of engineering deployment while reducing computing power requirements, and satisfying the needs of real-time detection.

[0024] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0025] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0026] Figure 1 This is a flowchart of a curb detection method provided according to an embodiment of this application;

[0027] Figure 2 This is a schematic diagram of point cloud feature extraction under BEV according to a specific embodiment of this application;

[0028] Figure 3 This is a schematic diagram of an Encoder and Decoder according to a specific embodiment of this application;

[0029] Figure 4 This is a schematic diagram of loss calculation from a BEV perspective for the Head portion provided according to a specific embodiment of this application;

[0030] Figure 5 This is a schematic diagram of a post-processing method provided according to a specific embodiment of this application;

[0031] Figure 6 This is a flowchart of a curb detection method according to a specific embodiment of this application;

[0032] Figure 7 This is a block diagram of a curb detection device provided according to an embodiment of this application;

[0033] Figure 8 This is a schematic diagram of a vehicle provided according to an embodiment of this application.

[0034] Explanation of reference numerals in the attached drawings: 10-curb detection device, 100-acquisition module, 200-input module, and 300-processing module. Detailed Implementation

[0035] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0036] Before introducing the curb detection method of the embodiments of this application, let's briefly introduce the curb detection methods in related technologies.

[0037] One proposed method for curb detection is based on LiDAR and fan-shaped spatial segmentation. This method acquires point cloud information using LiDAR scanning and preprocesses it to separate and extract ground data. Based on the data characteristics of LiDAR and point cloud, the space in the coordinate system is segmented into fan-shaped structures, and the road extension direction is identified. Candidate curb points in the point cloud are extracted and clustered to eliminate interference points. Finally, B-spline curves are fitted to the determined curb points to obtain the curb detection result. Another method is based on binocular vision. This method first acquires road image data and performs camera calibration, then preprocesses it to obtain a disparity map and determine the region of interest. The appearance and geometric features of the curb are extracted and fused. Inverse perspective transformation is performed to obtain a bird's-eye view. Smaller areas are then filtered out, along with interference from the road surface and roadside. Edge information is extracted from the image to filter out left and right curb candidate regions. Further filtering of interference points based on geometric features results in the selection of left and right curbs. The curb detection results are tracked and corrected to obtain the final detection result, completing the curb detection process.

[0038] However, the first method struggles to extract curb points at road edges, making it difficult to separate distant ground information from fan-shaped structures. Furthermore, the detected curb points are affected by traffic and occlusion, resulting in poor post-processing fitting. It is more suitable for curb detection in specific scenarios and for curbs at close range, limiting its effectiveness to specific conditions. Moreover, beyond 50m, the radar's curvature characteristics become less pronounced, making long-distance detection difficult. At high speeds, short-range detection is insufficient. The second method, based on visual features, uses Hough transform to extract road edge information, but this is only suitable for regular road sections with poor transferability. It performs poorly in discontinuous and occluded conditions, and its detection distance is limited, failing to detect distant curbs. In complex and rapidly changing scenarios, the Hough transform-based method also struggles to achieve good results.

[0039] To address the aforementioned issues, this application provides a roadside detection method. This method involves acquiring raw point cloud data, filtering and dividing the raw point cloud data into Regions of Interest (ROIs), obtaining point cloud ROI regions, and inputting the point cloud data within the ROI regions into a preset roadside detection network model to obtain point cloud information of key roadside points. The preset roadside detection network model consists of a point cloud feature extraction structure network, a 2D backbone network, a decoder network, and a head network. Based on a preset clustering and polar coordinate system sorting method, the point cloud information of the key roadside points is processed to obtain the roadside polyline, and the shape of the roadside is obtained by fitting the roadside polyline. This method solves the problems of susceptibility to interference, poor transferability, and unsatisfactory performance in related technologies, improving the effectiveness of roadside point cloud detection, enhancing the transferability and reusability of roadside detection, reducing computational requirements while meeting engineering deployment needs, and satisfying the requirements of real-time detection.

[0040] Specifically, Figure 1 This is a schematic flowchart of a curb detection method provided in an embodiment of this application.

[0041] like Figure 1 As shown, the curb detection method includes the following steps:

[0042] In step S101, the original point cloud data is acquired, and the original point cloud data is filtered and the region of interest (ROI) is divided to obtain the point cloud ROI region.

[0043] Specifically, based on prior statistics of point cloud data, it was found that the vast majority of road edges are below 50cm from the ground. The more point cloud data there is, the more computing resources are consumed. Therefore, in the preprocessing of point cloud data in this application embodiment, the original data is filtered and the ROI region is divided. Based on prior statistics and the actual application of autonomous driving, this application embodiment limits the overall point cloud ROI region to 120m before and after the x-axis, 80m to the left and right of the y-axis, and 3m above and below the z-axis. The data within this ROI completely includes the road edge data, meeting the requirements for road edge detection, while consuming fewer computing resources than the original data.

[0044] In step S102, the point cloud data within the point cloud ROI region is input into a preset road edge detection network model to obtain point cloud information of key road edge points. The preset road edge detection network model consists of a point cloud feature extraction structure network, a 2D backbone network, a decoder network, and a head network.

[0045] Those skilled in the art will understand that the input for 2D deep learning object detection in related technologies is a regular image size, and the point cloud is unordered, so it cannot be directly processed using 2D convolution. However, point cloud detection for road edges can achieve a greater detection distance and better detection effect from the BEV perspective. The embodiments of this application can perform voxelization processing on the point cloud from the BEV perspective to extract 2D pseudo-images, which are then input into the neural network to finally obtain the detected road edge data. This helps save computing resources and is more suitable for practical application.

[0046] Optionally, in some embodiments, the point cloud data within the point cloud ROI region is input into a preset roadside detection network model to obtain point cloud information of key roadside points, including: processing the point cloud data within the point cloud ROI region based on a point cloud feature extraction structure network to obtain 3D sparse convolution results; obtaining sparse features based on the 3D sparse convolution results, and projecting the sparse features onto the BEV plane based on a 2D backbone network, and extracting features on the 2D plane through a Decoder network to obtain target point cloud features; inputting the target point cloud features into a Head network to obtain point cloud information of key roadside points.

[0047] Optionally, in some embodiments, point cloud data within a point cloud ROI region is processed based on a point cloud feature extraction structure network to obtain 3D sparse convolution results, including: performing gridding processing on the point cloud data within the point cloud ROI region based on a preset voxel grid to obtain multiple voxel grids; downsampling or filling each voxel grid according to the maximum number of points in each voxel grid to obtain 3D sparse convolution results.

[0048] It should be understood that larger voxel grids achieve better processing speeds but reduce performance, while smaller voxel grids achieve better performance but reduce processing speed. This embodiment first divides the original point cloud into voxel grids, with the grid size being a hyperparameter determined by the number of point clouds in the LiDAR and the available computing power. Secondly, it defines the maximum number of point clouds in each grid, which is also a hyperparameter. Based on the maximum number of points, the number of points in each voxel grid is downsampled or padded. Converting the sparse 3D voxels to a BEV viewpoint yields more information and more comprehensive features than a range of views. Then, based on the four-dimensional features of the point cloud, feature extraction is performed in each voxel grid, essentially compressing it vertically to form a 2D pseudo-image from 3D space, allowing for 2D convolution operations. Therefore, compared to 3D convolution and other methods, it achieves better processing efficiency, as sparse convolution only computes the effective voxel units among all voxels. Statistics show that after point cloud voxelization, the effective voxels may only account for about 3% of all voxels. Therefore, using sparse convolution can reduce computing power to the greatest extent and retain 3D feature information to the greatest extent, making it more suitable for engineering implementation. Moreover, the limited height of the curb will not cause feature loss.

[0049] Optionally, in some embodiments, the 2D backbone network is bevbackbone.

[0050] Specifically, considering that 3D convolutional networks still require significant computational power for engineering applications, using 2D networks can reduce the computational demands. In this embodiment, after obtaining the result of 3D sparse convolution, as follows... Figure 2 As shown, sparse features can be projected onto the BEV (Bird Eye View) plane for 2D feature extraction, and a 2D backbone network can be used for further feature extraction.

[0051] Regarding the selection of the 2D backbone, this application embodiment conducted a series of comparative tests. Candidate backbones included ResNet18, ResNet34, ResNet50, and a bevbackbone with four downsampling iterations. See details... Figure 3 Backbone component. In ablation experiments, these four backbones were compared. Under the same data, bevbackbone achieved the best speed with negligible performance reduction, resulting in the best performance for engineering applications. Therefore, this application seems to favor using bevbackbone as the backbone network for point cloud feature extraction.

[0052] In autonomous driving scenarios, road edges are often irregular linear shapes, frequently truncated rather than continuous line segments, and exhibit significant curvature at distances. This poses a significant detection challenge, and detection anchors struggle to represent the linear irregularity of road edges. Linear anchors from line-CNNs also fail to adequately represent truncated and significantly curved shapes at distances. Therefore, this application's embodiments abandon anchor-based representation methods and employ an anchor-free approach for road edge detection. This significantly enhances subsequent model transferability and improves detection performance.

[0053] Based on 2D segmentation methods, embodiments of this application design a decoder part based on upsampling and concatenation for detecting key points along the road edge, such as... Figure 3 As shown, corresponding to the previous four downsampling operations, this stage employs four upsampling operations. While preserving the features of the current dimension, it can concatenate with the previous downsampling, thus maintaining features at multiple scales and retaining more features for subsequent feature extraction.

[0054] Optionally, in some embodiments, the Head network includes a classification loss function and a regression loss function. The target point cloud features are input into the Head network to obtain the point cloud information of the roadside key points, including: classifying the target point cloud features according to the classification loss function to obtain the point cloud classification result; regressing the target point cloud features according to the regression loss function to obtain the position of the roadside; and obtaining the point cloud information of the roadside key points according to the point cloud classification result and the position of the roadside.

[0055] Those skilled in the art will understand that, as Figure 4 As shown, the Head section mainly consists of two loss functions: one is the loss for point cloud classification, and the other is the loss for position regression of points classified as roadside points. Finally, the key points of the roadside are obtained, and after connecting them, the final roadside line segment is obtained based on the previously calculated offset.

[0056] Specifically, after processing by the decoder, the point cloud features are used as input to the detection head. In this embodiment, a classification loss function Lcls and a regression loss function Lreg are obtained. Since the number of roadside points is relatively small compared to the total point cloud, focal loss performs well compared to other loss functions in handling this type of imbalanced data. Therefore, focal loss can be chosen as the classification function, and experiments have shown that focal loss does indeed perform well. For regression, the specific location of the roadside can be determined using a regression function, which directly determines the location of the roadside point cloud. Thus, the final loss can be expressed as…

[0057] Loss total=a*Lcls+Lreg

[0058] Where Loss total is the final loss function, Lcls is the classification loss function, Lreg is the regression loss function, and 'a' is used to balance the parameters between the two losses, making adaptive adjustments to the final loss function based on the data.

[0059] In step S103, based on a preset clustering and polar coordinate system sorting method, the point cloud information of the key points of the roadside is processed to obtain the polyline of the roadside, and the shape of the roadside is obtained by fitting the polyline of the roadside.

[0060] It is understandable that the point cloud information of the key points of the roadside obtained by step S102 is not smooth because there will be some deviation in the single point information. Therefore, the obtained point cloud information needs to be post-processed.

[0061] Specifically, the number of keypoints obtained from the point cloud in step S102 varies depending on the road segment, generally ranging from a few hundred to several hundred. The overall number of point points is not large, therefore the post-processing method does not consume many resources. The obtained roadside point cloud can reflect the shape of the roadside, but because it contains single-point information, directly connecting them into a smooth polyline is quite difficult. Therefore, as... Figure 5 As shown, clustering and polar coordinate system sorting methods can be used to smooth the obtained roadside polyline, thereby better fitting the overall shape of the roadside, facilitating direct operation and intuitive evaluation downstream.

[0062] To enable those skilled in the art to further understand the curb detection method of the present application embodiments, the following detailed description is provided in conjunction with specific embodiments.

[0063] like Figure 6 As shown, Figure 6 This is a flowchart of a curb detection method provided according to an embodiment of this application.

[0064] First, the 3D point cloud is preprocessed by meshing and voxelization. Features are extracted from the 3D point cloud data to generate pseudo-images, allowing the use of 2D convolutional networks instead of 3D convolutional networks. This improves overall efficiency, reduces computational resource usage, and facilitates real-time inference deployment. Then, an anchor-free method is used, abandoning the limitations imposed by anchors and instead employing key point detection to detect key points along the roadside for subsequent fitting. This enables the application in complex and varied scenarios for autonomous driving, enhancing transferability and addressing the challenges of discontinuous roadside edges and significant curvature at distant points.

[0065] The curb detection method proposed in this application involves acquiring raw point cloud data, processing the raw point cloud data to obtain point cloud ROI regions, extracting voxelized mesh point cloud features to obtain 3D sparse convolution results, projecting the sparse features onto a BEV (Bridge Elevation Vehicle), extracting point cloud features through a 2D backbone, and detecting key points on the curb using a decoder. The decoder-processed point cloud features are used as input to the detection head, which outputs the key points, and post-processes to output a smoothed curve. This solves the problems of susceptibility to interference, poor transferability, and unsatisfactory performance in related technologies, improving the effectiveness of curb point cloud detection, enhancing the transferability and reusability of curb detection, reducing computational requirements while meeting engineering deployment needs, and satisfying real-time detection requirements.

[0066] Next, the curb detection device according to an embodiment of this application is described with reference to the accompanying drawings.

[0067] Figure 7 This is a block diagram of a curb detection device according to an embodiment of this application.

[0068] like Figure 7 As shown, the curb detection device 10 includes: an acquisition module 100, an input module 200, and a processing module 300.

[0069] The acquisition module 100 is used to acquire raw point cloud data, and to perform filtering and Region of Interest (ROI) division on the raw point cloud data to obtain point cloud ROI regions. The input module 200 is used to input the point cloud data within the point cloud ROI regions into a preset roadside detection network model to obtain point cloud information of key points on the roadside. The preset roadside detection network model consists of a point cloud feature extraction structure network, a 2D backbone network, a decoder network, and a head network. The processing module 300 is used to process the point cloud information of key points on the roadside based on a preset clustering and polar coordinate system sorting method to obtain the polyline of the roadside, and to fit the shape of the roadside based on the polyline of the roadside.

[0070] Optionally, in some embodiments, the input module 100 further includes: a processing unit, configured to process point cloud data within the point cloud ROI region based on a point cloud feature extraction structure network to obtain 3D sparse convolution results; an extraction unit, configured to obtain sparse features based on the 3D sparse convolution results, project the sparse features onto the BEV plane based on a 2D backbone network, and extract features on the 2D plane through a Decoder network to obtain target point cloud features; and a generation unit, configured to input the target point cloud features into a Head network to obtain point cloud information of roadside key points.

[0071] Optionally, in some embodiments, the processing unit is specifically used to: perform gridding processing on the point cloud data within the point cloud ROI region based on a preset voxel grid to obtain multiple voxel grids; and downsample or fill each voxel grid according to the maximum number of point clouds in each voxel grid to obtain a 3D sparse convolution result.

[0072] Optionally, in some embodiments, the Head network includes a classification loss function and a regression loss function. The target point cloud features are input into the Head network to obtain the point cloud information of the roadside key points, including: classifying the target point cloud features according to the classification loss function to obtain the point cloud classification result; regressing the target point cloud features according to the regression loss function to obtain the position of the roadside; and obtaining the point cloud information of the roadside key points according to the point cloud classification result and the position of the roadside.

[0073] Optionally, in some embodiments, the 2D backbone network is bevbackbone.

[0074] It should be noted that the foregoing explanation of the curb detection method embodiment also applies to the curb detection device of this embodiment, and will not be repeated here.

[0075] The curb detection device proposed in this application acquires raw point cloud data, processes the raw point cloud data to obtain point cloud ROI regions, performs voxelized mesh point cloud feature extraction to obtain 3D sparse convolution results, projects the sparse features onto the BEV (Bridge Elevation Vehicle), extracts point cloud features through a 2D backbone, and detects key points of the curb using a decoder. The decoder-processed point cloud features are used as input to the detection head and output as key points. Post-processing is then performed to output a smoothed curve. This solves the problems of susceptibility to interference, poor transferability, and unsatisfactory performance in related technologies, improving the performance of curb point cloud detection, enhancing the transferability and reusability of curb detection, reducing computational power requirements while meeting engineering deployment needs, and satisfying real-time detection requirements.

[0076] Figure 8 A schematic diagram of the structure of a vehicle provided in an embodiment of this application. The vehicle may include:

[0077] The memory 801, the processor 802, and the computer program stored on the memory 801 and capable of running on the processor 802.

[0078] When the processor 802 executes the program, it implements the curb detection method provided in the above embodiments.

[0079] Furthermore, the vehicle also includes:

[0080] Communication interface 803 is used for communication between memory 801 and processor 802.

[0081] The memory 801 is used to store computer programs that can run on the processor 802.

[0082] The memory 801 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0083] If the memory 801, processor 802, and communication interface 803 are implemented independently, then the communication interface 803, memory 801, and processor 802 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0084] Optionally, in a specific implementation, if the memory 801, processor 802, and communication interface 803 are integrated on a single chip, then the memory 801, processor 802, and communication interface 803 can communicate with each other through an internal interface.

[0085] The processor 802 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.

[0086] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described curb detection method.

[0087] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0088] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0089] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0090] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0091] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0092] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A curb detection method, characterized in that, Includes the following steps: The raw point cloud data is acquired, and the raw point cloud data is filtered and divided into regions of interest (ROIs) to obtain point cloud ROI regions. Point cloud data within the ROI region is input into a pre-defined road edge detection network model to obtain point cloud information of key road edge points. The pre-defined road edge detection network model consists of a point cloud feature extraction structure network, a 2D backbone network, a decoder network, and a head network. Based on a preset clustering and polar coordinate system sorting method, the point cloud information of the key points of the roadside is processed to obtain the polyline of the roadside, and the shape of the roadside is obtained by fitting the polyline of the roadside. The step of inputting the point cloud data within the point cloud ROI region into a preset roadside detection network model to obtain point cloud information of key roadside points includes: Based on the point cloud feature extraction structure network, the point cloud data within the point cloud ROI region is processed to obtain 3D sparse convolution results. Sparse features are obtained based on the 3D sparse convolution results, and the sparse features are projected onto the BEV plane based on the 2D backbone network. The Decoder network is then used to extract features on the 2D plane to obtain the target point cloud features. The target point cloud features are input into the Head network to obtain the point cloud information of the roadside key points.

2. The method according to claim 1, characterized in that, The point cloud data within the ROI region is processed by the point cloud feature extraction structure network to obtain 3D sparse convolution results, including: Based on a preset voxel grid, the point cloud data within the point cloud ROI region is processed into a grid to obtain multiple voxel grids. The 3D sparse convolution result is obtained by downsampling or filling each voxel based on the maximum number of point clouds in each voxel.

3. The method according to claim 2, characterized in that, The Head network includes a classification loss function and a regression loss function. The step of inputting the target point cloud features into the Head network to obtain the point cloud information of the roadside key points includes: The target point cloud features are classified according to the classification loss function to obtain the point cloud classification result. The location of the roadside is obtained by performing regression processing on the target point cloud features according to the regression loss function; The point cloud information of the key points of the roadside is obtained based on the point cloud classification results and the location of the roadside.

4. The method according to any one of claims 1-3, characterized in that, The 2D backbone network is bevbackbone.

5. A curb detection device, characterized in that, include: The acquisition module is used to acquire raw point cloud data, and to perform filtering processing and region of interest (ROI) division on the raw point cloud data to obtain point cloud ROI regions. The input module is used to input point cloud data within the point cloud ROI region into a preset road edge detection network model to obtain point cloud information of key road edge points. The preset road edge detection network model consists of a point cloud feature extraction structure network, a 2D backbone network, a decoder network, and a head network. The processing module is used to process the point cloud information of the key points of the roadside based on a preset clustering and polar coordinate system sorting method to obtain the polyline of the roadside, and to fit the shape of the roadside according to the polyline of the roadside. The input module further includes: The processing unit is used to process the point cloud data within the point cloud ROI region based on the point cloud feature extraction structure network to obtain 3D sparse convolution results. The extraction unit is used to obtain sparse features based on the 3D sparse convolution result, project the sparse features onto the BEV plane based on the 2D backbone network, and extract features on the 2D plane through the Decoder network to obtain target point cloud features. The generation unit is used to input the target point cloud features into the Head network to obtain the point cloud information of the roadside key points.

6. The apparatus according to claim 5, characterized in that, The processing unit is specifically used for: Based on a preset voxel grid, the point cloud data within the point cloud ROI region is processed into a grid to obtain multiple voxel grids. The 3D sparse convolution result is obtained by downsampling or filling each voxel based on the maximum number of point clouds in each voxel.

7. A vehicle, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the curb detection method as described in any one of claims 1-4.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the curb detection method as described in any one of claims 1-4.

Images