A road boundary classification method and device, computer equipment, and mobile tool

Abstract

The present invention discloses a road boundary classification method, comprising: step S1, collecting point cloud features of the road boundary near a vehicle and an image of the road boundary near the vehicle; step S2, classifying the road boundary near the vehicle based on the point cloud features of the road boundary near the vehicle to obtain a point cloud road boundary type; step S3, classifying the road boundary near the vehicle based on the image features of the image of the road boundary near the vehicle to obtain an image road boundary type; step S4, synchronously recording the single-frame point cloud features and single-frame image at the same moment or timestamp as the same frame point cloud and image; step S5, obtaining a final road boundary type based on the point cloud features in the same frame point cloud and image and the road boundary classification results of the image. The present invention also discloses a road boundary classification device, computer equipment, and mobile tool. The present invention classifies and fuses road boundaries using lidar point clouds and camera images to achieve automatic road boundary classification.

Description

Technical Field

[0001] The present invention relates to the field of unmanned driving technology, in particular to a road boundary classification method, a road boundary classification device, a computer device, and a mobile tool. Background Art

[0002] Map collection for autonomous driving often requires classifying road boundary types to determine whether a vehicle can pass or traverse them. Given the enormous workload associated with map collection and the abundance of road boundary information, being able to quickly and accurately distinguish road boundary types can significantly improve map collection efficiency and reduce costs.

[0003] At present, in the map collection work of unmanned driving, the distinction of road boundary types is mostly done by manual classification, that is, by observing the road boundary features in the image with the naked eye, manually distinguishing the road types and marking them accordingly in the map.

[0004] This manual classification method relies on visually observing road boundary features in images to manually distinguish and label boundary types in maps. This method is inefficient, requires a lot of manpower, and is slow.

[0005] In addition, when manually observing road boundary features to distinguish and label road types, there will inevitably be classification errors and mislabeling problems. Moreover, the road boundary scene features are complex, and the characteristics of various types of roads are vague. Different people also have different understandings of road types, which ultimately leads to a decrease in the accuracy of manually labeled road types, which will to some extent affect the safe and reliable operation of autonomous vehicles on the road. Summary of the Invention

[0006] The purpose of the present invention is to provide a road boundary classification method, a road boundary classification device, a computer device, and a mobile tool in order to address the technical defects of the prior art.

[0007] To this end, in a first aspect of the present invention, a road boundary classification method is provided, comprising the following steps:

[0008] Step S1, using a laser radar to collect point cloud features of the road boundary near the vehicle in real time, and using a camera to collect images of the road boundary near the vehicle in real time;

[0009] Step S2, classifying the road boundary near the vehicle based on the point cloud features of the road boundary near the vehicle obtained in step S1, and obtaining the corresponding point cloud road boundary type;

[0010] Step S3, for the image of the road boundary near the vehicle obtained in step S1, classify the road boundary near the vehicle according to the image features of the image to obtain the corresponding image road boundary type;

[0011] Step S4, synchronizing the single-frame point cloud features and the single-frame image at the same moment or timestamp together and recording them as the same frame point cloud and image;

[0012] In step S5, the point cloud and image of the same frame obtained in step S4 are fused according to the point cloud features in the point cloud and image of the same frame and the road boundary classification result of the image according to a preset fusion rule to obtain the final road boundary type.

[0013] Preferably, in step S1, the laser radar faces the front of the vehicle, and the laser radar collects point cloud features of the road boundary in front of the vehicle, and the camera faces the front of the vehicle, and the camera collects an image of the road boundary in front of the vehicle.

[0014] Preferably, in step S2, when the point cloud information of the road boundary near the vehicle obtained in step S1 has a point cloud feature that is a preset obvious high curb point cloud feature, the type of the road boundary near the vehicle is classified as a point cloud Class B road boundary;

[0015] Preset obvious high roadside point cloud features, that is, for the point cloud features of the road boundary near the vehicle collected by the lidar, which includes a roadside dividing line;

[0016] For the curb dividing line, the height difference of the point clouds on both sides of the curb dividing line is within the preset difference range;

[0017] The point cloud B class road boundary is the point cloud road boundary of the high curb class.

[0018] Preferably, in step S2, when the point cloud feature of the point cloud information of the road boundary near the vehicle obtained in step S1 is not a preset obvious high curb point cloud feature, the road boundary near the vehicle is classified as a point cloud AC mixed type road boundary;

[0019] Preset obvious high roadside point cloud features, that is, for the point cloud features of the road boundary near the vehicle collected by the lidar, which includes a roadside dividing line;

[0020] For the curb dividing line, the height difference of the point clouds on both sides of the curb dividing line is within the preset difference range;

[0021] The point cloud AC mixed-type road boundary includes two types of point cloud road boundaries: point cloud type A road boundaries and point cloud type C road boundaries.

[0022] Preferably, in step S3, the image road boundary type includes image A-type road boundary, image B-type road boundary and image C-type road boundary;

[0023] Among them, the road boundary of image class A is the road boundary of the image class of low bushes;

[0024] The road boundary of image class B is the road boundary of the image class of high curb;

[0025] The road boundary of image class C is the road boundary of the flat lawn class.

[0026] Preferably, in step S5, the specific preset fusion rules are as follows:

[0027] For the point cloud features and images in the same frame of point cloud and image, when the road boundary classification result of the point cloud feature is the point cloud class B road boundary, and the road boundary classification result of the image is any one of the image class A road boundary, image class B road boundary, and image class C road boundary, the final road boundary type obtained after fusion is: the final road boundary of class B, that is, the final road boundary of the high curb class;

[0028] When the road boundary classification result of the point cloud feature is the point cloud AC mixed class road boundary, and the road boundary classification result of the image is the image class A road boundary, the final road boundary type obtained after fusion is: Class A final road boundary, that is, the final road boundary of the low shrub class;

[0029] When the road boundary classification result of the point cloud feature is the point cloud AC mixed class road boundary, and the road boundary classification result of the image is the image class B road boundary, the final road boundary type obtained after fusion is: uncertain class road boundary;

[0030] When the road boundary classification result of the point cloud feature is the point cloud AC mixed class road boundary, and the road boundary classification result of the image is the image C class road boundary, the final road boundary type obtained after fusion is: Class C final road boundary, that is, the road boundary of the flat lawn class.

[0031] Preferably, in step S5, the following steps are further included:

[0032] When the final road boundary type obtained after fusion is an uncertain road boundary, the corresponding point cloud and image of the same frame are output for manual classification.

[0033] In a second aspect, the present invention further provides a road boundary classification device, comprising the following modules:

[0034] A laser radar is used to collect point cloud features of the road boundary near the vehicle in real time and then send them to the first road boundary classification module and the data synchronization module;

[0035] A camera, configured to collect images of road boundaries near the vehicle in real time and then send the images to the second road boundary classification module and the data synchronization module;

[0036] A first road boundary classification module is connected to the laser radar and is used to classify the road boundary near the vehicle based on the point cloud features of the road boundary near the vehicle sent by the laser radar, and obtain the corresponding point cloud road boundary type;

[0037] A second road boundary classification module is connected to the camera and is used to classify the road boundary near the vehicle according to the image features of the image sent by the camera to obtain the corresponding image road boundary type;

[0038] The data synchronization module is connected to the lidar and camera respectively. It is used to synchronize the single-frame point cloud information and single-frame image information sent by the lidar and camera at the same time or timestamp, record them as the same frame of point cloud and image, and then send the same frame of point cloud and image to the classification result fusion module;

[0039] The classification result fusion module is connected to the first road boundary classification module and the second road boundary classification module respectively, and is used to receive the same frame point cloud and image obtained by the data synchronization module, and fuse the point cloud features in the same frame point cloud and image and the road boundary classification results of the image according to the preset fusion rules to obtain the final road boundary type.

[0040] Preferably, the classification result fusion module is further configured to output the corresponding point cloud and image of the same frame for manual classification when the final road boundary type obtained after fusion is an uncertain road boundary.

[0041] According to a third aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the road boundary classification method according to the first aspect.

[0042] According to a fourth aspect of the present invention, a mobile tool is provided, comprising the computer device described in the third aspect.

[0043] As can be seen from the technical solutions provided above, compared to existing technologies, the present invention provides a road boundary classification method, a road boundary classification device, a computer device, and a mobile tool. This invention is scientifically designed, classifying road boundaries using both lidar point clouds and camera images, then fusing the classification results to automatically determine the road boundary type, thus enabling automated road boundary classification. This significantly reduces labor costs and significantly improves map collection efficiency, possessing significant practical significance.

[0044] The present invention can effectively avoid the mistakes and errors caused by manual distinction and annotation of road boundary types in maps, reduce the classification errors caused by the understanding deviations of different people in manual annotation, and greatly improve the accuracy of distinguishing and understanding road boundary types in maps. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The overall workflow diagram of a road boundary classification method provided by the present invention;

[0046] Figure 2a 、 Figure 2b They are the point cloud feature map and image collected by the lidar and camera respectively;

[0047] Figure 3 It is the point cloud feature map corresponding to the point cloud road boundary of the high curb class (i.e., the point cloud class B road boundary);

[0048] Figure 4a 、 Figure 4b They are the point cloud feature maps corresponding to the point cloud A-type road boundary (i.e., the point cloud road boundary of the low shrub class) and the point cloud C-type road boundary (i.e., the point cloud road boundary of the flat lawn class);

[0049] Figure 5a 、 Figure 5b and Figure 5c , which are images corresponding to three types of boundary scenes, namely, image A road boundary, image B road boundary, and image C road boundary in the example of the present invention. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and implementation methods.

[0051] It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present application, and are not intended to limit the scope of the present application in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0052] Those skilled in the art will appreciate that the embodiments of the present application may be implemented as a system, apparatus, device, method, or computer program product. Therefore, the present disclosure may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software.

[0053] For ease of understanding, the technical terms involved in this application are explained below:

[0054] The mobile tools referred to in this application may include but are not limited to vehicles with six autonomous driving technology levels, L0-L5, as established by the Society of Automotive Engineers International (SAE International) or the Chinese national standard "Automotive Driving Automation Classification".

[0055] In some possible implementations, the mobile tool may be a vehicle device or a robotic device having the following functions:

[0056] (1) Passenger-carrying function, such as family cars and buses;

[0057] (2) Cargo carrying function, such as ordinary trucks, box trucks, trailer trucks, closed trucks, tank trucks, flatbed trucks, container trucks, dump trucks, special structure trucks, etc.;

[0058] (3) Tool functions, such as logistics delivery vehicles, automated guided vehicles (AGVs), patrol cars, cranes, hoists, excavators, bulldozers, forklifts, rollers, loaders, off-road engineering vehicles, armored engineering vehicles, sewage treatment vehicles, sanitation vehicles, vacuum trucks, floor scrubbers, sprinkler trucks, sweeping robots, food delivery robots, shopping guide robots, lawn mowers, golf carts, etc.;

[0059] (4) Entertainment functions, such as entertainment vehicles, amusement park self-driving devices, balance vehicles, etc.;

[0060] (5) Special rescue functions, such as fire trucks, ambulances, power repair trucks, engineering rescue trucks, etc.

[0061] See also Figures 1 to 5c , the present invention provides a road boundary classification method, comprising the following steps:

[0062] Step S1, using a laser radar to collect point cloud features of the road boundary near the vehicle in real time, and using a camera to collect images of the road boundary near the vehicle in real time;

[0063] In step S1, it should be noted that the point cloud features of the laser radar contain three-dimensional coordinate information for each point, which is commonly known as the three coordinate elements of X, Y, and Z, and sometimes also contain color information, reflection intensity information, echo number information, etc.

[0064] In step S1, in a specific implementation, the laser radar preferably adopts a 16-line laser radar, and the present invention includes but is not limited to a 16-line laser radar.

[0065] In step S1, in a specific implementation, the camera is preferably a wide-angle high-definition camera, and the present invention includes but is not limited to wide-angle high-definition cameras.

[0066] In step S1, in a specific implementation, the laser radar is directed toward the front of the vehicle, and preferably collects point cloud features of the road boundary in front of the vehicle through the laser radar, and the camera is directed toward the front of the vehicle, and preferably collects an image of the road boundary in front of the vehicle through the camera. Figure 2a 、 Figure 2b As shown, Figure 2a 、 Figure 2b They are the point cloud features and images collected by the lidar and camera respectively.

[0067] In step S1, in a specific implementation, the camera is installed in a manner such that the camera faces forward. The present invention includes but is not limited to the manner in which the camera is installed facing forward.

[0068] Step S2, classifying the road boundary near the vehicle based on the point cloud features of the road boundary near the vehicle obtained in step S1, and obtaining the corresponding point cloud road boundary type;

[0069] In step S2, in a specific implementation, the road boundary classification scope takes the right front road boundary of the vehicle as an example. The present invention includes but is not limited to classifying the right front road boundary of the vehicle.

[0070] In step S2, in a specific implementation, when the point cloud feature of the road boundary near the vehicle obtained in step S1 is a preset obvious high curb point cloud feature, the type of the road boundary near the vehicle is classified as a point cloud Class B road boundary;

[0071] See also Figure 3 , preset obvious high roadside point cloud features, that is, for the point cloud features of the road boundary near the vehicle collected by the lidar, it includes a roadside dividing line (such as Figure 3 The line a shown is the point cloud boundary line);

[0072] For the point cloud features of the road boundary near the vehicle collected by the laser radar, the height difference of the point clouds on both sides of the curb dividing line a is within the preset difference range; the height difference of the point clouds on both sides of the curb dividing line a is preferably between 10cm and 40cm.

[0073] For example, the heights of the point clouds on one side of the line are all within a preset first height range, while the heights of the point clouds on the other side are all within a preset second height range; the maximum value of the preset first height range is less than the minimum value of the preset second height range; that is, the point clouds on one side of the point cloud dividing line are all at almost the same height, while the point clouds on the other side are almost all at another same height. In specific implementations, the difference between the maximum value of the second height range and the minimum value of the preset first height range (i.e., the height difference) is within a preset difference range (e.g., 10cm to 40cm, which is the vertical height of a typical curb).

[0074] The point cloud class B road boundary is a point cloud road boundary of the high curb class, which is an insurmountable boundary that can be identified by the lidar, such as a boundary with a clear high curb.

[0075] It should be noted that the point cloud attributes include characteristic information such as the position (x, y, z) and reflection intensity of each point in the class point cloud. The point cloud features obtained in the road boundary classification task are point cloud distribution features (i.e., point cloud height features). The present invention classifies the curb through point cloud distribution features (i.e., height features).

[0076] See also Figure 3 As shown in , for the point cloud B-type road boundary, the point cloud features are more obvious, and the B-type results can be output stably and accurately. The point cloud features corresponding to the point cloud B-type road boundary are as follows Figure 3 As shown, Figure 3 Taking the high roadside as an example, the point cloud features of the road boundary are obvious, so the corresponding road boundary type can be classified and output stably and accurately, that is, the classification output point cloud is Class B road boundary.

[0077] In step S2, in a specific implementation, when the point cloud feature of the road boundary near the vehicle obtained in step S1 is not a preset obvious high curb point cloud feature, the road boundary near the vehicle is classified as a point cloud AC mixed type road boundary;

[0078] It should be noted that there is no obvious high roadside point cloud feature, that is, the point cloud distribution is relatively scattered and messy, and does not have the above-mentioned Class B road boundary point cloud features.

[0079] Point cloud AC mixed road boundaries include both point cloud A road boundaries and point cloud C road boundaries. It is impossible to distinguish whether the road boundary is point cloud A road boundary or point cloud B road boundary.

[0080] Among them, the point cloud A road boundary is the point cloud road boundary of the low bush class, which is an insurmountable boundary and cannot be recognized by the lidar (such as the boundary of a low roadside or bush);

[0081] The point cloud C road boundary is a point cloud road boundary of the flat lawn type, which is a traversable boundary (there is no serious risk in traversing it).

[0082] It should be noted that for point cloud Class A road boundaries and point cloud Class C road boundaries, the point cloud features of these two types of road boundaries are relatively complex, and it is difficult to distinguish them by point cloud features. Therefore, for the present invention, point cloud Class A road boundaries and point cloud Class C road boundaries are temporarily classified into one category (point cloud AC category), that is, point cloud AC mixed category road boundaries.

[0083] For the present invention, the point cloud information corresponding to the road boundary of type A and the road boundary of type C are as follows: Figure 4a 、 Figure 4b As shown, Figure 4a It is the point cloud feature map corresponding to the road boundary of category A. In this specific example, low bushes are used as an example. Figure 4b is the point cloud feature map corresponding to the road boundary of point cloud type C. In this specific example, a flat lawn is used as an example. Figure 4a 、 Figure 4b It can be seen that the point cloud features of the road boundary of type A and the point cloud of type C are relatively similar. Therefore, the point cloud features of the road boundary near the vehicle cannot distinguish between the road boundaries of type A and type C.

[0084] It should be noted that, for the present invention, the road boundaries within a certain range in front of the vehicle can be classified through the point cloud features of the laser radar.

[0085] In the present invention, in step S2, the road boundaries can be divided into three categories based on the point cloud features of the road boundaries. Point cloud category A is the point cloud road boundaries of the high curb type, which are insurmountable and unrecognizable by LiDAR (such as low curbs or bushes). Category B is the point cloud road boundaries of the low bush type, which are insurmountable and recognizable by LiDAR (such as obvious high curbs). Category C is the point cloud road boundaries of the flat lawn type, which are surmountable boundaries (there is no serious risk of crossing). The LiDAR point cloud features are used to understand (i.e., classify) the road boundaries within a certain range in front of the vehicle.

[0086] It should be noted that, in a specific implementation, in step S2, dividing the road boundaries into three categories is only an example, and the present invention should include but is not limited to dividing the road boundaries into three categories.

[0087] Step S3, for the image of the road boundary near the vehicle obtained in step S1, classify the road boundary near the vehicle according to the image features of the image to obtain the corresponding image road boundary type;

[0088] In step S3, in a specific implementation, by adopting a deep learning method (specifically an existing deep learning classification algorithm), based on the image features of the image, the road boundaries in the image are feature extracted and identified and classified.

[0089] It should be noted that image features are pixel features reflected from an image. With respect to the road boundary classification of the present invention, image features specifically refer to road boundary features, such as image features of road boundaries such as high curbs, low curbs, and bushes.

[0090] In this invention, a deep learning classification algorithm is designed to classify road boundaries. Specifically, the following operations are involved:

[0091] First, multiple batches of images containing various road boundaries are collected in advance, and the various road boundaries in the images are manually classified and labeled;

[0092] Then, the image data with classification annotations is used to train a deep learning algorithm model (specifically, an existing deep learning algorithm model is used); the trained deep learning algorithm model is then capable of classifying roadside images;

[0093] Then, for any input image information (such as the image information of the road boundary near the vehicle obtained in step S1), the image road boundary type (i.e., the classification category of the curb) corresponding to the image can be obtained through calculation using the trained deep learning algorithm model.

[0094] It should be noted that, for the present invention, the classification method of camera image boundary types takes the deep learning classification algorithm as an example, and the present invention includes but is not limited to the deep learning classification method.

[0095] In step S3, in specific implementation, the image road boundary types include image A road boundaries, image B road boundaries, and image C road boundaries;

[0096] Among them, the road boundary of image class A is the road boundary of the image class of low bushes;

[0097] The road boundary of image class B is the road boundary of the image class of high curb;

[0098] The road boundary of image class C is the road boundary of the flat lawn class;

[0099] like Figure 5a 、 Figure 5b and Figure 5c The three images shown are images of three types of boundary scenes in the present invention. The road boundary classification results based on the image features of these three images are low shrubs (Class A), high curbs (Class B), and flat lawns (Class C). Based on existing deep learning classification algorithms, image features can be used to accurately and stably output the classification information of image road boundaries.

[0100] Step S4: Synchronize the single-frame point cloud information and the single-frame image information at the same moment or timestamp and record them as the same frame point cloud and image. This ensures that the same frame point cloud and image are collected at the same moment or timestamp and are used for fusion of road boundary classification.

[0101] In step S5, for the same frame point cloud and image obtained in step S4, according to the point cloud features in the same frame point cloud and image and the road boundary classification result of the image (i.e., the point cloud road boundary type corresponding to the point cloud features in the same frame point cloud and image obtained by executing step S2, and the image road boundary type corresponding to the image in the same frame point cloud and image obtained by executing step S3), a fusion is performed according to a preset fusion rule to obtain the final road boundary type.

[0102] It should be noted that, for the present invention, the point cloud and image synchronized in the same frame represent the road information collected at the same time. By fusing the classification results of the two, the final boundary classification result can be output.

[0103] In step S5, in specific implementation, the specific preset fusion rules are as follows:

[0104] For the point cloud features and images in the same frame of point cloud and image, when the road boundary classification result of the point cloud feature is the point cloud class B road boundary, and the road boundary classification result of the image is any one of the image class A road boundary, image class B road boundary, and image class C road boundary, the final road boundary type obtained after fusion is: the final road boundary of class B, that is, the final road boundary of the high curb class;

[0105] When the road boundary classification result of the point cloud feature is the point cloud AC mixed class road boundary, and the road boundary classification result of the image is the image class A road boundary, the final road boundary type obtained after fusion is: Class A final road boundary, that is, the final road boundary of the low shrub class;

[0106] When the road boundary classification result of the point cloud feature is the point cloud AC mixed class road boundary, and the road boundary classification result of the image is the image class B road boundary, the final road boundary type obtained after fusion is: uncertain class road boundary;

[0107] When the road boundary classification result of the point cloud feature is the point cloud AC mixed class road boundary, and the road boundary classification result of the image is the image C class road boundary, the final road boundary type obtained after fusion is: Class C final road boundary, that is, the road boundary of the flat lawn class.

[0108] It should be noted that, for the present invention, the above-described method for fusing point cloud features and image classification results is only an example, and the present invention should include but is not limited to the described method for fusing point cloud and image classification results.

[0109] In the present invention, in specific implementation, in step S5, the following steps may also be included:

[0110] When the final road boundary type obtained after fusion is an uncertain road boundary, the corresponding point cloud and image of the same frame are output for manual classification. That is, the point cloud and image of the same frame obtained in step S4 are output in a timely manner and personnel are arranged to perform manual classification to obtain the final road boundary type.

[0111] It should be noted that when the road boundary classification result of the point cloud features in the same frame point cloud and image is a point cloud AC mixed class road boundary, and the road boundary classification result of the image in the same frame point cloud and image is an image class B road boundary, the final road boundary type obtained after fusion is: uncertain class road boundary; at this time, manual classification is required to obtain the boundary category.

[0112] In the present invention, in specific implementation, after step S5, the following steps may be further included:

[0113] Step S6, finally outputting the road classification result: after manually classifying the uncertain road boundaries, the classification results are integrated to obtain the final road boundary classification result.

[0114] Based on the above technical solutions, it can be seen that for the present invention, the road boundary classification method of the present invention classifies the road boundaries through the point cloud features of the lidar, and then classifies the road boundaries through the image features in the image collected by the camera. The boundary types classified by the two are fused to finally obtain the final classification result of the road boundary.

[0115] In the present invention, in general, the processing steps include seven parts: acquisition of laser radar point cloud features and camera images, classification of road boundaries through laser radar point cloud features, classification of road boundaries through image features in camera images, data synchronization between the laser radar and the camera, fusion of the classification results of the two synchronized to the same frame and output of the boundary classification fusion result, manual classification of uncertain classifications of the fusion results, and finally output of the road classification result.

[0116] The present invention uses both point cloud features collected by LiDAR and image features collected by cameras to classify road boundaries separately, and then fuses the classification results to determine the final road boundary type. This method significantly reduces the labor cost of map acquisition, shortens map creation time, and improves map creation efficiency. It also avoids the errors and inaccuracies associated with manual classification and annotation, as well as the potential for human misunderstanding in manual annotation, significantly improving the accuracy of boundary understanding and annotation.

[0117] In addition, based on the road boundary classification method provided by the present invention, in order to implement the road boundary classification method, the present invention also provides a road boundary classification device, which includes the following modules:

[0118] A laser radar is used to collect point cloud features of the road boundary near the vehicle (for example, in front of the right side) in real time, and then send them to the first road boundary classification module and the data synchronization module;

[0119] A camera is used to collect images of the road boundary near the vehicle (e.g., right front) in real time and then send them to the second road boundary classification module and the data synchronization module;

[0120] A first road boundary classification module is connected to the laser radar and is used to classify the road boundary near the vehicle based on the point cloud features of the road boundary near the vehicle sent by the laser radar, and obtain the corresponding point cloud road boundary type;

[0121] A second road boundary classification module is connected to the camera and is used to classify the road boundary near the vehicle (e.g., right front) according to the image features of the image sent by the camera to obtain the corresponding image road boundary type;

[0122] The data synchronization module is connected to the lidar and camera respectively. It is used to synchronize the single-frame point cloud information and single-frame image information sent by the lidar and camera at the same time or time stamp, record them as the same frame of point cloud and image, and then send the same frame of point cloud and image to the classification result fusion module. This ensures that the same frame of point cloud and image is collected at the same time or time stamp for fusion of road boundary classification.

[0123] The classification result fusion module is connected to the first road boundary classification module and the second road boundary classification module respectively, and is used to receive the same frame point cloud and image obtained by the data synchronization module, and fuse the point cloud features in the same frame point cloud and image and the road boundary classification results of the image (that is, the point cloud road boundary type corresponding to the point cloud features in the same frame point cloud and image obtained by running the first road boundary classification module, and the image road boundary type corresponding to the image in the same frame point cloud and image obtained by running the second road boundary classification module) according to the preset fusion rules to obtain the final road boundary type.

[0124] In the present invention, in a specific implementation, for the classification result fusion module, the specific preset fusion rules are as follows:

[0125] For the point cloud features and images in the same frame of point cloud and image, when the road boundary classification result of the point cloud feature is the point cloud class B road boundary, and the road boundary classification result of the image is any one of the image class A road boundary, image class B road boundary, and image class C road boundary, the final road boundary type obtained after fusion is: the final road boundary of class B, that is, the final road boundary of the high curb class;

[0126] When the road boundary classification result of the point cloud feature is the point cloud AC mixed class road boundary, and the road boundary classification result of the image is the image class A road boundary, the final road boundary type obtained after fusion is: Class A final road boundary, that is, the final road boundary of the low shrub class;

[0127] When the road boundary classification result of the point cloud feature is the point cloud AC mixed class road boundary, and the road boundary classification result of the image is the image class B road boundary, the final road boundary type obtained after fusion is: uncertain class road boundary;

[0128] When the road boundary classification result of the point cloud feature is the point cloud AC mixed class road boundary, and the road boundary classification result of the image is the image C class road boundary, the final road boundary type obtained after fusion is: Class C final road boundary, that is, the road boundary of the flat lawn class.

[0129] In the present invention, in a specific implementation, the classification result fusion module is also used to output the corresponding same-frame point cloud and image for manual classification when the final road boundary type obtained after fusion is an uncertain road boundary. That is, the same-frame point cloud and image obtained by the data synchronization module are output in a timely manner so that personnel can be arranged to perform manual classification to obtain the final road boundary type.

[0130] It should be noted that when the road boundary classification result of the point cloud information is a point cloud AC mixed class road boundary, and the road boundary classification result of the image information is an image class B road boundary, the final road boundary type obtained after fusion is: uncertain class road boundary; at this time, manual classification is required to obtain the boundary category.

[0131] It should be noted that, in the present invention, after manually classifying the uncertain road boundaries, the classification results are integrated to obtain the final road boundary classification result.

[0132] In addition, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the road boundary classification method.

[0133] In addition, the present invention also provides a mobile tool, comprising the computer device.

[0134] In summary, compared to existing technologies, the present invention provides a road boundary classification method, a road boundary classification device, a computer device, and a mobile tool. This invention is scientifically designed, classifying road boundaries using both lidar point clouds and camera images, then fusing the two classification results to automatically determine the road boundary type, thus enabling automated road boundary classification. This significantly reduces labor costs and significantly improves map collection efficiency, possessing significant practical significance.

[0135] The present invention can effectively avoid the mistakes and errors caused by manual distinction and annotation of road boundary types in maps, reduce the classification errors caused by the understanding deviations of different people in manual annotation, and greatly improve the accuracy of distinguishing and understanding road boundary types in maps.

[0136] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A road boundary classification method, characterized in that: include: Step S1, using a laser radar to collect point cloud features of the road boundary near the vehicle in real time, and using a camera to collect images of the road boundary near the vehicle in real time; Step S2, classifying the road boundary near the vehicle based on the point cloud features of the road boundary near the vehicle obtained in step S1, and obtaining the corresponding point cloud road boundary type; Step S3, for the image of the road boundary near the vehicle obtained in step S1, classify the road boundary near the vehicle according to the image features of the image to obtain the corresponding image road boundary type; Step S4, synchronizing the single-frame point cloud features and the single-frame image at the same moment or timestamp together and recording them as the same frame point cloud and image; Step S5: For the same frame point cloud and image obtained in step S4, the point cloud features in the same frame point cloud and image and the road boundary classification result of the image are fused according to a preset fusion rule to obtain a final road boundary type; In step S5, the specific preset fusion rules are as follows: For the point cloud features and images in the same frame of point cloud and image, when the road boundary classification result of the point cloud feature is the point cloud class B road boundary, and the road boundary classification result of the image is any one of the image class A road boundary, image class B road boundary, and image class C road boundary, the final road boundary type obtained after fusion is: the final road boundary of class B, that is, the final road boundary of the high curb class; When the road boundary classification result of the point cloud feature is the point cloud AC mixed class road boundary, and the road boundary classification result of the image is the image class A road boundary, the final road boundary type obtained after fusion is: Class A final road boundary, that is, the final road boundary of the low shrub class; When the road boundary classification result of the point cloud feature is the point cloud AC mixed class road boundary, and the road boundary classification result of the image is the image class B road boundary, the final road boundary type obtained after fusion is: uncertain class road boundary; When the road boundary classification result of the point cloud feature is the point cloud AC mixed class road boundary, and the road boundary classification result of the image is the image C class road boundary, the final road boundary type obtained after fusion is: Class C final road boundary, that is, the road boundary of the flat lawn class.

2. The road boundary classification method according to claim 1, wherein: In step S1, the laser radar faces the front of the vehicle and collects point cloud features of the road boundary in front of the vehicle through the laser radar, and the camera faces the front of the vehicle and collects an image of the road boundary in front of the vehicle through the camera.

3. The road boundary classification method according to claim 1, wherein: In step S2, when the point cloud information of the road boundary near the vehicle obtained in step S1 has a point cloud feature that is a preset obvious high curb point cloud feature, the type of the road boundary near the vehicle is classified as a point cloud type B road boundary; Preset obvious high roadside point cloud features, that is, for the point cloud features of the road boundary near the vehicle collected by the lidar, which includes a roadside dividing line; For the curb dividing line, the height difference of the point clouds on both sides of the curb dividing line is within the preset difference range; The point cloud B class road boundary is the point cloud road boundary of the high curb class.

4. The road boundary classification method according to claim 1, wherein: In step S2, when the point cloud information of the road boundary near the vehicle obtained in step S1 has a point cloud feature that is not a preset obvious high curb point cloud feature, the road boundary near the vehicle is classified as a point cloud AC mixed type road boundary; Preset obvious high roadside point cloud features, that is, for the point cloud features of the road boundary near the vehicle collected by the lidar, which includes a roadside dividing line; For the curb dividing line, the height difference of the point clouds on both sides of the curb dividing line is within the preset difference range; The point cloud AC mixed-type road boundary includes two types of point cloud road boundaries: point cloud type A road boundaries and point cloud type C road boundaries.

5. The road boundary classification method according to claim 1, wherein: In step S3, the image road boundary type includes image A-type road boundary, image B-type road boundary and image C-type road boundary; Among them, the road boundary of image class A is the road boundary of the image class of low bushes; The road boundary of image class B is the road boundary of the image class of high curb; The road boundary of image class C is the road boundary of the flat lawn class.

6. The road boundary classification method according to claim 1, wherein: In step S5, the following steps are also included: When the final road boundary type obtained after fusion is an uncertain road boundary, the corresponding point cloud and image of the same frame are output for manual classification.

7. A road boundary classification device, characterized in that: Includes the following modules: A laser radar is used to collect point cloud features of the road boundary near the vehicle in real time and then send them to the first road boundary classification module and the data synchronization module; A camera, configured to collect images of road boundaries near the vehicle in real time and then send the images to the second road boundary classification module and the data synchronization module; A first road boundary classification module is connected to the laser radar and is used to classify the road boundary near the vehicle based on the point cloud features of the road boundary near the vehicle sent by the laser radar, and obtain the corresponding point cloud road boundary type; A second road boundary classification module is connected to the camera and is used to classify the road boundary near the vehicle according to the image features of the image sent by the camera to obtain the corresponding image road boundary type; The data synchronization module is connected to the lidar and camera respectively. It is used to synchronize the single-frame point cloud information and single-frame image information sent by the lidar and camera at the same time or timestamp, record them as the same frame of point cloud and image, and then send the same frame of point cloud and image to the classification result fusion module; a classification result fusion module, connected to the first road boundary classification module and the second road boundary classification module, respectively, for receiving the same frame point cloud and image obtained by the data synchronization module, and fusing the point cloud features in the same frame point cloud and image and the road boundary classification result of the image according to a preset fusion rule to obtain a final road boundary type; The preset fusion rules are as follows: For the point cloud features and images in the same frame of point cloud and image, when the road boundary classification result of the point cloud feature is the point cloud class B road boundary, and the road boundary classification result of the image is any one of the image class A road boundary, image class B road boundary, and image class C road boundary, the final road boundary type obtained after fusion is: the final road boundary of class B, that is, the final road boundary of the high curb class; When the road boundary classification result of the point cloud feature is the point cloud AC mixed class road boundary, and the road boundary classification result of the image is the image class A road boundary, the final road boundary type obtained after fusion is: Class A final road boundary, that is, the final road boundary of the low shrub class; When the road boundary classification result of the point cloud feature is the point cloud AC mixed class road boundary, and the road boundary classification result of the image is the image class B road boundary, the final road boundary type obtained after fusion is: uncertain class road boundary; When the road boundary classification result of the point cloud feature is the point cloud AC mixed class road boundary, and the road boundary classification result of the image is the image C class road boundary, the final road boundary type obtained after fusion is: Class C final road boundary, that is, the road boundary of the flat lawn class.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory, wherein: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A mobile tool, characterized in that: A computer device comprising the computer device of claim 8.

Images