Vehicle driving area identification method based on road end laser radar

Abstract

The invention discloses a vehicle driving area identification method based on a road end laser radar, and relates to the technical field of traffic, and the method comprises the steps: carrying out the point cloud matching of a plurality of frames of original laser point cloud images of a traffic scene obtained by the road end laser radar; overlapped point clouds in multiple frames of original laser point cloud pictures are screened out to obtain a scene point cloud picture so as to construct a fixed scene in a traffic scene, and then the scene point cloud picture is rasterized and superposed in a front view direction, and then a scene binary image is extracted; and performing linear fitting on the effective grids to determine a vehicle driving area in the traffic scene. The method provides a feasible solution for realizing automatic and accurate division of a vehicle driving area, and has remarkable advantages in road end laser radar application.

Description

Technical Field

[0001] The present application relates to the field of traffic technology, and in particular to a vehicle driving area recognition method based on road-side laser radar. Background Art

[0002] With the development of intelligent transportation, various road-side monitoring devices installed on street light poles and traffic signal poles are gradually being used. Common road-side monitoring devices include various video acquisition devices and radars, etc. These road-side monitoring devices have a higher installation position and a wider field of view, and can cover a wider monitoring area, thereby providing global information collection and analysis for the entire transportation system. It can be used to assist in traffic flow detection, illegal capture and traffic accident detection, etc., which is particularly important for intelligent transportation management.

[0003] However, due to the wide coverage of road-side monitoring equipment, the detection area of ​​road-side monitoring equipment often covers other background areas in addition to the vehicle driving area. This results in the data perceived by the road-side monitoring equipment to include various data in the background area in addition to the vehicle driving data in the vehicle driving area. The various data in the background area will interfere with the road-side perception, thereby affecting the accuracy and reliability of the road-side perception, and will also bring more redundant data, increasing the data processing load. If the vehicle driving area in the traffic scene can be determined and the data in the vehicle driving area can be processed in a targeted manner, it will be of great help in improving the road-side perception effect. However, the current practice is mainly to perform regional calibration after manual installation and adjustment, which is time-consuming, labor-intensive and inefficient. Summary of the invention

[0004] In response to the above problems and technical requirements, this application proposes a vehicle driving area recognition method based on road-side laser radar. The technical solution of this application is as follows:

[0005] A vehicle driving area recognition method based on road-side laser radar, the vehicle driving area recognition method comprising:

[0006] Use road-side laser radar to obtain multiple frames of original laser point cloud images of traffic scenes;

[0007] Perform point cloud matching on multiple frames of original laser point cloud images, and filter out overlapping point clouds in multiple frames of original laser point cloud images to obtain a scene point cloud image;

[0008] The scene point cloud image is rasterized and superimposed in the front view direction to obtain a scene binary image. The scene binary image includes multiple grids arranged in rows and columns. Each grid is a valid grid or an invalid grid. The valid grid is the grid where the scene point cloud is located.

[0009] The vehicle driving area in the traffic scene is determined by linear fitting of the valid grids in the scene binary image.

[0010] A further technical solution is to perform linear fitting on the effective grids in the scene binary image to determine the vehicle driving area in the traffic scene, including:

[0011] Extracting a valid scene image consisting of multiple grid rows between the ground and the highest point of the scene from the scene binary image;

[0012] Linear fitting is performed on the valid grids in the valid scene image, and the area that reaches the flatness threshold is determined as the vehicle driving area in the traffic scene.

[0013] A further technical solution is that determining an area reaching a flatness threshold as a vehicle driving area in a traffic scene includes:

[0014] A sliding window is used to perform piecewise linear fitting on the effective scene image. The window width of the sliding window is smaller than a predetermined width threshold and smaller than the image of the effective scene image. Each time, linear fitting is performed on multiple grid columns of the effective scene image located in the sliding window, and the linear fitting results in multiple sliding windows are combined to obtain the linear fitting segment in the effective scene image to obtain the vehicle driving area in the traffic scene.

[0015] A further technical solution is that extracting a valid scene image includes:

[0016] Detect the number of valid grids contained in each grid row in the scene binary image;

[0017] The first grid row from top to bottom in the scene binary image containing valid grids reaching the number threshold is determined to be the grid row where the highest point of the scene is located, and the first grid row from bottom to top in the scene binary image containing valid grids reaching the number threshold is determined to be the grid row where the ground is located, and the valid scene image is extracted.

[0018] A further technical solution is that obtaining a scene binary image includes:

[0019] The scene point cloud image is rasterized and superimposed in the front view direction to obtain a scene point cloud heat map in the front view direction, wherein the scene point cloud heat map includes a plurality of grids arranged in rows and columns, and each grid contains a plurality of scene point clouds;

[0020] The scene point cloud heat map is binarized, and the grids whose scene point clouds reach the point cloud quantity threshold in the scene point cloud heat map are set as valid grids, and the grids whose scene point clouds do not reach the point cloud quantity threshold are set as invalid grids to obtain a scene binary image.

[0021] A further technical solution is to perform point cloud matching on multiple frames of original laser point cloud images, including:

[0022] The first frame of the original laser point cloud image is initialized as the scene point cloud image. Starting from the first frame of the original laser point cloud image, one frame of the original laser point cloud image is selected as the update point cloud frame every K frames of the original laser point cloud image. For any source point cloud p in the scene point cloud image, when the distance between the source point cloud p and its corresponding candidate point cloud q in the update point cloud frame is less than the distance threshold, it is determined that the candidate point cloud q and the source point cloud p are a set of point cloud pairs with a matching relationship, otherwise it is determined that the candidate point cloud q and the source point cloud p do not have a matching relationship, and all point cloud pairs with a matching relationship in the update point cloud frame and the scene point cloud image are retained, and other point cloud pairs are filtered out to update the scene point cloud image.

[0023] A further technical solution is that determining the candidate point cloud q corresponding to the source point cloud p in the scene point cloud image in the update point cloud frame includes:

[0024] Determine the normal line of the source point cloud p in the scene point cloud image along the direction of the normal vector;

[0025] The point cloud with the closest normal distance to the source point cloud p in the update point cloud frame is determined as the candidate point cloud q corresponding to the source point cloud p.

[0026] A further technical solution is that determining the normal of the source point cloud p in the scene point cloud image along the direction of the normal vector includes:

[0027] Calculate the covariance matrix of the positions of all point clouds in the neighborhood of the source point cloud p in the scene point cloud map, calculate the eigenvector of the covariance matrix, and take the eigenvector corresponding to the minimum eigenvalue as the normal vector of the source point cloud p.

[0028] The beneficial technical effects of this application are:

[0029] The present application discloses a vehicle driving area recognition method based on road-side laser radar. The method utilizes a point cloud registration algorithm to effectively integrate data collected by the road-side laser radar, can more accurately construct fixed scenes in traffic scenes, and uses a linear fitting method to automatically locate the vehicle driving area. The method has a high degree of automation and provides a feasible solution for realizing automatic and accurate division of vehicle driving areas. The method has significant advantages in the application of road-side laser radar.

[0030] In addition, this method uses piecewise linear fitting to better adapt to complex road geometry, making the division of vehicle driving areas more robust and reliable. This method has high computational efficiency, can be effectively applied in application scenarios with high real-time requirements, and has good vehicle driving area division accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a method flow chart of a vehicle driving area identification method in one embodiment of the present application.

[0032] Figure 2 It is a frame of original laser point cloud image obtained in an instance.

[0033] Figure 3 It is a frame of original laser point cloud image obtained in another instance.

[0034] Figure 4 It is a schematic diagram of updating the scene point cloud map.

[0035] Figure 5 yes Figure 2 A schematic diagram of the vehicle driving area is extracted in the example. DETAILED DESCRIPTION

[0036] The specific implementation of the present application is further described below in conjunction with the accompanying drawings.

[0037] This application discloses a vehicle driving area recognition method based on road-side laser radar, please refer to Figure 1 As shown in the flowchart, the vehicle driving area recognition method includes the following steps:

[0038] Step 1: Use road-side laser radar to obtain multiple frames of original laser point cloud images of the traffic scene.

[0039] Roadside LiDARs are installed on various roadside devices such as streetlight poles and traffic signal poles. They face the traffic scene, and their detection area includes the vehicle driving area and background area in the traffic scene. Roadside LiDARs continuously acquire raw laser point cloud images frame by frame.

[0040] The actual installation position and shooting angle of road-side laser radar are varied. In an example scenario, the road-side laser radar is installed at an intersection parallel to the road surface, and the shooting direction of the road-side laser radar is along the lane direction in the vehicle driving area. Then the original laser point cloud image obtained by the road-side laser radar is as follows: Figure 2 In another example scenario, the roadside laser radar is installed on a street light pole and obliquely looks down at the vehicle driving area. The original laser point cloud image obtained by the roadside laser radar is as follows: Figure 3 shown.

[0041] Step 2: perform point cloud matching on multiple frames of original laser point cloud images, and filter out overlapping point clouds in multiple frames of original laser point cloud images to obtain a scene point cloud image.

[0042] Elements inherent in actual traffic scenes, such as lanes, green belts, buildings, etc., are stationary and relatively stable, while elements entering the traffic scene, such as vehicles and pedestrians, are in motion. Therefore, this application uses the overlap matching of multiple frames of original laser point cloud images to extract the point cloud information of these fixed elements in the traffic scene.

[0043] Considering that in actual traffic scenes, moving elements such as vehicles and pedestrians may also be stationary, it is difficult to accurately extract the scene point cloud map by only comparing the overlapping parts of two frames of original laser point cloud maps, which is highly random. Therefore, in one embodiment, a point cloud matching operation is performed every K frames of original laser point cloud maps, where K can be customized, such as K=20.

[0044] The method for extracting scene point cloud images while the road-side laser radar continuously acquires the original laser point cloud images includes the following steps, please refer to Figure 4 The flowchart shown:

[0045] Step 410: Initialize the first frame of original laser point cloud image as a scene point cloud image.

[0046] Step 420, starting from the first frame of the original laser point cloud image, one frame of the original laser point cloud image is selected as the updated point cloud frame every K frames of the original laser point cloud image. For example, the K+1th frame is selected as the updated point cloud frame for the first time, and the 2K+1th frame is selected as the updated point cloud frame again, and so on.

[0047] In the process of the road-side laser radar continuously acquiring the original laser point cloud image, when it is determined that the interval between the currently acquired original laser point cloud image and the last updated scene point cloud image has not reached K frames, the current scene point cloud image is kept unchanged. When it is determined that the interval between the currently acquired original laser point cloud image and the last updated scene point cloud image has reached K frames, the currently acquired original laser point cloud image is used as the updated point cloud frame, and subsequent scene point cloud image update operations are performed.

[0048] Step 430: retain all point cloud pairs that have a matching relationship between the current scene point cloud image and the updated point cloud frame, and filter out other source point clouds in the current scene point cloud image to obtain an updated scene point cloud image. The method for filtering out point cloud pairs that have a matching relationship between the current scene point cloud image and the updated point cloud frame includes:

[0049] (1) Traverse each source point cloud in the current scene point cloud image in turn, and for any source point cloud p in the current scene point cloud image traversed, determine the candidate point cloud q corresponding to the source point cloud p in the update point cloud frame. Specifically:

[0050] Determine the normal line of the source point cloud p in the current scene point cloud image along the normal vector direction. The normal line is a ray starting from the source point cloud p and along the normal vector of the source point cloud p. This step requires determining the normal vector of the source point cloud p. The normal vector is a vector perpendicular to the surface of the object. In one embodiment, a normal vector calculation formula based on the least squares method is used, including:

[0051] Calculate the covariance matrix of the positions of all point clouds in the scene point cloud graph that are within the neighborhood of the source point cloud p N p It is a set consisting of the neighborhood of the source point cloud p, and its neighborhood can be formed by the k nearest neighbor points of the source point cloud p. is the average value of all point clouds in the neighborhood of the source point cloud p, |N p | is the number of point clouds contained in the neighborhood of the source point cloud p, p j is any point cloud in the neighborhood of the source point cloud p.

[0052] Then the eigenvector of the above covariance matrix is ​​calculated, and the eigenvector corresponding to the minimum eigenvalue is taken as the normal vector of the source point cloud P. Once the normal vector of the source point cloud P is calculated, a ray can be extended along the direction of the normal vector of the source point cloud P as the normal line of the source point cloud P along the direction of the normal vector.

[0053] Then determine the point cloud in the updated point cloud frame that is closest to the normal of the source point cloud p as the candidate point cloud q corresponding to the source point cloud p. After determining the normal of the source point cloud p, the parametric equation can be used to represent the normal. Then, the distance between the point cloud in the updated point cloud frame and the normal can be determined through the distance formula between the point and the line.

[0054] (2) Detect the Euclidean distance between the source point cloud p and its corresponding candidate point cloud q.

[0055] (3) When the distance between the source point cloud p and the candidate point cloud q corresponding to it in the updated point cloud frame is less than the distance threshold, the candidate point cloud q and the source point cloud p are determined to be a set of point cloud pairs with a matching relationship, otherwise it is determined that the candidate point cloud q and the source point cloud p do not have a matching relationship. In other words, the point cloud pairs with a matching relationship must also meet the restriction of the distance threshold. The purpose of this constraint is to improve the accuracy of the matching process and prevent the influence of mismatching or interference points.

[0056] Each source point cloud in the current scene point cloud can be matched with the point cloud in the update point cloud frame by the above method. For any source point cloud p in the scene point cloud: when it is determined that the update point cloud frame contains a candidate point cloud q that matches the source point cloud p, the source point cloud p is retained and the candidate point cloud q that matches it is also added to the scene point cloud. When it is determined that the update point cloud frame does not contain a candidate point cloud q that matches the source point cloud p, the source point cloud p is deleted. After completing the above operations, the current scene point cloud can be updated.

[0057] Step 3: rasterize the scene point cloud image and extract the scene binary image after superimposing it in the front view direction.

[0058] The extracted scene binary image includes multiple grids arranged in rows and columns, each grid is a valid grid or an invalid grid. The valid grid is the grid where the scene point cloud is located, including:

[0059] (1) First, the scene point cloud map is rasterized and superimposed in the front view direction to obtain the scene point cloud heat map in the front view direction.

[0060] The superposition in the front view direction is to project the scene point cloud image in the front view direction, and the obtained scene point cloud heat map includes a plurality of grids arranged in rows and columns, and each grid contains a plurality of scene point clouds whose projection positions are within the grid.

[0061] (2) Binarization is performed on the scene point cloud heat map, and the grids whose scene point clouds in the scene point cloud heat map reach the point cloud quantity threshold are set as valid grids, and the grids whose scene point clouds do not reach the point cloud quantity threshold are set as invalid grids, thereby obtaining a scene binary image.

[0062] Step 4: Perform linear fitting on valid grids in the scene binary image to determine the vehicle driving area in the traffic scene.

[0063] When fitting, a valid scene image consisting of multiple grid rows between the ground and the highest point of the scene is first extracted from the scene binary image to exclude areas such as the sky and reduce the amount of subsequent data processing, including: detecting the number of valid grids contained in each grid row in the scene binary image, and then determining that the first grid row from top to bottom in the scene binary image whose number of valid grids reaches a number threshold is the grid row where the highest point of the scene is located, determining that the first grid row from bottom to top in the scene binary image whose number of valid grids reaches a number threshold is the grid row where the ground is located, and extracting the valid scene image.

[0064] Since the vehicle driving area is often a flat area in the traffic scene, the effective grid in the effective scene image is linearly fitted to determine the area that reaches the flatness threshold as the vehicle driving area in the traffic scene. Linear fitting is a simple and intuitive fitting method, which assumes that there is a linear relationship between data points. The goal of linear fitting is to find a straight line that minimizes the sum of the distances from all effective grids to the straight line. Generally, the least squares method is used to achieve linear fitting, and the goal of the least squares method is to minimize the residual between the actual data points and the fitted straight line. In order to improve the fitting accuracy and improve the adaptability in different traffic scenes, in one embodiment, a sliding window is used to perform piecewise linear fitting on the effective scene image, and the window width of the sliding window is less than the predetermined width threshold and less than the image of the effective scene image. Each time, a plurality of grid columns of the effective scene image located in the sliding window are linearly fitted, and the linear fitting results in the plurality of sliding windows are combined to obtain the linear fitting segment in the effective scene image as the vehicle driving area in the traffic scene. The piecewise linear fitting divides the given data into multiple segments, performs linear fitting on each segment, and then calculates the fitting error and selects the appropriate linear segment according to the set threshold. Compared with the global linear fitting method, it has better fitting effect and adaptability.

[0065] For example, Figure 2 In the example of Figure 5 As shown in (a) of , the scene binary image obtained by binarization is as follows Figure 5 As shown in (b), Figure 5 In (b), white indicates valid grids and black indicates invalid grids. Then the valid scene image is further extracted as shown in Figure 5 As shown in the red dashed box in (c), the vehicle driving area determined by the final fitting is as follows Figure 5 In this example, the shooting direction of the roadside laser radar is along the lane direction in the vehicle driving area, so the extracted vehicle driving area is basically along the grid row direction, but in fact, in Figure 3 In other shooting angle examples, the vehicle driving area can also be extracted.

[0066] After determining the vehicle driving area in the traffic scene, the point cloud data in the original laser point cloud map located in the vehicle driving area can be targeted for subsequent road section perception processing without processing the point cloud in other areas, which is conducive to improving the efficiency of road-side perception.

[0067] The above is only a preferred embodiment of the present application, and the present application is not limited to the above embodiments. It is understood that other improvements and changes directly derived or associated by those skilled in the art without departing from the spirit and concept of the present application should be considered to be included in the protection scope of the present application.

Claims

1. A vehicle driving area recognition method based on roadside laser radar, characterized in that: The vehicle driving area recognition method comprises: Use road-side laser radar to obtain multiple frames of original laser point cloud images of traffic scenes; Perform point cloud matching on multiple frames of original laser point cloud images, and filter out overlapping point clouds in multiple frames of original laser point cloud images to obtain a scene point cloud image; The scene point cloud image is rasterized and superimposed in the front view direction to obtain a scene binary image, wherein the scene binary image includes a plurality of grids arranged in rows and columns, each grid is a valid grid or an invalid grid, and a valid grid is a grid where the scene point cloud is located; Linear fitting is performed on valid grids in the scene binary image to determine a vehicle driving area in the traffic scene.

2. The vehicle driving area recognition method according to claim 1, characterized in that: Performing linear fitting on valid grids in the scene binary image to determine a vehicle driving area in the traffic scene includes: Extracting a valid scene image consisting of a plurality of grid rows between the ground and the highest point of the scene from the scene binary image; Linear fitting is performed on the effective grids in the effective scene image to determine an area that reaches a flatness threshold as a vehicle driving area in the traffic scene.

3. The vehicle driving area recognition method according to claim 2, characterized in that: The area that reaches the flatness threshold is determined as the vehicle driving area in the traffic scene, including: The effective scene image is piecewise linearly fitted using a sliding window, wherein the window width of the sliding window is smaller than a predetermined width threshold and smaller than the image of the effective scene image, and linear fitting is performed on multiple grid columns of the effective scene image located within the sliding window each time, and the linear fitting results within the multiple sliding windows are combined to obtain the linear fitting segments within the effective scene image to obtain the vehicle driving area in the traffic scene.

4. The vehicle driving area recognition method according to claim 2, characterized in that: The extracted valid scene images include: Detecting the number of valid grids contained in each grid row in the scene binary image; Determine that the first grid row from top to bottom in the scene binary image whose number of valid grids reaches a threshold is the grid row where the highest point of the scene is located, determine that the first grid row from bottom to top in the scene binary image whose number of valid grids reaches a threshold is the grid row where the ground is located, and extract the valid scene image.

5. The vehicle driving area recognition method according to claim 1, characterized in that: Obtaining a scene binary image includes: The scene point cloud image is rasterized and superimposed in the front view direction to obtain a scene point cloud heat map in the front view direction, wherein the scene point cloud heat map includes a plurality of grids arranged in rows and columns, and each grid contains a plurality of scene point clouds; The scene point cloud heat map is binarized, and grids whose scene point clouds reach a point cloud quantity threshold in the scene point cloud heat map are set as valid grids, and grids whose scene point clouds do not reach the point cloud quantity threshold are set as invalid grids, so as to obtain the scene binarized image.

6. The vehicle driving area recognition method according to claim 1, characterized in that: The point cloud matching of multiple frames of original laser point cloud images includes: The first frame of the original laser point cloud image is initialized as the scene point cloud image. Starting from the first frame of the original laser point cloud image, one frame of the original laser point cloud image is selected as the update point cloud frame every K frames of the original laser point cloud image. For any source point cloud p in the scene point cloud image, when the distance between the source point cloud p and its corresponding candidate point cloud q in the update point cloud frame is less than the distance threshold, it is determined that the candidate point cloud q and the source point cloud p are a set of point cloud pairs with a matching relationship, otherwise it is determined that the candidate point cloud q and the source point cloud p do not have a matching relationship, and all point cloud pairs with a matching relationship in the update point cloud frame and the scene point cloud image are retained, and other point cloud pairs are filtered out to update the scene point cloud image.

7. The vehicle driving area recognition method according to claim 6, characterized in that: Determining the candidate point cloud q in the updated point cloud frame corresponding to the source point cloud p in the scene point cloud image includes: Determine the normal line of the source point cloud p in the scene point cloud image along the direction of the normal vector; The point cloud with the closest normal distance to the source point cloud p in the update point cloud frame is determined as the candidate point cloud q corresponding to the source point cloud p.

8. The vehicle driving area recognition method according to claim 7, characterized in that: Determining the normal of the source point cloud p in the scene point cloud image along the direction of the normal vector includes: Calculate the covariance matrix of the positions of all point clouds in the neighborhood of the source point cloud p in the scene point cloud map, calculate the eigenvector of the covariance matrix, and take the eigenvector corresponding to the minimum eigenvalue as the normal vector of the source point cloud p.

Images