Environmental perception system, method, vehicle and storage medium for intelligent driving vehicle
By combining lidar point cloud data, IMU information and high-precision maps, fitting the road boundary line, the problem of reduced perception ability of intelligent driving vehicles when they lose high-precision maps and RTK signals is solved, and accurate interest at intersections is achieved Area establishment and false detection have been reduced, improving user experience and perceived security performance.
Patent Information
- Application Number
- CN202111100769.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-18
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-09-18
AI Technical Summary
When intelligently driving vehicles lose high-precision maps and RTK signals, it is difficult to accurately establish areas of interest, and traffic participants in other lanes are easily filtered out at intersections, resulting in reduced perception and poor user experience.
By obtaining lidar point cloud data, IMU information and high-precision maps, and fitting the road boundary line with spline interpolation, we can determine whether the point cloud is in the lane line. If it is there, it will be retained, and if it is not there, it will be filtered out; when the RTK signal is lost, it will be lost; , use bicycle sensors to detect road isolation belts and curbs, perform point cloud filtering, create grid maps and perform target clustering.
When high-precision maps and RTK signals are lost, the area of interest can be established more accurately, the error detection is reduced, the adaptability and practicality of the perception system can be improved, and traffic participants in other lanes can be filtered out at intersections, thereby enhancing the perception of the system Security performance and user experience.
Smart Images

Figure CN113734176B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of environmental perception of intelligent driving vehicles, and specifically relates to an environmental perception system, method, vehicle and storage medium for intelligent driving vehicles. Background Art
[0002] Intelligent driving vehicles require the perception system to maintain stable detection in different environments and working conditions. It is generally composed of millimeter wave, ultrasonic, surround view, perimeter view and laser radar. Among them, the visual sensor as the main sensor perception system is greatly affected by light. The fusion of multiple laser radars for target detection can effectively increase the adaptability and robustness of the perception system, especially when the light is weak and the visual sensor's target detection ability is insufficient or fails. In addition, compared with a single radar system, the perception system combining multiple radars, RTK and high-precision maps has a smaller blind spot and a stronger perception of the surrounding environment, which can effectively enhance the system's perception safety performance and improve user experience. For example, the patent document CN111060923A discloses a method and system for detecting obstacles in automobile driving with multiple laser radars, which uses a combination of high-precision maps and laser radars, and requires the use of other sensors to determine the ROI area. There is no alternative after losing the high-precision map, and high-frequency scenes at intersections are not considered. Another example is a method based on multi-lidar data fusion disclosed in patent document CN111413684A. This method can effectively make full use of the data of multiple lidars, but it has the characteristics of complex conversion, time-consuming processing, lack of scalability and good practicality. In fact, for lidar, objects with obvious height outside the road often cause serious false detection and time-consuming, resulting in a decline in user experience.
[0003] Therefore, it is necessary to develop a new environment perception system, method, vehicle and storage medium for intelligent driving vehicles. Summary of the invention
[0004] The purpose of the present invention is to provide an environmental perception system, method, vehicle and storage medium for an intelligent driving vehicle, which can accurately establish an area of interest when a high-precision map and RTK signal are lost, and avoid filtering out traffic participants in other lanes at intersections.
[0005] In a first aspect, an environment perception method for an intelligent driving vehicle according to the present invention comprises the following steps:
[0006] Get the point cloud UDP package output by the laser radar, and connect it to the original point cloud driver module for processing to obtain the point cloud data format;
[0007] Obtain IMU information, combine it with the IMU information to find the conversion relationship between the radar position and the initial position of each point cloud, and convert all points in each frame to the initial radar position, that is, perform point cloud distortion compensation;
[0008] The distortion-compensated point cloud is combined with the radar calibration file to complete the conversion from the radar coordinate system to the vehicle coordinate system, and the point cloud in the vehicle coordinate system is obtained;
[0009] Obtain RTK positioning information and high-precision map information to determine whether there is an RTK signal;
[0010] If there is an RTK signal, it is determined whether the vehicle is at an intersection; if it is at an intersection, the chain facility information is fitted to the road boundary line through spline interpolation, and then combined with the vehicle positioning information given by RTK to determine whether the point cloud is within the lane line. If it is within the lane line, it is retained and a grid map is established, otherwise the point cloud is filtered out; if it is not at an intersection, it is retained and a grid map is established;
[0011] If there is no RTK signal, the road boundary line is fitted through spline interpolation based on the road isolation belt and road curb checked by the vehicle sensor to determine whether the point cloud is within the lane line. If it is within the lane line, it is retained and a grid map is established. Otherwise, the point cloud is filtered out.
[0012] After obtaining a complete grid map, objects are clustered and visualized according to the grid occupancy status.
[0013] Optionally, create a grid map, specifically:
[0014] Set the size and resolution of the grid map, and determine the grid area, i.e. the forward detection distance, the rearward detection distance, and the lateral detection distance, based on the vehicle's position in the grid;
[0015] Determine whether the laser radar point under the vehicle body coordinates is a noise point. If it is a noise point, it will not be put into the grid. If it is not a noise point, it will be put into the grid and determine whether it is a target grid. If it is a target grid, target clustering will be performed. If it is not a target grid, clustering will not be performed.
[0016] Optionally, for a nearby point cloud, a height difference between the highest point and the lowest point in the grid is calculated, and if the height difference is greater than a first preset threshold, it is determined to be a target grid, otherwise it is determined to be a non-target grid;
[0017] For the distant point cloud, it is determined whether the highest point in the grid is higher than the second preset threshold. If it is higher, it is determined to be the target grid, otherwise it is determined to be a non-target grid.
[0018] Optionally, the target clustering specific steps include:
[0019] The flooding method is used to cluster targets and determine whether the target grid meets the preset constraints. If not, the target is deleted. If it is satisfied, the target is output and visualized.
[0020] Optionally, each UDP packet consists of several blocks, and each block contains the data of all laser transmitters of the lidar at the same time.
[0021] Optionally, before connecting to the original point cloud driving module, load the corresponding parameter file of the laser radar for correction.
[0022] In a second aspect, an environment perception system for an intelligent driving vehicle according to the present invention comprises:
[0023] LiDARs installed on the top and left and right sides of the vehicle;
[0024] RTK equipment installed on the vehicle to output centimeter-level high-precision vehicle position information;
[0025] An IMU installed on the vehicle to detect the vehicle's inertial measurement information;
[0026] High-precision maps for outputting road environment information;
[0027] And a controller, which is respectively connected to each laser radar, RTK equipment, IMU and high-precision map, and is configured to execute the steps of the environmental perception method of the intelligent driving vehicle as described in the present invention.
[0028] In a third aspect, a vehicle described in the present invention adopts the environmental perception system of the intelligent driving vehicle described in the present invention.
[0029] In a fourth aspect, a storage medium described in the present invention stores a computer-readable program therein, and when the computer-readable program is called, it can execute the steps of the environmental perception method of the intelligent driving vehicle described in the present invention.
[0030] The present invention has the following advantages: when the high-precision map and RTK signals are lost, the method of roadside detection and boundary fitting can effectively divide the vehicle's surrounding environment into two subspaces: inside the road (including the roadside) and outside the road. The perception module can more accurately establish the area of interest, thereby establishing an environment perception system based on laser radar, RTK and high-precision maps that is moderate in information, scalable, can work independently, and has strong practicality. The present invention can also avoid filtering out traffic participants in other lanes at intersections. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 is a flow chart of this embodiment;
[0032] Figure 2 is a flow chart of establishing a target grid map in this embodiment;
[0033] Figure 3 is a flow chart of target clustering in this embodiment;
[0034] Figure 4 It is a functional block diagram of this embodiment;
[0035] Figure 5 Schematic diagram of the installation positions of three laser radars in this embodiment;
[0036] In the figure, 1-main radar, 2-side radar, 3-IMU, 4-controller, 5-high-precision map, 6-RTK equipment, 7-vehicle. DETAILED DESCRIPTION
[0037] The present invention will be further described below in conjunction with the accompanying drawings.
[0038] like Figure 4 and Figure 5 As shown, in this embodiment, an environment perception system for an intelligent driving vehicle includes: laser radars installed on the top of a vehicle 7 and on the left and right sides of the vehicle 7; an RTK device 6 installed on the vehicle for outputting centimeter-level high-precision vehicle position information; an IMU 3 installed on the vehicle for detecting inertial measurement information of the vehicle; and a high-precision ground motion sensor for outputting road environment information. Figure 5 ; and a controller 4, the controller 4 is respectively connected to each laser radar, IMU3, RTK equipment 6 and high-precision Figure 5 connect.
[0039] Among them, the laser radar installed on the roof is the main radar 1, and its measurement distance can reach 150m at a reflectivity of 10%. In addition, the smaller vertical angular resolution and horizontal angular resolution can better depict the target, so that the number of point clouds hitting the target object is larger and the point cloud lines are denser, so that the target in front can be discovered earlier, thereby leaving more time for planning and control. However, since the vertical installation distance of the main radar 1 is higher than the ground, it has a larger blind spot around the vehicle. Therefore, it is necessary to install laser radars with relatively few wiring harnesses at lower positions on both sides of the vehicle, and manually set a certain installation angle as needed, which can effectively reduce the blind spot. The laser radars installed on the left and right sides of the vehicle 7 are called side radars 2. The side radars 2 are on both sides of the front of the vehicle or below the rearview mirror. The radars behind the vehicle body can be installed as needed. The system framework has a preset interface, which can quickly access the newly added radar data. The RTK device 6 includes an RTK antenna and a receiver. The RTK antenna and receiver are mainly installed at the rear of the vehicle body. The main principle is to be less interfered and minimize the impact on the vehicle body shape.
[0040] In this embodiment, the multi-radar raw data of the environment perception system of the intelligent driving vehicle needs to be converted into a structure predetermined by the program through a driver. The original data structure of the laser radar is sent in the form of a UDP packet. Each UDP data packet consists of several blocks. Each block contains the data of all laser emitters of the laser radar at the same time. The UDP data packet is stored in the form of a single line and a single frame in the computing processing unit through a special radar data driver. In addition, the conversion drive of different radar point cloud data to the vehicle body coordinate system is obtained by the original radar calibration information based on the correction of the inertial navigation unit data, and then transmitted to the perception system software processing unit at a frequency higher than the point cloud scanning frequency. The main input interface of the software includes: single-line single-frame point cloud data processed by the point cloud driver and the conversion matrix of different radars-vehicle bodies processed by the conversion driver.
[0041] For the received high-precision map chain facility information, since it is a discrete sequence, it is interpolated and fitted into a curve. This line is the dividing line between road space and non-road space. Combined with the vehicle positioning information given by the RTK device 6, when the reflection point is located in the non-road space, it is filtered out. On the contrary, if it is located in the road space, the point is used for subsequent processing, which greatly reduces the false detection and computing power consumption caused by point cloud targets outside the road, reduces the latency of the perception system, and improves the user experience.
[0042] In this embodiment, the environmental perception system of the intelligent driving vehicle, after the data is converted to the vehicle body coordinate system, is filtered through prior knowledge to reduce the subsequent load on computing power and grid maps. The parameters related to the grid map can be modified through the configuration file according to the actual scene to achieve a better match between scene-program, computing power-efficiency. The data of different radars are projected onto the grid plane (OXY plane), and all point clouds are placed in the grid process of different IDs according to the (x, y) position information of the point cloud. In this process, the abnormal point cloud is deleted through the denoising module to reduce the interference of noise on the perception system. After the point cloud projection is completed, according to the sparse characteristics of the point cloud at a long distance, different strategies are used to judge the attributes of the grid at different distances, which can effectively reduce the probability of false detection and missed detection. The establishment of an effective grid map has an important impact on the subsequent target clustering effect and efficiency.
[0043] In this embodiment, the environment perception system of the intelligent driving vehicle uses high-precision Figure 5The discrete sequence points of the chain facilities within 200 meters ahead are fitted into a curve according to spline interpolation to form the dividing line between road space and non-road space. Then, the coordinate information of the object reflection point is used to determine whether the point is located in the road space. If it is outside the road, it is filtered out, and the points inside the road will be processed later. Among them, for the processing of intersections, if the above method is used, the traffic participants in other lanes will be filtered out, which will bring a greater risk of missed detection. For this case, it is necessary to establish an intersection template, determine whether it is an intersection through straight line detection, and then establish an electronic fence (a closed polygonal area composed of multiple points) with the help of high-precision map information. When the reflection point is inside the electronic fence, the point is filtered out. Since this process is essentially a search process, if the search area is too large, the real-time performance of the algorithm will be reduced. Therefore, when establishing the electronic fence, it should be as concise and accurate as possible. When the RTK signal is lost and cannot be accurately positioned, the road boundary line is fitted based on the detection of the curb and isolation belt within 100 meters in front of the vehicle.
[0044] For the point cloud in the road, the target detection of the raster map is used to traverse the entire map and cluster it. Considering the sparseness of the point cloud at a distance, the clustering threshold needs to be modified adaptively according to the distance, which can effectively reduce the over-segmentation and under-segmentation of the target detection. The clustered targets are screened and some targets that do not meet the conditions are filtered out. The screening conditions can be flexibly modified through the configuration file to meet the needs of the scene. In the target visualization stage, the display can be set as a two-dimensional rectangular box, the original two-dimensional convex hull, and a three-dimensional box by modifying the visualization configuration parameters.
[0045] like Figure 1 As shown, in this embodiment, an environment perception method for an intelligent driving vehicle includes:
[0046] When the power supply conditions are met and the main radar 1 and the side radar 2 are in normal state, they will generate point cloud UDP packets according to the established strategy. The factory setting parameters of each radar are different. The corresponding parameter files need to be loaded for correction, and then connected to the original point cloud driver module together to finally obtain the pre-defined laser radar point cloud data format. During the movement of the vehicle, the laser radar continuously obtains information about the surrounding environment according to the preset strategy, which is expressed as circles of point clouds. However, since the radar itself moves with the vehicle body, the points in the same frame of point cloud are collected at different positions, that is, point cloud distortion. Point cloud distortion has a negative impact on the subsequent processing accuracy. Therefore, it is necessary to combine the information of IMU3 to obtain the conversion relationship between the radar position and the initial position of each point cloud, and then convert all points in each frame to the initial radar position. This is point cloud distortion compensation. After distortion compensation, the point cloud is converted from the radar coordinate system to the vehicle coordinate system through the calibration file to obtain the point cloud in the vehicle coordinate system. In order to reduce the amount of calculation and improve efficiency, by receiving real-time RTK positioning information and high-precision map information, it is judged whether there is an RTK signal. If so, the chain facility information within 200m in front of the vehicle is pressed into the queue, and then it is judged whether it is an intersection; if it is at an intersection, the chain facility information is fitted to the road boundary line through spline interpolation, and then combined with the vehicle positioning information given by RTK to determine whether the point cloud is within the lane line. If it is within the lane line, it is retained and a grid map is established, otherwise the point cloud is filtered out; if it is not at an intersection, the grid map is retained and established. If there is no RTK signal, the road isolation belt and road curb are detected around the vehicle, and then the road boundary line is fitted through spline interpolation, and then it is judged whether the point cloud is within the lane line. If it is within the lane line, the grid map is retained and established, otherwise the point cloud is filtered out. After obtaining a complete grid map, the target can be clustered and visualized according to the grid occupancy state. This process mainly involves the determination of clustering strategy, clustering threshold and selection of visualization content.
[0047] like Figure 2As shown, in this embodiment, the grid map establishment process sets the size and resolution of the grid map. Combined with the position of the vehicle in the grid, the grid area, i.e., the forward detection distance, the rearward detection distance, and the lateral detection distance, can be determined. The parameters can be flexibly set for different scenarios to achieve a balance between detection effect and efficiency. Generally speaking, the larger the grid map, the smaller the resolution, the higher the detection accuracy, but the algorithm takes a long time. For some scenarios, such as only paying attention to the information of the two lanes adjacent to the left and right, the lateral distance can be set smaller. For high-speed scenarios, the forward detection distance needs to be set larger. For the laser radar points under the vehicle body coordinates, some points are noise points or abnormal points, which need to be screened by the noise point removal module, and non-noise points are placed in the grid. For the attributes of each grid, the height difference between the highest point and the lowest point in the grid is calculated in the near distance. If the height difference is greater than a certain threshold, it is determined to be the target grid, otherwise it is not the target grid and no clustering is required. Since the point cloud in the distance is sparse, there will be fewer points in the distant grid. Based on the absolute height, that is, whether the highest point in the grid is higher than a certain threshold, if it is greater than the threshold, it is determined to be the target grid, otherwise it is determined to be a non-target grid.
[0048] like Figure 3 As shown, in this embodiment, after the grid map and grid attributes are established, target clustering is performed. The clustering method adopts the flooding method. For distant target grids, the clustering threshold is large, and for nearby target grids, the clustering threshold is small. The acquisition of the specific threshold needs to be determined according to the experimental results under this general principle, which can effectively reduce over-segmentation and under-segmentation. Since target detection based on simple reliance on height information still results in inaccurate detection, it is necessary to determine the constraints of abnormal targets based on experience or experiments to delete abnormal targets. After the process is completed, a series of detected targets and related information including size and speed can be obtained. This information can be given to downstream modules and visualizations. The visualization module can choose to output a two-dimensional convex hull, a two-dimensional rectangular frame, a three-dimensional convex hull, and a three-dimensional cuboid frame.
[0049] like Figure 5 As shown, the installation position relationship between the main radar 1 and the two side radars 2.
[0050] Although exemplary embodiments are described above, it is not intended that these embodiments describe all possible forms encompassed by the claims. The words used in the specification are descriptive rather than restrictive, and it should be understood that various changes may be made without departing from the spirit and scope of the present disclosure. As previously described, the features of the various embodiments may be combined to form further embodiments of the present invention that may not be explicitly described or shown. Although the various embodiments may have been described as providing advantages or being superior to other embodiments or prior art implementations in terms of one or more desired characteristics, it is recognized by those of ordinary skill in the art that, depending on the specific application and implementation, one or more features or characteristics may be compromised to achieve the desired overall system properties. These properties may include, but are not limited to, cost, strength, durability, life cycle cost, appearance, size, manufacturability, functional robustness, etc. In this way, embodiments described as less desirable than other embodiments or prior art implementations in terms of one or more characteristics are not outside the scope of the present disclosure and may be expected to be used in specific applications.
Claims
1. An environment perception system method for an intelligent driving vehicle, characterized in that: The following steps are involved: Get the point cloud UDP package output by the laser radar, and connect it to the original point cloud driver module for processing to obtain the point cloud data format; Obtain IMU information, combine it with the IMU information to find the conversion relationship between the radar position and the initial position of each point cloud, and convert all points in each frame to the initial radar position, that is, perform point cloud distortion compensation; The distortion-compensated point cloud is combined with the radar calibration file to complete the conversion from the radar coordinate system to the vehicle coordinate system, and the point cloud in the vehicle coordinate system is obtained; Obtain RTK positioning information and high-precision map information to determine whether there is an RTK signal; If there is an RTK signal, it is determined whether the vehicle (7) is at an intersection; if it is at an intersection, the chain facility information is fitted to the road boundary line through spline interpolation, and then combined with the vehicle positioning information given by RTK to determine whether the point cloud is within the lane line. If it is within the lane line, it is retained and a grid map is established, otherwise the point cloud is filtered out; if it is not at an intersection, it is retained and a grid map is established; If there is no RTK signal, the vehicle sensor will be used to check the road isolation belt and road curb within 100 meters in front of the vehicle, and the road boundary line will be fitted through spline interpolation to determine whether the point cloud is within the lane line. If it is within the lane line, it will be retained and a grid map will be established, otherwise the point cloud will be filtered out. For the processing of intersections, an intersection template needs to be established, and straight line detection is used to determine whether it is an intersection. An electronic fence is established with the help of high-precision map information. When the reflection point of the lidar is within the electronic fence, the point is filtered out. After obtaining a complete grid map, objects are clustered and visualized according to the grid occupancy status.
2. The method for an environment perception system of an intelligent driving vehicle according to claim 1, characterized in that: Create a raster map, specifically: Setting the size and resolution of the grid map, and determining the grid area, namely the forward detection distance, the rearward detection distance, and the lateral detection distance, in combination with the position of the vehicle (7) in the grid; Determine whether the laser radar point under the vehicle body coordinates is a noise point. If it is a noise point, it will not be put into the grid. If it is not a noise point, it will be put into the grid and determine whether it is a target grid. If it is a target grid, target clustering will be performed. If it is not a target grid, clustering will not be performed.
3. The environment perception system method of the intelligent driving vehicle according to claim 2, characterized in that: For the nearby point cloud, the height difference between the highest point and the lowest point in the grid is calculated. If the height difference is greater than a first preset threshold, it is determined to be a target grid, otherwise it is determined to be a non-target grid; For the distant point cloud, it is determined whether the highest point in the grid is higher than the second preset threshold. If it is higher, it is determined to be the target grid, otherwise it is determined to be a non-target grid.
4. The environment perception system method of an intelligent driving vehicle according to any one of claims 1 to 3, characterized in that: The specific steps of target clustering include: The flooding method is used to cluster targets and determine whether the target grid meets the preset constraints. If not, the target is deleted. If it is satisfied, the target is output and visualized.
5. The method for the environment perception system of an intelligent driving vehicle according to claim 4, characterized in that: Each UDP packet consists of several blocks, and each block contains the data of all laser transmitters of the lidar at the same time.
6. The method for the environment perception system of an intelligent driving vehicle according to claim 5, characterized in that: Before connecting to the original point cloud driver module, load the corresponding parameter file of the laser radar for correction.
7. An environment perception system for an intelligent driving vehicle, characterized in that: include: Laser radars installed on the top of the vehicle (7) and on the left and right sides of the vehicle (7); An RTK device (6) installed on the vehicle for outputting centimeter-level high-precision vehicle position information; An IMU (3) mounted on the vehicle for detecting inertial measurement information of the vehicle (7); High-precision map for outputting road environment information (5); And a controller (4), wherein the controller (4) is connected to each laser radar, RTK equipment (6), IMU (3) and high-precision map (5) respectively, and the controller (4) is configured to execute the steps of the environment perception system method of an intelligent driving vehicle as described in any one of claims 1 to 6.
8. A vehicle, characterized in that: An environmental perception system for an intelligent driving vehicle as described in claim 7 is adopted.
9. A storage medium, characterized in that: A computer-readable program is stored therein, and when the computer-readable program is called, the steps of the environment perception system method for an intelligent driving vehicle as described in any one of claims 1 to 6 can be executed.
Citation Information
Patent Citations
Automobile driving obstacle detection method and system based on multiple laser radars
CN111060923A
Data fusion method based on multiple laser radars
CN111413684A
Lane fitting method
CN108573272A
Multi-sensor fusion road extraction and indexing method based on global and local grid maps
CN111273305A
High-precision map generation method and device, electronic equipment and storage medium
CN111667545A
