A method for generating occupancy grid maps by integrating fisheye traversable space detection

Abstract

The present invention discloses a method for generating an occupied grid map by integrating fisheye traversable space detection. The method integrates fisheye traversable space detection with laser radar traversable space detection to generate a more accurate and stable traversable space occupied grid map. The fisheye image is rich in information and will not detect rain, fog, dust, etc. as impassable. The grid updated by laser detection is back-projected into the fisheye image, and the occupancy probability of the grid detected as traversable by the fisheye is reduced. Laser false detection is suppressed, and the impact of the laser falsely detecting dust, rain, fog, etc. as impassable is reduced. At the same time, since the horizontal and vertical field of view angles of the fisheye camera are 180 degrees, there are basically no blind spots. The traversable space situation within the laser blind spot can be detected. The occupancy probability of the grid in the blind spot and detected as impassable by the fisheye is maintained, thereby ensuring the safety of autonomous driving.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent driving technology and relates to occupancy grid map generation for scenes such as mining areas and bulk cargo terminals. In particular, it relates to an occupancy grid map generation method integrating fisheye traversable space detection. Background Art

[0002] In the field of autonomous driving, traversable space generally refers to the area around a vehicle that is safe for driving. During driving, traversable space information can be used to determine whether the vehicle is about to stray from the safe zone or collide with an obstacle. If the vehicle is detected as about to enter a non-traversable space, the system can promptly take safety measures such as braking or evasive maneuvers. Traversable space also provides constraints for the path planning algorithm. By accurately sensing traversable space, the path planning module can generate a safer and more efficient driving path.

[0003] Due to the advantages of LiDAR, such as direct acquisition of three-dimensional information and its unaffected by lighting, the current generation of traversable spaces is primarily based on LiDAR detection. However, LiDAR is also susceptible to interference from rain, fog, dust, and other factors, leading to false detections, and lacks information such as color, texture, and semantics. Furthermore, to ensure long-range target detection, LiDARs on autonomous vehicles are typically mounted high. Due to their limited vertical field of view, there is a certain blind spot for laser perception at close ranges to the vehicle body. If an obstacle enters this blind spot, it cannot be detected by the laser, posing a greater risk of collision. Summary of the Invention

[0004] In response to the above problems, the main purpose of the present invention is to design an occupancy grid map generation method that integrates fisheye traversable space detection, introduces a fisheye camera for traversable space detection, and solves the problems of large blind spots of lidar and susceptibility to false detection caused by rain, fog, and dust.

[0005] In order to achieve the above purpose, the present invention adopts the following technical solutions:

[0006] A method for generating an occupancy grid map by integrating fisheye traversable space detection is provided. The method is used for a vehicle equipped with a laser radar and a fisheye camera, and comprises the following steps:

[0007] Establish a benchmark occupancy grid map by combining lidar blind spot information;

[0008] Get LiDAR traversable space detection and fisheye traversable space detection as input data;

[0009] Based on the timestamp of the received lidar traversable space detection in the current frame and the reference occupancy grid map, an occupancy grid map at the current moment is established;

[0010] According to the motion information of the ego vehicle, the occupancy grid map information of the previous moment is converted into the occupancy grid map of the current moment;

[0011] Subtract a priori value from the occupancy probability of all grids in the current occupancy grid map, so that the occupancy probability of grids that have not been updated by the lidar traversable space detection can be reduced;

[0012] Project the current frame's lidar traversable space detection onto the current moment's occupancy grid map, fuse the fisheye traversable space detection results, and update the grid occupancy probability;

[0013] Through fisheye traversable space detection, determine whether the grids in the lidar blind area that have not been updated by the lidar traversable space detection should perform grid occupancy probability maintenance;

[0014] The moving object is not expressed in the form of an occupied grid map, and the occupancy probability of the grid within the target outline is cleared by the received target tracking results;

[0015] Extract the index of the grid whose occupancy probability is greater than the threshold in the current occupancy grid map and output it to downstream for use.

[0016] As a further description of the present invention, the lidar traversable space detection is the point cloud information of stationary or moving obstacles around the vehicle at the current moment; The laser radar traversable space detection at this moment is , It consists of a series of points, each of which Contains location and type information ;in, is the point under the vehicle system axis, Axis and Axis coordinates, the vehicle coordinate system takes the center of the vehicle as the origin, and the forward direction is Axis positive direction, left is Axis positive direction, upward Axis positive direction, Indicates the type information of the point.

[0017] As a further description of the present invention, the fisheye traversable space detection is composed of a set of horizontally continuous pixel points on the image information; the pixel points represent the boundary of the traversable space on the image, the upper side of the boundary is the impassable area, and the lower side of the boundary is the traversable area; The detection of the boundary of the fisheye traversable space at a certain moment is expressed as:

[0018] , , ;

[0019] in, is the height of the fisheye image, that is, the number of pixels in the vertical direction of the fisheye image. is the width of the fisheye image, that is, the number of pixels in the horizontal direction of the fisheye image; is the horizontal coordinate of the detection point in the fisheye image pixel coordinate system, ranging from 1 to Continuous growth, that is, the boundary points of the traversable space on the fisheye image have a total of indivual;

[0020] by As the index, you can query The vertical coordinate of the traversable boundary on the corresponding column ,Right now , .

[0021] As a further description of the present invention, the occupied grid map includes a plurality of grids, each of which is a square, and the physical size of the occupied grid map is ,in, is the side length of the grid, representing the resolution of the grid, is the number of rows in the grid map, is the number of columns of the raster map, is the total number of grids in the grid map;

[0022] The starting point of the grid map is the upper left corner of the grid map, that is, the grid at row 0 and column 0. , the origin of the vehicle is located at index The grid of Row, No. The grid of columns is represented as ,in, ;

[0023] The properties of each grid are ,in, is the position of the grid under the vehicle system, express, The grid position under the vehicle system axis, Axis coordinates, is the probability that the grid is inaccessible, that is, the grid occupancy probability, It is the flag indicating whether the grid is in the blind area of ​​the laser radar. Indicates that the grid is in the blind area of ​​the lidar. Indicates that the grid is not in the blind area of ​​the lidar. Whether the grid occupancy probability is updated by the lidar point, Indicates whether the grid occupancy probability is maintained.

[0024] As a further description of the present invention, the establishment of the reference occupancy grid map includes the following steps:

[0025] Initialize a OK, Base occupancy grid map for columns , traverse each grid and initialize the grid properties; for Middle Row, No. Column Grid Initialize the properties:

[0026] grid Occupancy probability initialization, grid The occupation probability is set to the initial value , that is, grid Occupancy probability ;

[0027] grid Position under the vehicle system Initialized as:

[0028] ;

[0029] ;

[0030] Blind spot sign calculation, the blind spot is a fixed area around the vehicle, using closed area Indicates; if the grid Position under the vehicle system In the blind spot of the car If the grid is inside The blind spot flag is initialized to If the grid Position under the vehicle system Not in the blind spot of the vehicle If the grid is inside The blind spot flag is initialized to ;

[0031] grid Other flags are initialized:

[0032] ;

[0033] ;

[0034] in, Indicates whether the grid occupancy probability is updated by the lidar point. The default value of the base occupancy grid map is that all grid occupancy probabilities have not been updated by the lidar point. , Indicates whether the grid occupancy probability is maintained. The default value for the base grid occupancy map is that all grid occupancy probabilities are not maintained. .

[0035] As a further description of the present invention, establishing an occupancy grid map at the current moment includes the following steps:

[0036] LiDAR traversable space detection input at the current moment The timestamp is , based on the base occupancy grid map Create a The new occupation grid map at all times , .

[0037] As a further description of the present invention, the previous frame Occupancy grid map at each moment Information conversion to occupancy grid map The occupancy probability of the whole grid is reduced, including the following steps:

[0038] according to The position of the vehicle in the world coordinate system at this moment and orientation angle ,as well as The position of the vehicle in the world coordinate system at this moment and orientation angle , calculated arrive The translation vector of the vehicle at time and the rotation matrix ;

[0039] Rotation Matrix The expression is:

[0040] ;

[0041] in, for arrive The difference in orientation angle at each moment, ;

[0042] Translation vector The expression is:

[0043] ;

[0044] According to the translation vector and the rotation matrix ,Will Always occupy the grid map Middle Row, No. Column Grid Convert the position to the current moment From the car system, the grid is obtained The position of the vehicle at the time , the expression is:

[0045] ;

[0046] ;

[0047] in, For grid exist Always from the position under the vehicle;

[0048] calculate Occupies the grid map at the current moment Index in , the expression is:

[0049] ;

[0050] ;

[0051] in, is the horizontal index, is the horizontal index of the vehicle origin grid, is the vertical index, is the vertical index of the vehicle origin grid, is the side length of the grid, representing the resolution of the grid, Indicates that the result is rounded;

[0052] Determine whether the index occupies the grid map In, that is, the index satisfies and ,in, is the number of rows in the grid map, is the number of columns of the raster map;

[0053] If the index is in the occupied grid map Inside, then Occupies the grid map at the current moment The corresponding new grid for: ;

[0054] like , then assign the original grid occupancy probability to the new grid, that is ;in, is the occupancy probability of the new grid, is the original grid occupancy probability;

[0055] Once the conversion is complete, it will occupy the grid map The occupancy probability of all grids in minus the prior value .

[0056] As a further description of the present invention, update the current moment occupied grid map The grid occupancy probability of , including the updated grid occupancy probability of the lidar traversable space detection and the updated grid occupancy probability of the fisheye traversable space detection, is as follows:

[0057] Traversal LiDAR traversable space detection at all times , calculate the Laser points Occupies the grid map at the current moment Index in , the expression is:

[0058] ;

[0059] ;

[0060] in, Laser point In the car system Axis coordinates, For laser point In the car system Axis coordinates, is the horizontal index, is the horizontal index of the vehicle origin grid, is the vertical index, is the vertical index of the vehicle origin grid, is the side length of the grid, representing the resolution of the grid, Indicates that the result is rounded;

[0061] Determine whether the index occupies the grid map Within the range, that is, the index satisfies and ;in, is the number of rows in the grid map, is the number of columns of the raster map;

[0062] If the index is in the occupied grid map If the laser point occupies the grid map The corresponding grid in is: ;

[0063] Update Raster Occupancy probability:

[0064] If the grid update flag is for ,Right now , indicating that the grid has been updated by other laser points before and does not need to be updated again. Continue to process the lidar traversable space detection The next laser point;

[0065] If the grid update flag is for ,Right now , indicating that the grid has not been updated by other laser points, and the grid occupancy probability needs to be updated. Then the grid occupancy probability Add a fixed value , representing the grid Since there is a laser detection point hit, the grid occupancy probability rise;

[0066] Update the position information of the grid, that is ; and set the grid update flag position to ,Right now , indicating that the occupancy probability of the grid has been updated;

[0067] A raster that will be updated by lidar traversable space detection , projected onto the fisheye image, and obtain the corresponding pixel points of the grid on the fisheye image The coordinates of ;in, is the horizontal coordinate of the pixel point in the fisheye image pixel coordinate system, is the ordinate of the pixel point in the fisheye image pixel coordinate system;

[0068] pass The horizontal axis , the boundary detected from the fisheye traversable space at the current moment Get The vertical coordinate of the fisheye traversable space detection boundary ,Right now ;like , then the grid In the passable space of fisheye detection, that is, the fisheye passable space detection considers the grid Passable, at this time the grid occupancy probability Subtract a fixed value , reducing the grid occupancy probability; if , then the grid If the grid is not within the traversable space of the fisheye detection, the occupancy probability of the grid is not adjusted.

[0069] As a further description of the present invention, determining whether to maintain grid occupancy probability for grids that are not updated by the lidar in the lidar blind area includes the following steps:

[0070] Traverse the grid map occupied at the current moment For all grids in Row, No. Column Grid ,if It is in the blind area of ​​the laser radar traversable space and has not been detected and updated by the laser radar traversable space, that is, for , for , and the grid occupancy probability has not been maintained, that is, for , then the grid Project it onto the fisheye image to get the corresponding pixel points of the grid on the fisheye image The coordinates of ;in, is the horizontal coordinate of the pixel point in the fisheye image pixel coordinate system, is the ordinate of the pixel point in the fisheye image pixel coordinate system;

[0071] pass The horizontal axis , the boundary detected from the fisheye traversable space at the current moment Get The vertical coordinate of the fisheye traversable space detection boundary ,Right now ;like , then the grid If the grid is in the impassable space detected by the fisheye, the globally reduced prior probability is restored, that is, the current grid occupancy probability Plus ;like , then the grid Within the traversable space of fisheye detection, the occupancy probability of the grid is not maintained.

[0072] As a further description of the present invention, the moving object is not expressed in the form of an occupied grid map, and the grid occupancy probability within the target outline is cleared through the target tracking result and output to the downstream, including the following steps:

[0073] Use the closed area to define the target's outline Indicates that the grid map occupied at the current moment is traversed For all grids in Row, No. Column Grid , if the grid Position under the vehicle system In target outline If the grid is within the range of Set as the initial value of the grid occupancy probability ,Right now ;

[0074] Traverse the current moment's occupancy grid map For all grids in Row, No. Column Grid , if its occupation probability Greater than the output occupation probability threshold , then the index Put it into the output index collection;

[0075] The index set obtained after the traversal is the occupancy grid map composed of high occupancy probability grids. The index set is sent to the downstream, and the downstream is based on the index. Combined grid resolution , restore the position of the occupied grid under the own vehicle system.

[0076] Compared with the prior art, the technical effects of the present invention are:

[0077] The present invention provides a method for generating an occupancy grid map by integrating fisheye traversable space detection. The method integrates fisheye traversable space detection with laser radar traversable space detection to generate a more accurate and stable traversable space occupancy grid map. Since fisheye images are rich in information, rain, fog, dust, etc. will not be detected as impassable. The grid updated by laser detection is back-projected into the fisheye image, and the occupancy probability of the grid detected as impassable by the fisheye is reduced, which can suppress laser false detection and reduce the impact of the laser falsely detecting dust, rain, fog, etc. as impassable. At the same time, since the horizontal and vertical field of view angles of the fisheye camera are 180 degrees, there is basically no blind spot, and the traversable space situation within the laser blind spot can be detected. For the grid in the blind spot and detected as impassable by the fisheye, the occupancy probability is maintained. Therefore, after the obstacle grid enters the blind spot, it will not be judged as impassable because the occupancy probability decreases rapidly after the laser detection disappears, thereby ensuring the safety of autonomous driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 Schematic diagram of the overall method of the present invention;

[0079] Figure 2 This is a schematic diagram of the laser radar traversable space detection of the present invention;

[0080] Figure 3 For the purpose of detecting the fisheye traversable space of the present invention;

[0081] Figure 4 A schematic diagram of an occupation grid map in the present invention;

[0082] Figure 5 Schematic diagram of updating grid occupancy probability for fisheye traversable space detection according to the present invention. DETAILED DESCRIPTION

[0083] The present invention is described in detail below with reference to the accompanying drawings:

[0084] In one embodiment of the present invention, a method for generating an occupancy grid map integrating fisheye traversable space detection is disclosed. In order to reduce the blind spots of the LiDAR and reduce the impact of the laser misdetecting rain, fog, and dust as impassable, a fisheye camera is introduced to detect traversable space. The fisheye camera has rich semantic information for traversable space detection, can accurately distinguish between water mist and dust, and can suppress the laser's misdetection of rain, fog, and dust. At the same time, the fisheye camera has a large vertical field of view, usually up to 180 degrees, which can well compensate for the blind spots of the LiDAR at close range. Specifically, refer to Figure 1-5 As shown, the method is used for a vehicle equipped with a laser radar and a fisheye camera, comprising the following steps:

[0085] Establish a benchmark occupancy grid map by combining lidar blind spot information;

[0086] Get LiDAR traversable space detection and fisheye traversable space detection as input data;

[0087] Based on the timestamp of the received lidar traversable space detection in the current frame and the reference occupancy grid map, an occupancy grid map at the current moment is established;

[0088] According to the motion information of the ego vehicle, the occupancy grid map information of the previous moment is converted into the occupancy grid map of the current moment;

[0089] Subtract a priori value from the occupancy probability of all grids in the current occupancy grid map, so that the occupancy probability of grids that have not been updated by the lidar traversable space detection can be reduced;

[0090] Project the current frame's lidar traversable space detection onto the current moment's occupancy grid map, fuse the fisheye traversable space detection results, and update the grid occupancy probability;

[0091] Through fisheye traversable space detection, determine whether the grids in the lidar blind area that have not been updated by the lidar traversable space detection should perform grid occupancy probability maintenance;

[0092] The moving object is not expressed in the form of an occupied grid map, and the occupancy probability of the grid within the target outline is cleared by the received target tracking results;

[0093] Extract the index of the grid whose occupancy probability is greater than the threshold in the current occupancy grid map and output it to downstream for use.

[0094] It should be noted that, in this embodiment, the above-mentioned operation process is performed based on the grid map occupied at the current moment; among them, operations including converting the previous frame information to the current frame, reducing the global probability, updating the occupation probability, maintaining the blind spot, etc. are all performed on the grid map occupied at the current moment, specifically, operations are performed on a certain grid therein.

[0095] This embodiment provides a detailed description of the method for integrating the above-mentioned fisheye camera passable space detection with the laser radar passable space detection, as follows:

[0096] 1. LiDAR and fisheye traversable space detection input instructions

[0097] 1. LiDAR traversable space detection

[0098] like Figure 2 As shown in the figure, the lidar traversable space detection is the point cloud information of the static or moving obstacles around the vehicle at the current moment; The laser radar traversable space detection at this moment is , It consists of a series of points, each of which Contains location and type information ;in, is the point under the vehicle system axis, Axis and Axis coordinates, the vehicle coordinate system takes the center of the vehicle as the origin, and the forward direction is Axis positive direction, left is Axis positive direction, upward Axis positive direction, Indicates the type information of the point.

[0099] 2. Fisheye traversable space detection

[0100] like Figure 3 As shown, the fisheye passable space detection consists of a group of horizontally continuous pixel points on the image information; the pixel points represent the boundary of the passable space on the image. The upper side of the passable boundary on the fisheye image is the impassable area, and the lower side of the boundary is the passable area. Specifically, the upper side of the boundary is a fence, which is an impassable area.

[0101] In this embodiment, the detection points of the fisheye traversable space boundary are expressed as:

[0102] , , ;

[0103] in, is the height of the fisheye image, that is, the number of pixels in the vertical direction of the fisheye image. is the width of the fisheye image, that is, the number of pixels in the horizontal direction of the fisheye image; is the horizontal coordinate of the detection point in the fisheye image pixel coordinate system, ranging from 1 to Continuous growth, that is, the boundary points of the traversable space on the fisheye image have a total of ; As the index, you can query The vertical coordinate of the traversable boundary on the corresponding column ,Right now , .

[0104] 2. Description of the Occupied Grid Map

[0105] 1. Definition of Occupancy Grid

[0106] In this embodiment, the traversable space is represented by occupying a grid map. Figure 4 As shown, the grid map consists of several grids, each of which is a square. The physical size of the grid map is ,in, is the side length of the grid, representing the resolution of the grid, is the number of rows in the grid map, is the number of columns of the raster map, The total number of grid cells in the grid map. The starting point of the grid map is the upper left corner of the grid map, that is, the grid cell at row 0 and column 0. , the origin of the vehicle is located at index 's grid.

[0107] No. Row, No. The grid of columns is represented as ,in, ; The attributes of each grid are ;in, is the position of the grid under the vehicle system, usually express, The grid position under the vehicle system axis, Axis coordinates, is the probability that the grid is inaccessible, that is, the grid occupancy probability, Is the flag bit of whether the grid is in the blind area of ​​the laser radar, When , it means the grid is in the blind area of ​​the laser radar. When , it means the grid is not in the blind area of ​​the laser radar. Whether the grid occupancy probability has been updated by the lidar point, Indicates whether the grid occupancy probability is maintained.

[0108] 2. Baseline occupancy grid map Establish

[0109] Initialize a OK, Base occupancy grid map for columns , traverse each grid and initialize the grid properties.

[0110] Specifically, this embodiment targets Middle Row, No. Column grid with Indicates that for the grid Initialize the properties:

[0111] (1) Grid Occupancy probability initialization

[0112] grid The occupation probability is set to the initial value , that is, grid Occupancy probability ;

[0113] (2) Grid Position under the vehicle system Initialized as:

[0114] ;

[0115] ;

[0116] (3) Calculation of blind spot markers

[0117] The blind spot is a fixed area around the vehicle, which is determined by the installation position of the LiDAR and the field of view of the LiDAR. Therefore, the index of the blind spot grid in each occupied grid map is fixed and will not change according to the movement of the vehicle. The flag only needs to be calculated once based on the actual blind spot situation, and can be used when creating each new occupation grid map.

[0118] Use closed area in the blind spot of the vehicle Indicates that the closed area is usually a polygon; if the grid Position under the vehicle system In the blind spot of the car If the grid is within the range of , the blind area flag of this grid is initialized to If the grid Position under the vehicle system Not in the blind spot of the vehicle If the grid is within the range of , the blind area flag of this grid is initialized to ;

[0119] grid Other flags are initialized:

[0120] ;

[0121] ;

[0122] in, Indicates whether the grid occupancy probability is updated by the lidar point. The default value of the base occupancy grid map is that all grid occupancy probabilities have not been updated by the lidar point. , Indicates whether the grid occupancy probability is maintained. The default value for the base grid occupancy map is that all grid occupancy probabilities are not maintained. .

[0123] It should be noted that, any subsequent creation of Occupancy grid map , directly copy That's it, .

[0124] 3. Processing flow of the occupation grid map generation method

[0125] According to the timestamp of the received lidar traversable space detection in the current frame, the current occupancy grid map is initialized, and the occupancy grid map information of the previous frame is transformed into the current occupancy grid map according to the posture of the vehicle.

[0126] Subtracting the fixed prior probability from the occupancy probability of all grids ensures that the occupancy probability of the grid updated without lidar traversable space detection can be reduced.

[0127] The current frame's lidar traversable space detection is projected onto the current moment's occupied grid map, and combined with the fisheye traversable space detection, the occupancy probability of the grid is updated.

[0128] Combined with the fisheye traversable space detection, determine whether it is necessary to maintain the occupancy probability of the grids in the blind area that have not been updated by the lidar traversable space detection.

[0129] Expressing moving objects in the form of targets rather than occupied grids is more conducive to the prediction and planning module to accurately predict the trajectory of the target. Specifically, based on the input target tracking results, the occupied grid map is operated to reduce the occupancy probability of the grid inside the moving target outline to the initial value and not output it externally.

[0130] 4. Establish the current moment occupancy grid map

[0131] In this embodiment, the current moment occupancy grid map is established, and the previous frame Occupancy grid map at each moment Information conversion to occupancy grid map The occupancy probability of the whole grid is reduced, including the following steps:

[0132] 1. Establish Always occupy the grid map

[0133] The timestamp of the lidar traversable space detection input at the current moment is , based on the base occupancy grid map Create a The new occupation grid map at all times , .

[0134] 2. Information conversion to

[0135] Previous frame The occupancy grid map at the moment, according to The position of the vehicle in the world coordinate system at this moment and orientation angle ,as well as The position of the vehicle in the world coordinate system at this moment and orientation angle , calculated arrive The translation vector of the vehicle at time and the rotation matrix ;

[0136] Rotation Matrix The expression is:

[0137] ;

[0138] in, for arrive The difference in orientation angle at each moment, ;

[0139] Translation vector pass The position at the moment minus The position at the moment is obtained, and the expression is:

[0140] ;

[0141] according to arrive The translation vector of the vehicle at time and the rotation matrix ,Will Occupancy grid map at each moment Information conversion to the current moment Occupancy grid map The specific conversion steps are:

[0142] (1) According to the translation vector and the rotation matrix ,Will Always occupy the grid map Middle Grid Convert the position to the current moment From the car system, the grid is obtained The position of the vehicle at the time , the expression is:

[0143] ;

[0144] ;

[0145] in, For grid exist Always from the position under the vehicle;

[0146] (2) Calculation Occupies the grid map at the current moment Index in , the expression is:

[0147] ;

[0148] ;

[0149] in, is the horizontal index, is the horizontal index of the vehicle origin grid, is the vertical index, is the vertical index of the vehicle origin grid, is the side length of the grid, representing the resolution of the grid, Indicates that the result is rounded;

[0150] (3) Determine whether the index occupies the grid map Within the range, that is, the index satisfies and ,in, is the number of rows in the grid map, is the number of columns in the grid map; if the index occupies the grid map Within the range, In the current grid map The corresponding new grid for ;like , then assign the original grid occupancy probability to the new grid, that is ;in, is the occupancy probability of the new grid, is the original grid occupancy probability.

[0151] 3. Global occupation probability is reduced

[0152] Once the conversion is complete, it will occupy the grid map The occupancy probability of all grids in minus a priori value ; Ensure that the probability of grid occupancy for updated detections of spaces without lidar traversable space can be reduced.

[0153] 5. Update grid occupancy probability

[0154] Update the grid map occupied at the current moment The grid occupancy probability includes: updating the grid occupancy probability by LiDAR traversable space detection and updating the grid occupancy probability by fisheye traversable space detection; calculating the grid corresponding to the LiDAR traversable space detection point in the current occupied grid map; confirming whether the grid occupancy probability needs to be updated based on the grid update flag; if the occupancy probability needs to be updated, the grid occupancy probability is updated in combination with the LiDAR traversable space detection and fisheye traversable space detection; if the grid occupancy probability does not need to be updated, no operation is performed. The specific steps are as follows:

[0155] 1. Calculate the grid corresponding to the lidar traversable space detection point in the current occupied grid map

[0156] Traversal LiDAR traversable space detection at all times , calculate the Laser points On the currently occupied grid map Index in , the expression is:

[0157] ;

[0158] ;

[0159] in, For laser point In the car system Axis coordinates, For laser point In the car system Axis coordinates, is the horizontal index, is the horizontal index of the vehicle origin grid, is the vertical index, is the vertical index of the vehicle origin grid, is the side length of the grid, representing the resolution of the grid, Indicates that the result is rounded;

[0160] Determine whether the index occupies the grid map Within the range, that is, the index satisfies and ;in, is the number of rows in the grid map, is the number of columns in the grid map; if the index occupies the grid map range, then this laser point occupies the grid map The corresponding grid in is: .

[0161] 2. According to the grid update flag, confirm whether the grid occupancy probability needs to be updated

[0162] If the grid update flag is for ,Right now , indicating that the grid has been updated by other laser points before and does not need to be updated again. Continue to process the lidar traversable space detection The next laser point.

[0163] If the grid update flag is for ,Right now , indicating that the grid has not been updated by other laser points and the grid occupancy probability needs to be updated.

[0164] 3. Update grid occupancy probability by combining lidar traversable space detection and fisheye traversable space detection

[0165] (1) Update grid occupancy probability based on lidar traversable space detection

[0166] If the grid update flag is for , then the occupancy probability of the grid is Add a fixed value , representing the grid Since there is a laser detection point hit, the grid occupancy probability Rise; then update the position information of the grid, that is ; and set the grid update flag position to ,Right now , indicating that the occupancy probability of the grid has been updated; even if the laser clicks hit this grid in the future, it will not lead to an increase in the occupancy probability of the grid, preventing this grid from being updated repeatedly.

[0167] (2) Calculate the projection coordinates of the grid in the fisheye image

[0168] A raster that will be updated by lidar traversable space detection , projected onto the fisheye passable space for detection, and obtain the corresponding pixel points of the grid on the fisheye image The coordinates of .

[0169] In this embodiment, according to the installation angle of the fisheye camera , calculate the rotation matrix from the camera system to the vehicle system, the expression is:

[0170] ;

[0171] in, are the yaw, pitch and roll angles of the camera installed under the ego vehicle.

[0172] The grid map is built on a plane with a vehicle height of 0, so the grid Position under the vehicle system: ,in, The position of the grid under the vehicle. According to the installation position of the fisheye camera under the vehicle , and the rotation matrix from the camera system to the vehicle system , calculate the grid Position in the fisheye camera coordinate system: ; Expressed as ,but With optical axis Axis angle , the expression is: .

[0173] Calculate the projection radius based on the fisheye projection model , assuming that the fisheye camera projection model is an isometric projection model, then ,in, Indicates focal length; considering the distortion coefficient of the fisheye camera , for the projection radius Perform distortion correction using the radial distortion model and the corrected projection radius .

[0174] Use the camera's intrinsic matrix , the corrected Projected onto the fisheye image plane to obtain a grid Pixel coordinates of the fisheye image projected back :

[0175] ;

[0176] in, and are the horizontal and vertical coordinates of the pixel point in the fisheye image pixel coordinate system; is the horizontal focal length scaling factor in pixels, is the vertical focal length scaling factor, in pixels; and is the center point of the fisheye image pixel coordinate system.

[0177] (3) Update grid occupancy probability based on fisheye traversable space detection

[0178] In this embodiment, by The horizontal axis , which can be detected from the fisheye traversable space at the current moment Query and obtain The vertical coordinate of the fisheye traversable space detection boundary ,Right now .like , then the grid In the passable space of fisheye detection, that is, the fisheye passable space detection considers the grid Passable, at this time it is necessary to set the grid occupancy probability Subtract a fixed value , reducing the grid occupancy probability; if , then the grid If the grid is not in the passable space of fisheye detection, the occupancy probability of the grid is not adjusted, such as Figure 5 shown.

[0179] 6. Blind spot grid occupancy probability is maintained

[0180] Because the LiDAR has a blind spot near the vehicle body, as the vehicle or target moves, if an obstacle enters the LiDAR blind spot, the occupancy probability of the grid in this area will decrease due to the lack of laser observation, causing it to be judged as passable and creating a collision risk. Therefore, for grids in the blind spot, it is necessary to combine fisheye passable space detection to maintain occupancy probability.

[0181] Specifically, in this embodiment, determining whether to maintain grid occupancy probability for grids that are not updated by the lidar within the lidar blind area includes the following steps:

[0182] Traverse the grid map occupied at the current moment For all grids in Row, No. Column Grid ,if It is in the blind area of ​​the laser radar traversable space and has not been detected and updated by the laser radar traversable space, that is, for , for , and the grid occupancy probability has not been maintained, that is, for , then the grid Project it onto the fisheye image to get the corresponding pixel points of the grid on the fisheye image The coordinates of ;in, is the horizontal coordinate of the pixel point in the fisheye image pixel coordinate system, is the ordinate of the pixel point in the fisheye image pixel coordinate system;

[0183] pass The horizontal axis , which can be detected from the fisheye traversable space at the current moment Get The vertical coordinate of the fisheye traversable space detection boundary ,Right now .like , then the grid If the grid is in the impassable space detected by the fisheye, the globally reduced prior probability is added back, that is, the current grid occupancy probability Plus , and at the same time, the occupation probability needs to be maintained at the flag position Set to ;like , then the grid Within the traversable space of fisheye detection, the occupancy probability of the grid is not maintained.

[0184] 7. Clear the grid occupancy probability within the target outline and output the occupied grid map

[0185] 1. Clear the grid occupancy probability within the target outline

[0186] Expressing moving objects in the form of targets rather than occupied grids is more conducive to the vehicle prediction and planning module to accurately predict the trajectory of the target. Therefore, it is necessary to operate the occupied grid map based on the input target tracking results, reduce the grid occupancy probability within the moving target outline to the initial value, and do not output it externally.

[0187] Specifically, the target's outline is enclosed in a closed area. Indicates that the closed area is usually a polygon; traverse the grid map occupied at the current moment For all grids in Row, No. Column Grid , if the grid Position under the vehicle system In target outline If the grid is within the range of Set as the initial value of the grid occupancy probability ,Right now .

[0188] 2. Occupancy grid map output

[0189] Traverse the current moment's occupancy grid map For all grids in Row, No. Column Grid , if its occupation probability Greater than the output occupation probability threshold , then the index Put it into the output index set; after the traversal is completed, the index set obtained is the occupancy grid map composed of high occupancy probability grids, and the index set is sent to the downstream, and the downstream is based on the index Combined grid resolution , restore the position of the occupied grid under the own vehicle system.

[0190] The above disclosure discloses specific embodiments of the present invention. It should be noted that the aforementioned fisheye traversable space detection is not limited to being performed on the original fisheye image. It also includes performing it on a histogram of the original fisheye image. The processing is similar, with the grid back-projected onto the fisheye image for correlation verification and blind spot maintenance. Furthermore, the aforementioned calculation of grid occupancy probability is a cumulative addition and subtraction of probabilities, but this is not limited to this method. Other methods for calculating occupancy probability are also included, such as occupancy probability accumulation using methods such as Bayesian theorem and evidence theory.

[0191] Based on the above, the present invention discloses a method for generating an occupancy grid map that integrates fisheye traversable space detection and lidar traversable space detection. The method is applicable to scenarios such as mining areas and bulk cargo terminals. Compared with the existing technology, the present invention has the following advantages:

[0192] 1. The present invention establishes the occupancy grid map template under the vehicle series as the base occupancy grid map. The flag indicating whether each grid in the template is a blind spot only needs to be calculated once and then copied subsequently, thereby improving the efficiency of generating the occupancy grid map.

[0193] 2. Based on the changes in the vehicle's motion state, the present invention converts the occupancy grid map from the previous moment to the current moment and reduces the global occupancy probability to ensure that the occupancy probability of grids not updated by laser detection is reduced;

[0194] 3. This invention projects the fisheye image back onto the grids updated by laser detection, reducing the occupancy probability of grids detected as passable by the fisheye. This can suppress laser false detections and reduce the impact of the laser falsely detecting dust, rain, and fog as impassable.

[0195] 4. This invention combines fisheye traversable space detection to maintain occupancy probabilities for cells within blind spots that are detected as impassable by the fisheye. This prevents obstruction cells from entering the blind spot from being deemed traversable simply because the occupancy probability decreases rapidly after laser detection ceases, thus ensuring the safety of autonomous driving.

[0196] 5. The output of the present invention is shaped grid coordinates, rather than directly transmitting the floating-point position information of the grid points, which can save transmission bandwidth during transmission.

[0197] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the scope of the present invention. Other modifications or equivalent substitutions made to the technical solutions of the present invention by ordinary technicians in this field should be included in the scope of the claims of the present invention as long as they do not depart from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for generating an occupancy grid map by integrating fisheye traversable space detection, characterized in that: The method is used for a vehicle equipped with a laser radar and a fisheye camera, and includes the following steps: Establish a benchmark occupancy grid map by combining lidar blind spot information; Get LiDAR traversable space detection and fisheye traversable space detection as input data; Based on the timestamp of the received lidar traversable space detection in the current frame and the reference occupancy grid map, an occupancy grid map at the current moment is established; According to the motion information of the ego vehicle, the occupancy grid map information of the previous moment is converted into the occupancy grid map of the current moment; Subtract a priori value from the occupancy probability of all grids in the current occupancy grid map, so that the occupancy probability of grids that have not been updated by the lidar traversable space detection can be reduced; The current frame lidar passable space detection is projected onto the occupied grid map at the current moment. For the grid clicked by the laser detection, a fixed value is added to its occupancy probability. The grid with the laser projection is then back-projected onto the fisheye image to obtain the corresponding pixel coordinates, which are compared with the fisheye passable space boundary. If the grid pixel coordinates are below the fisheye passable space detection boundary, that is, the fisheye passable space detection considers the grid passable, then the grid occupancy probability is subtracted by a fixed value to reduce the grid occupancy probability. If the grid pixel coordinates are above the fisheye passable space detection boundary, that is, the fisheye passable space detection considers the grid impassable, then the grid occupancy probability is not processed, thereby achieving the effect of fusing the fisheye passable space detection results and updating the grid occupancy probability. The grids in the lidar blind spot that have not been updated by the lidar traversable space detection are back-projected into the fisheye image to obtain the corresponding pixel coordinates, which are compared with the fisheye traversable space boundary. If the grid pixel coordinates are below the fisheye traversable space detection boundary, that is, the fisheye traversable space detection considers the grid traversable, then the grid occupancy probability is not processed; if the grid pixel coordinates are above the fisheye traversable space detection boundary, that is, the fisheye traversable space detection considers the grid impassable, then the prior probability of this grid subtracted when the global probability decreases is added back, so as to achieve the effect of maintaining the blind spot in combination with fisheye detection; The moving object is not expressed in the form of an occupied grid map, and the occupancy probability of the grid within the target outline is cleared by the received target tracking results; Extract the index of the grid whose occupancy probability is greater than the threshold in the current occupancy grid map and output it to downstream for use.

2. The method for generating an occupancy grid map integrating fisheye traversable space detection according to claim 1, characterized in that: The lidar traversable space detection is the point cloud information of stationary or moving obstacles around the vehicle at the current moment; The laser radar traversable space detection at this moment is , It consists of a series of points, each of which Contains location and type information ;in, is the point under the vehicle system axis, Axis and Axis coordinates, the vehicle coordinate system takes the center of the vehicle as the origin, and the forward direction is Axis positive direction, left is Axis positive direction, upward Axis positive direction, Indicates the type information of the point.

3. The method for generating an occupancy grid map integrating fisheye traversable space detection according to claim 1, characterized in that: Fisheye traversable space detection consists of a set of horizontally continuous pixel points on the image information; the pixel points represent the boundary of the traversable space on the image, the upper side of the boundary is the impassable area, and the lower side of the boundary is the traversable area; The detection of the boundary of the fisheye traversable space at a certain moment is expressed as: ， ， ； in, is the height of the fisheye image, that is, the number of pixels in the vertical direction of the fisheye image. is the width of the fisheye image, that is, the number of pixels in the horizontal direction of the fisheye image; is the horizontal coordinate of the detection point in the fisheye image pixel coordinate system, ranging from 1 to Continuous growth, that is, the boundary points of the traversable space on the fisheye image have a total of indivual; by As the index, you can query The vertical coordinate of the traversable boundary on the corresponding column ,Right now , .

4. The method for generating an occupancy grid map integrating fisheye traversable space detection according to claim 1, characterized in that: The grid map consists of several grids, each of which is a square. The physical size of the grid map is ,in, is the side length of the grid, representing the resolution of the grid, is the number of rows in the grid map, is the number of columns of the raster map, is the total number of grids in the grid map; The starting point of the grid map is the upper left corner of the grid map, that is, the grid at row 0 and column 0. , the origin of the vehicle is located at index The grid of Row, No. The grid of columns is represented as ,in, ; The properties of each grid are ,in, is the position of the grid under the vehicle system, express, The grid position under the vehicle system axis, Axis coordinates, is the probability that the grid is inaccessible, that is, the grid occupancy probability, It is the flag indicating whether the grid is in the blind area of ​​the laser radar. Indicates that the grid is in the blind area of ​​the lidar. Indicates that the grid is not in the blind area of ​​the lidar. Whether the grid occupancy probability is updated by the lidar point, Indicates whether the grid occupancy probability is maintained.

5. The method for generating an occupancy grid map integrating fisheye traversable space detection according to claim 4, characterized in that: The establishment of the baseline occupancy grid map includes the following steps: Initialize a OK, Base occupancy grid map for columns , traverse each grid and initialize the grid properties; for Middle Row, No. Column Grid Initialize the properties: grid Occupancy probability initialization, grid The occupation probability is set to the initial value , that is, grid Occupancy probability ; grid Position under the vehicle system Initialized as: ； ； Blind spot sign calculation, the blind spot is a fixed area around the vehicle, using closed area Indicates; if the grid Position under the vehicle system In the blind spot of the car If the grid is inside The blind spot flag is initialized to If the grid Position under the vehicle system Not in the blind spot of the vehicle If the grid is inside The blind spot flag is initialized to ; grid Other flags are initialized: ； ； in, Indicates whether the grid occupancy probability is updated by the lidar point. The default value of the base occupancy grid map is that all grid occupancy probabilities have not been updated by the lidar point. , Indicates whether the grid occupancy probability is maintained. The default value for the base grid occupancy map is that all grid occupancy probabilities are not maintained. .

6. The method for generating an occupancy grid map integrating fisheye traversable space detection according to claim 5, characterized in that: Establishing the current occupancy grid map includes the following steps: LiDAR traversable space detection input at the current moment The timestamp is , based on the base occupancy grid map Create a The new occupation grid map at all times , .

7. The method for generating an occupancy grid map integrating fisheye traversable space detection according to claim 6, characterized in that: The previous frame Occupancy grid map at each moment Information conversion to occupancy grid map The occupancy probability of the whole grid is reduced, including the following steps: according to The position of the vehicle in the world coordinate system at this moment and orientation angle ,as well as The position of the vehicle in the world coordinate system at this moment and orientation angle , calculated arrive The translation vector of the vehicle at time and the rotation matrix ; Rotation Matrix The expression is: ； in, for arrive The difference in orientation angle at each moment, ; Translation vector The expression is: ； According to the translation vector and the rotation matrix ,Will Always occupy the grid map Middle Row, No. Column Grid Convert the position to the current moment From the car system, the grid is obtained The position of the vehicle at the time , the expression is: ； ； in, For grid exist Always from the position under the vehicle; calculate Occupies the grid map at the current moment Index in , the expression is: ； ； in, is the horizontal index, is the horizontal index of the vehicle origin grid, is the vertical index, is the vertical index of the vehicle origin grid, is the side length of the grid, representing the resolution of the grid, Indicates that the result is rounded; Determine whether the index occupies the grid map In, that is, the index satisfies and ,in, is the number of rows in the grid map, is the number of columns of the raster map; If the index is in the occupied grid map Inside, then Occupies the grid map at the current moment The corresponding new grid for: ; like , then assign the original grid occupancy probability to the new grid, that is ;in, is the occupancy probability of the new grid, is the original grid occupancy probability; Once the conversion is complete, it will occupy the grid map The occupancy probability of all grids in minus the prior value .

8. The method for generating an occupancy grid map integrating fisheye traversable space detection according to claim 1, characterized in that: Update the grid map occupied at the current moment The grid occupancy probability of , including the updated grid occupancy probability of the lidar traversable space detection and the updated grid occupancy probability of the fisheye traversable space detection, is as follows: Traversal LiDAR traversable space detection at all times , calculate the Laser points Occupies the grid map at the current moment Index in , the expression is: ； ； in, For laser point In the car system Axis coordinates, For laser point In the car system Axis coordinates, is the horizontal index, is the horizontal index of the vehicle origin grid, is the vertical index, is the vertical index of the vehicle origin grid, is the side length of the grid, representing the resolution of the grid, Indicates that the result is rounded; Determine whether the index occupies the grid map Within the range, that is, the index satisfies and ;in, is the number of rows in the grid map, is the number of columns of the raster map; If the index is in the occupied grid map If the laser point occupies the grid map The corresponding grid in is: ; Update Raster Occupancy probability: If the grid update flag is for ,Right now , indicating that the grid has been updated by other laser points before and does not need to be updated again. Continue to process the lidar traversable space detection The next laser point; If the grid update flag is for ,Right now , indicating that the grid has not been updated by other laser points, and the grid occupancy probability needs to be updated. Then the grid occupancy probability Add a fixed value , representing the grid Since there is a laser detection point hit, the grid occupancy probability rise; Update the position information of the grid, that is ; and set the grid update flag position to ,Right now , indicating that the occupancy probability of the grid has been updated; A raster that will be updated by lidar traversable space detection , projected onto the fisheye image, and obtain the corresponding pixel points of the grid on the fisheye image The coordinates of ;in, is the horizontal coordinate of the pixel point in the fisheye image pixel coordinate system, is the ordinate of the pixel point in the fisheye image pixel coordinate system; pass The horizontal axis , the boundary detected from the fisheye traversable space at the current moment Get The vertical coordinate of the fisheye traversable space detection boundary ,Right now ;like , then the grid In the passable space of fisheye detection, that is, the fisheye passable space detection considers the grid Passable, at this time the grid occupancy probability Subtract a fixed value , reducing the grid occupancy probability; if , then the grid If the grid is not within the traversable space of the fisheye detection, the occupancy probability of the grid is not adjusted.

9. The method for generating an occupancy grid map integrating fisheye traversable space detection according to claim 1, characterized in that: Determining whether to maintain grid occupancy probability for grids that are not updated by the lidar in the lidar blind area includes the following steps: Traverse the grid map occupied at the current moment For all grids in Row, No. Column Grid ,if It is in the blind area of ​​the laser radar traversable space and has not been detected and updated by the laser radar traversable space, that is, for , for , and the grid occupancy probability has not been maintained, that is, for , then the grid Project it onto the fisheye image to get the corresponding pixel points of the grid on the fisheye image The coordinates of ;in, is the horizontal coordinate of the pixel point in the fisheye image pixel coordinate system, is the ordinate of the pixel point in the fisheye image pixel coordinate system; pass The horizontal axis , the boundary detected from the fisheye traversable space at the current moment Get The vertical coordinate of the fisheye traversable space detection boundary ,Right now ;like , then the grid If the grid is in the impassable space detected by the fisheye, the globally reduced prior probability is restored, that is, the current grid occupancy probability Plus ;like , then the grid Within the traversable space of fisheye detection, the occupancy probability of the grid is not maintained.

10. The method for generating an occupancy grid map integrating fisheye traversable space detection according to claim 1, characterized in that: Moving objects are not represented in the form of an occupied grid map. The grid occupancy probability within the target outline is cleared through the target tracking results and output to the downstream, including the following steps: Use the closed area to define the target's outline Indicates that the grid map occupied at the current moment is traversed For all grids in Row, No. Column Grid , if the grid Position under the vehicle system In target outline If the grid is within the range of Set as the initial value of the grid occupancy probability ,Right now ; Traverse the current moment's occupancy grid map For all grids in Row, No. Column Grid , if its occupation probability Greater than the output occupation probability threshold , then the index Put it into the output index collection; The index set obtained after the traversal is the occupancy grid map composed of high occupancy probability grids. The index set is sent to the downstream, and the downstream is based on the index. Combined grid resolution , restore the position of the occupied grid under the own vehicle system.

Images