A method and apparatus for detecting black obstacles, and related products

Abstract

This invention relates to a method and apparatus for detecting black obstacles, and related products, comprising: performing raster image processing on the obstacle detection area to generate an obstacle detection raster map; determining a quadrilateral region corresponding to the obstacle's environmental depth image; generating environmental point cloud data based on environmental perception data; mapping the environmental point cloud data onto the quadrilateral region, marking rasters with environmental point cloud data in the quadrilateral region as rasters in a first raster set, and marking rasters without environmental point cloud data as rasters in a second raster set; performing connected component detection on the second raster set, and marking elements in the second raster set that satisfy the connection constraint conditions as elements in a third raster set; extracting target obstacle point cloud data from the environmental point cloud data to generate a target obstacle set; and marking elements in the third raster set that satisfy the connection conditions between elements in the third raster set and target obstacles in the target obstacle set as elements in a first black obstacle raster set.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a method, apparatus, program product, medium, computer device, and mobile tool for detecting black obstacles. Background Technology

[0002] Currently, autonomous vehicles or robots primarily acquire information about surrounding obstacles using their own ranging sensors during operation, and then perform real-time path planning based on this obstacle information. Commonly used ranging sensors in autonomous driving include 3D structured light, 3D time-of-flight (TOF), single-line LiDAR, and ultrasonic sensors.

[0003] 3D structured light and 3D Time-of-Flight (TOF) calculate object distance by emitting laser light and receiving it, or by comparing the emitted and received patterns. However, black objects strongly absorb light, so most of the emitted laser light is absorbed, with only a small portion reflected back. The reflected laser power is insufficient for distance measurement, meaning 3D structured light and 3D TOF often fail to detect the distance to black objects. While single-line lidar and ultrasonic sensors can detect the distance to black objects, the limited point cloud prevents the detection of complete boundary information. When precise boundary information of black obstacles cannot be obtained, the path planning module may cross the boundary of the black object, potentially causing autonomous vehicles or robots to encounter them during operation, leading to safety issues. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing a method and apparatus for detecting black obstacles, as well as related products, to solve the problems existing in the use of lasers to detect black obstacles in the prior art.

[0005] To achieve the above objectives, the first aspect of the present invention provides a method for detecting black obstacles, the method comprising:

[0006] The obstacle detection area is processed into a raster image to generate an obstacle detection raster map;

[0007] Acquire an obstacle environment depth image, and determine the quadrilateral region within the obstacle detection grid corresponding to the obstacle environment depth image based on the camera's internal and external parameters;

[0008] Acquire environmental perception data and generate environmental point cloud data based on the environmental perception data;

[0009] The environmental point cloud data is mapped to a quadrilateral region. The grid cells with environmental point cloud data in the quadrilateral region are marked as grid cells in the first grid cell set, and the grid cells without environmental point cloud data are marked as grid cells in the second grid cell set.

[0010] Perform connectivity detection on the second grid set, and mark the elements in the second grid set that satisfy the connectivity constraints as elements in the third grid set;

[0011] The environmental point cloud data is processed using an obstacle detection algorithm to extract target obstacle point cloud data and generate a target obstacle set.

[0012] Elements in the third grid set that satisfy the connection condition between the elements in the third grid set and the target obstacles in the target obstacle set are marked as elements in the first black obstacle grid set.

[0013] Furthermore, determining the quadrilateral region within the obstacle detection grid corresponding to the obstacle environment depth image based on the camera's intrinsic and extrinsic parameters includes:

[0014] Determine the four edge points of the obstacle environment depth image;

[0015] Calculate the four projection points of the four edge points on the obstacle detection grid based on the camera's intrinsic and extrinsic parameters;

[0016] Connect the four projection points to form the quadrilateral region.

[0017] Furthermore, the step of performing connected component detection on the second grid set and marking elements in the second grid set that satisfy the connectivity constraints as elements in the third grid set includes:

[0018] Perform connected component detection on the second grid to generate a set of connected components;

[0019] Traverse the elements in the connected component set, and mark the elements in the connected component set whose raster area is greater than the first preset area and less than the second preset area as elements in the third raster set.

[0020] Furthermore, the step of extracting target obstacle point cloud data from the environmental point cloud data using an obstacle detection algorithm to generate a target obstacle set includes:

[0021] The environmental point cloud data is segmented to generate ground point cloud data and non-ground point cloud data;

[0022] The non-ground point cloud data is clustered to generate a set of undetermined target obstacles;

[0023] The target obstacle set is generated by extracting point cloud data of the target obstacle set by the target obstacle screening and target obstacle projection algorithm.

[0024] Further, the step of marking elements in the third grid set that satisfy the connection conditions between elements in the third grid set and target obstacles in the target obstacle set as elements in the first black obstacle grid set includes:

[0025] Calculate the first center coordinates of the target obstacle based on the target obstacle point cloud data;

[0026] The first center coordinates are mapped to the obstacle detection grid to generate the second center coordinates;

[0027] Using the second center coordinates as the base point, search for the edge grid of the target obstacle, and mark the edge grid as a grid in the target obstacle edge grid set;

[0028] Elements in the third grid set that are connected to any grid in the target obstacle edge grid set are marked as elements in the first black obstacle grid set.

[0029] Further, the step of searching for the edge grid of the target obstacle using the second center coordinates as the base point and marking the edge grid as a grid in the target obstacle edge grid set includes:

[0030] Using the second center coordinates as the base point, search for five edge grids of the target obstacle along the horizontal left, horizontal right, vertical down, lower left 45 degrees, and lower right 45 degrees directions, and mark the five edge grids as grids in the target obstacle edge grid set.

[0031] Furthermore, before searching for the edge grid of the target obstacle using the second center coordinates as the base point and marking the edge grid as a grid in the target obstacle edge grid set, the method further includes:

[0032] Determine whether the second center coordinates are within the quadrilateral region;

[0033] If so, search for the edge grid of the target obstacle using the second center coordinate as the base point, and mark the edge grid as a grid in the target obstacle edge grid set;

[0034] If not, return the first center coordinates of the target obstacle calculated based on the target obstacle point cloud data.

[0035] Furthermore, after marking the elements in the third grid set that satisfy the connection conditions between the elements in the third grid set and the target obstacles in the target obstacle set as elements in the first black obstacle grid set, the method further includes:

[0036] Grids in the first black obstacle grid set whose grid coordinates are not less than the second center coordinate and whose distance from the grid coordinates to the second center coordinates is less than a preset distance are selected and marked as grids in the second black obstacle grid set.

[0037] Furthermore, after filtering out the grid cells in the first black obstacle grid set whose grid coordinates are not less than the second center coordinate and whose distance from the grid coordinates to the second center coordinates is less than a preset distance value, and marking them as grid cells in the second black obstacle grid set, the method further includes: calculating the grid coordinates of the grid cells in the second black obstacle grid set that are symmetrical about the second center coordinate.

[0038] The grids that belong to the third grid set and are symmetrical about the second center coordinates to the grids in the second black obstacle grid set are marked as grids in the third black obstacle grid set.

[0039] Combine the second and third black obstacle grid sets to generate a fourth black obstacle grid set.

[0040] A second aspect of the present invention provides a black obstacle detection device, comprising:

[0041] The detection data processing module is used to perform raster image processing on the obstacle detection area to generate an obstacle detection raster map.

[0042] The data acquisition module is used to acquire depth images of the obstacle environment;

[0043] The detection data processing module is also used to determine the quadrilateral region in the obstacle detection grid corresponding to the obstacle environment depth image based on the internal and external parameters of the camera.

[0044] The data acquisition module is also used to acquire environmental perception data;

[0045] The detection data processing module is further configured to generate environmental point cloud data based on the environmental perception data; map the environmental point cloud data onto a quadrilateral region, where grids containing environmental point cloud data are marked as grids in a first grid set, and grids without environmental point cloud data are marked as grids in a second grid set; perform connected component detection on the second grid set, and mark elements in the second grid set that satisfy the connectivity constraints as elements in a third grid set; extract target obstacle point cloud data from the environmental point cloud data using an obstacle detection algorithm to generate a target obstacle set; and mark elements in the third grid set that satisfy the connection conditions between elements in the third grid set and target obstacles in the target obstacle set as elements in a first black obstacle grid set.

[0046] A third aspect of the present invention provides a computer program product that, when run on a computer, causes the computer to perform the black obstacle detection method as described in any of the first aspects.

[0047] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the first aspects.

[0048] A fifth aspect of the present invention provides a computer apparatus, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method according to any one of the first aspects.

[0049] A sixth aspect of the present invention provides a mobile tool, characterized in that it includes the computer device described in the fifth aspect.

[0050] The black obstacle detection method, device, and related products provided by this invention detect areas without environmental point cloud data, perform connected component detection and filtering on elements in the areas, and determine the connection conditions between the filtered elements and existing obstacles to achieve the detection of target black obstacles. Attached Figure Description

[0051] Figure 1 This is one of the flowcharts for the black obstacle detection method provided in Embodiment 1 of the present invention;

[0052] Figure 2 This is a schematic diagram of the gridded image of the obstacle detection area provided in Embodiment 1 of the present invention;

[0053] Figure 3 This is the second flowchart of the black obstacle detection method provided in Embodiment 1 of the present invention;

[0054] Figure 4 This is the third flowchart of the black obstacle detection method provided in Embodiment 1 of the present invention;

[0055] Figure 5 This is the fourth flowchart of the black obstacle detection method provided in Embodiment 1 of the present invention;

[0056] Figure 6 This is a schematic diagram of the black obstacle detection device provided in Embodiment 2 of the present invention;

[0057] Figure 7 This is a schematic diagram of the computer program product structure provided in Embodiment 3 of the present invention;

[0058] Figure 8 This is a schematic diagram of the computer device structure provided in Embodiment 5 of the present invention. Detailed Implementation

[0059] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0060] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0061] The black obstacle detection method, medium, computer device, and mobile tool provided in this invention detect target black obstacles by detecting areas without environmental point cloud data, performing connected component detection and filtering on elements in the areas, and determining the connection conditions between the filtered elements and existing obstacles. When determining the connection conditions between elements in the candidate black obstacle grid set and existing obstacles, the connection relationship between the filtered elements and the edge grids found by searching the center coordinates of existing obstacles is used. Connection judgment is made through the key grids of existing obstacles, improving the efficiency of black obstacle detection. Misjudged black obstacle grids in the first black obstacle grid set are removed through distance relationships. Grids symmetrical to those in the second black obstacle grid set are calculated through symmetry relationships, recovering some lost black obstacle grids. The accuracy of black obstacle detection is improved by judging distance and symmetry relationships.

[0062] Example 1

[0063] Figure 1 This is one of the flowcharts for a black obstacle detection method provided in Embodiment 1 of the present invention. The executing entity of this application is a terminal, server, or processor with computing capabilities. By way of example and not limitation, this application illustrates the application of this method to an autonomous driving robot (hereinafter referred to as a robot). When this method is applied to a robot, the executing entity is the robot's automatic control unit. Figure 1 As shown, the black obstacle detection method includes:

[0064] Step 110: Perform raster image processing on the obstacle detection area to generate an obstacle detection raster map.

[0065] The obstacle detection region can be defined as an area extending outwards from the robot's center; this region can be any regular or irregular shape. When the obstacle detection region is rasterized, the length and width of the grid can be customized. In one possible implementation, Figure 2 This is a schematic diagram of the gridded image of the obstacle detection area provided in Embodiment 1 of the present invention, as shown below. Figure 2As shown in (a) and (b), Figure 2 (b) is Figure 2 (a) is an enlarged view of the large grid 104. In the robot coordinate system 100, with the robot 101 as the center, the obstacle detection area 102 is set with H meters in front and behind and W meters to the left and right. The obstacle detection area 102 is rasterized with small grids 103 with a length and width of a meters. Each large grid 104 is divided into multiple small grids 103 with a length and width of a meters. Each small grid 103 represents a pixel of the image. The obstacle detection area 102 can be converted into an image with the number of rows R = 2 * H / a and the number of columns C = 2 * W / a. The conversion relationship between the coordinates of the obstacle detection area and the coordinates of the obstacle detection raster is shown in formula (1):

[0066] row = R / 2 - (x / a)

[0067] col=C / 2-(y / a) (1)

[0068] Where x is the x-coordinate of the object point in robot coordinate system 100, y is the y-coordinate of the object point in robot coordinate system 100, row is the row coordinate of the object point in the raster image coordinate system, and col is the column coordinate of the object point in the raster image coordinate system. The origin of the robot coordinate system is at the center of the robot, and the origin of the raster image coordinate system 105 is at the extreme position of the upper left corner of the obstacle detection area.

[0069] If we take H = 5 meters, W = 3 meters, and a = 0.025 meters, according to the number of rows R = 2 * H / a and the number of columns C = 2 * W / a, we can obtain a raster image of (2 * 5 / 0.025) * (2 * 3 / 0.025) = 400 * 240. The unit of the raster image is pixels.

[0070] Step 120: Obtain the obstacle environment depth image and determine the quadrilateral region in the obstacle detection grid corresponding to the obstacle environment depth image based on the internal and external parameters of the camera.

[0071] Depth cameras are not affected by the object's own conditions, such as reflectivity, and can image any scene within the shooting field of view. Therefore, the range of obstacles can be determined first through depth images.

[0072] In one possible implementation, step 120 includes:

[0073] S1 determines the four edge points of the obstacle environment depth image.

[0074] S2, calculate the four projection points of the four edge points on the obstacle detection grid based on the camera's intrinsic and extrinsic parameters.

[0075] S3 connects four projection points to form a quadrilateral region.

[0076] Specifically, obstacle environment depth images can be captured by depth cameras, which are mounted on the robot body. There can be multiple depth cameras with multiple perspectives from the front, back, left, and right of the robot. Taking the robot's front-view depth camera as an example, the four edge points of the obstacle environment depth image represent the front-view scene range that the depth camera can capture, including the front-view width, height range, and front-view depth range.

[0077] Based on the camera imaging principle, the three-dimensional coordinates of the four edge points of the obstacle environment depth image in the camera coordinate system are calculated using the camera intrinsic parameters. Based on the camera extrinsic parameters, the three-dimensional coordinates of the camera coordinate origin and the four edge points are mapped to five three-dimensional coordinates in the robot coordinate system. The camera coordinate origin is connected to the four edge points respectively and extended to obtain four rays. The intersection of the rays with the obstacle detection area plane is the four projection points of the four edge points on the obstacle detection area. Then, based on the mapping relationship between the robot coordinate system and the grid image coordinate system, the coordinates of the four projection points of the four edge points on the obstacle detection grid are generated.

[0078] In a specific example, suppose the width of the obstacle environment depth image is dw, the height is dh, and the coordinates of the four edge points are P. lu (0,0), P ru (0,0), P rd (0,0), P ld (0,0), based on the camera imaging principle, the three-dimensional coordinates of the four edge points at a distance of L meters from the camera in the camera coordinate system can be reversibly derived, denoted as P. lu_c P ru_c P rd_c P ld_c Let the coordinates of the camera origin be O. c Connecting the camera's origin with the four edge points in three-dimensional coordinates yields four rays, namely O... c P lu_c O c P ru_c O c P rd_c O c P ld_c Based on the pose relationship between the camera and the robot, the four rays can be transformed into the robot coordinate system and denoted as O. cw P lu_cw O cw P ru_cw O cw P rd_cw O cw P ld_cwThe coordinates of the intersection points are obtained by jointly solving the ray equation and the ground equation, where the ground equation is the ground equation of the obstacle detection area. Let the coordinates of the intersection points corresponding to the four rays be P... lu_g P ru_g P rd_g P ld_g Quadrilateral P lu_g P ru_g P rd_g P ld_g The resulting region is the projection area of ​​the obstacle environment depth image onto the obstacle detection area. Then, based on the mapping relationship between the robot coordinate system and the grid image coordinate system, the quadrilateral region of the obstacle environment depth image within the obstacle detection grid is obtained. For example... Figure 2 As shown, this includes a quadrilateral region 106 of the obstacle environment depth image mapped to the obstacle detection raster.

[0079] Step 130: Acquire environmental perception data and generate environmental point cloud data based on the environmental perception data.

[0080] Environmental perception data is obtained by measuring distance values ​​detected by different sensors. Point cloud data of corresponding environmental objects in the sensor coordinate system is obtained according to the detection principles of different sensors. Environmental point cloud data detected by multiple sensors can be calculated based on the pose relationship between the sensors and the robot.

[0081] Step 140: Map the environmental point cloud data to a quadrilateral region. Grids with environmental point cloud data within the quadrilateral region are marked as grids in the first grid set, and grids without environmental point cloud data are marked as grids in the second grid set.

[0082] Based on the mapping relationship between the robot coordinate system and the grid image coordinate system, the environmental point cloud data is mapped to a quadrilateral region. Grids within the quadrilateral region containing environmental point cloud data represent grids where the sensor can receive reflected light from obstacles, while grids without environmental point cloud data represent grids where the sensor cannot receive reflected light. The inability to receive reflected light includes situations where there are low-reflectivity obstacles or occluded areas within the quadrilateral region. Low-reflectivity obstacles include objects with black surfaces. Black surfaces have a high light absorption rate, often resulting in invalid point clouds. For ease of description, low-reflectivity obstacles in this embodiment are all referred to as black obstacles. The scheme can be used to detect all low-reflectivity obstacles, not limited to black obstacles. In one possible implementation, grids in the first grid set and grids in the second grid set are marked with binary 0s and 1s, respectively.

[0083] Step 150: Perform connectivity detection on the second grid set, and mark the elements in the second grid set that satisfy the connectivity constraints as elements in the third grid set.

[0084] For each grid cell in the second grid set, perform connectivity detection, mark each group of connected grid cells as an element, and generate a connected component set. One element may include one or more grid cells. Mark the elements in the second grid set that satisfy the connectivity constraints as the third grid set. The third grid set is the candidate black obstacle grid set.

[0085] In one alternative approach, elements in the second raster set that satisfy the connectivity constraint are marked as elements in the third raster set, including:

[0086] Traverse the elements in the connected component set, and mark the elements in the connected component set whose raster area is greater than the first preset area and less than the second preset area as elements in the third raster set.

[0087] Specifically, when the grid area of ​​an element in the connected component set is not between the first and second preset areas, it indicates that the element is not an obstacle, or does not belong to a black obstacle, or is caused by factors such as noise or false detection.

[0088] Step 160: Extract target obstacle point cloud data from environmental point cloud data using an obstacle detection algorithm to generate a target obstacle set.

[0089] The specific steps include: segmenting the environmental point cloud data to generate ground point cloud data and non-ground point cloud data.

[0090] Non-ground point cloud data is used to cluster obstacles using a clustering algorithm to generate a set of undetermined target obstacles.

[0091] The target obstacle point cloud data is extracted from the target obstacle set by the target obstacle screening and target obstacle projection algorithm to generate the target obstacle set around the robot. Each connected component, i.e. each element, in the target obstacle set represents a target obstacle. The target obstacles in the target obstacle set include incomplete target obstacles with black parts.

[0092] Step 170: Mark the elements in the third grid set that satisfy the connection condition between the elements in the third grid set and the target obstacles in the target obstacle set as elements in the first black obstacle grid set.

[0093] In one alternative approach, step 170 includes:

[0094] The point cloud data of one target obstacle from the target obstacle set is selected and mapped onto the obstacle detection grid map, marked as the target obstacle grid set to be matched. It is then determined whether each element in the third grid set is connected to any grid in the target obstacle grid set to be matched, and the connected elements are marked as elements in the first black obstacle grid set. Similarly, for each target obstacle in the target obstacle set, elements in the third grid set that match each target obstacle in the target obstacle set are found and marked as elements in the first black obstacle grid set. All elements in the first black obstacle grid set constitute the first black obstacle grid set, which represents the mapping of all black obstacles on the obstacle detection grid map. The grid coordinates of the elements in the first black obstacle grid set are inversely transformed to the robot coordinate system and merged with the elements in the target obstacle set to generate a new obstacle boundary, which can be used for path planning in the control module.

[0095] In another alternative solution, Figure 3 This is a second flowchart of the black obstacle detection method provided in Embodiment 1 of the present invention, as follows: Figure 3 As shown, step 170 includes:

[0096] Step 210: Calculate the first center coordinates of the target obstacle based on the target obstacle point cloud data.

[0097] Specifically, the point cloud data of one target obstacle in the target obstacle set is selected to calculate the first center coordinate of the target obstacle in the robot coordinate system.

[0098] Step 220: Map the first center coordinates to the obstacle detection grid to generate the second center coordinates.

[0099] Specifically, based on the mapping relationship between the robot coordinate system and the grid image coordinate system, the first center coordinates are mapped to the obstacle detection grid image to generate the second center coordinates, which are grid pixel coordinates.

[0100] Step 230: Search for the edge grid of the target obstacle using the second center coordinates as the base point, and mark the edge grid as a grid in the target obstacle edge grid set.

[0101] In one possible implementation, using the second center coordinates as a base point, five edge grids along the horizontal left, horizontal right, vertical downward, lower left 45 degrees, and lower right 45 degrees directions of the target obstacle are searched, and these five edge grids are marked as the grids in the target obstacle edge grid set. Since the obstacle point cloud along the horizontal upward direction of the target obstacle is in the direction away from the ranging sensor in the actual scene, the point cloud data may be occluded and unable to perform grid mapping on the top of the obstacle, so the search for edge grids upward is abandoned.

[0102] Step 240: Mark the elements in the third grid set that are connected to any grid in the target obstacle edge grid set as elements in the first black obstacle grid set.

[0103] Based on one possible implementation of step 230, in one possible implementation of step 240, five edge grids are searched using the second center coordinates as the base point. The connection relationship between the elements in the third grid set and the five edge grids is detected in turn. As long as the element is connected to any one of the five edge grids, the element in the third grid set is marked as an element in the first black obstacle grid set.

[0104] In steps 210-240, the center coordinates of the target obstacle are used to search for edge grids as key grids for connection judgment, which improves the marking efficiency of elements in the first black obstacle grid set.

[0105] In one alternative approach, prior to step 230, the following steps are also included:

[0106] Determine whether the coordinates of the second center are within the quadrilateral region.

[0107] If so, proceed to step 230.

[0108] If not, return to step 210.

[0109] Specifically, some elements in the target obstacle set may be oriented elements captured by cameras of other oriented depth cameras of the robot. Before step 230, by using the position of the second center coordinates, some elements that are not in the quadrilateral area of ​​the current environment depth image are first filtered out. This can reduce the number of target obstacle elements in the target obstacle set that are executed in steps 230 and 240, thereby improving detection efficiency.

[0110] In a preferred embodiment, Figure 4 This is the third flowchart of the black obstacle detection method provided in Embodiment 1 of the present invention, as follows: Figure 4 As shown, step 240 is followed by:

[0111] Step 310: Select grid cells in the first black obstacle grid set whose grid ordinate value is not less than the second center ordinate value and whose grid coordinate is less than the distance value between the grid coordinate and the second center coordinate is less than a preset distance value, and mark them as grid cells in the second black obstacle grid set.

[0112] The first black obstacle grid set contains one or more elements, and each element contains one or more grids. The first black obstacle grid set is filtered grid by grid. A grid is marked as a grid in the second black obstacle grid set if it simultaneously meets two conditions: its ordinate value is not less than the ordinate value of the second center and its distance from the second center is less than a preset distance value. Since there is no lateral obstruction, the distance relationship between the grid's x-coordinate and the second center coordinate is not considered here. The preset distance value can be set according to the actual size range of the black obstacles to be detected.

[0113] In a further optimized solution, Figure 5 This is the fourth flowchart of the black obstacle detection method provided in Embodiment 1 of the present invention, as follows: Figure 5 As shown, step 310 is followed by:

[0114] Step 410: Calculate the grid coordinates symmetric about the second center coordinates in the second black obstacle grid set.

[0115] Step 420: Mark the grids that belong to the third grid set and are symmetrical about the second center coordinates to the grids in the second black obstacle grid set as grids in the third black obstacle grid set.

[0116] Specifically, the grids in the second black obstacle grid set are connected to the second center grid and extended to symmetrical positions to generate grid coordinates symmetrical to the coordinates of the second center. In the grid image, the coordinates of each grid are the pixel coordinates of the image.

[0117] Step 430: Combine the second black obstacle grid set and the third black obstacle grid set to generate the fourth black obstacle grid set.

[0118] The grids in the fourth black obstacle grid set are the final detected black obstacle grids. Then, based on the inverse transformation relationship between the robot coordinate system and the grid image coordinate system, the coordinates of the black obstacles in the robot coordinate system are calculated. The inverse transformation relationship between the robot coordinate system and the grid image coordinate system is shown in formula (2):

[0119] x = (R / 2 - row) * a

[0120] y=(C / 2-col)*a (2)

[0121] Where x is the x-coordinate of the object point in the robot coordinate system, y is the y-coordinate of the object point in the robot coordinate system, row is the row coordinate of the object point in the raster image coordinate system, and col is the column coordinate of the object point in the raster image coordinate system. The origin of the robot coordinate system is at the center of the robot, and the origin of the raster image coordinate system is at the extreme position of the upper left corner of the obstacle detection area.

[0122] During the filtering of elements in the third grid set in steps 210-240, it is possible to misjudge and delete some grids that do not belong to black obstacles or to mistakenly delete some grids that do belong to black obstacles. Step 310 can remove some grids that were mistakenly identified as not belonging to black obstacles. Steps 410-430 can recover some of the missing grids, thereby improving the detection accuracy of black obstacles.

[0123] For the point cloud data of each target obstacle in the target obstacle set, steps 210-240, 310, and 410-430 are performed. All elements in the annotated fourth black obstacle grid set constitute the fourth black obstacle grid set. The elements in this fourth black obstacle grid set represent the mapping of all black obstacles onto the obstacle detection grid map. The grid coordinates of the elements in the fourth black obstacle grid set are inversely transformed to the robot coordinate system and merged with the elements in the target obstacle set to generate a new obstacle boundary. This new obstacle boundary is the obstacle boundary within the current depth camera's field of view.

[0124] Example 2

[0125] Embodiment 2 of the present invention provides a black obstacle detection device. Figure 6 This is a schematic diagram of the structure of the black obstacle detection device provided in Embodiment 2 of the present invention. The black obstacle detection device 300 includes: a detection data processing module 301 and a data acquisition module 302.

[0126] The detection data processing module 301 is used to perform raster image processing on the obstacle detection area to generate an obstacle detection raster map.

[0127] The data acquisition module 302 is used to acquire depth images of the obstacle environment.

[0128] The detection data processing module 301 is also used to determine the quadrilateral region in the obstacle detection grid corresponding to the obstacle environment depth image based on the internal and external parameters of the camera.

[0129] The data acquisition module 302 is also used to acquire environmental perception data.

[0130] The detection data processing module 301 is also used to generate environmental point cloud data based on environmental perception data. The environmental point cloud data is mapped onto a quadrilateral region. Grids within the quadrilateral region containing environmental point cloud data are marked as grids in a first grid set, and grids without environmental point cloud data are marked as grids in a second grid set. Connectivity detection is performed on the second grid set, and elements in the second grid set that satisfy the connectivity constraints are marked as elements in a third grid set. A target obstacle set is generated by extracting target obstacle point cloud data from the environmental point cloud data using an obstacle detection algorithm. Elements in the third grid set that satisfy the connection conditions between elements in the third grid set and target obstacles in the target obstacle set are marked as elements in a first black obstacle grid set.

[0131] The detection data processing module 301 determines the quadrilateral region in the obstacle detection grid corresponding to the obstacle environment depth image based on the camera's internal and external parameters, including: determining the four edge points of the obstacle environment depth image; calculating the four projection points of the four edge points on the obstacle detection grid based on the camera's internal and external parameters; and connecting the four projection points to form a quadrilateral region.

[0132] The detection data processing module 301 performs connected component detection on the second grid set and marks the elements in the second grid set that satisfy the connected constraint conditions as elements in the third grid set. This includes: performing connected component detection on the second grid to generate a connected component set; traversing the elements in the connected component set and marking the elements in the connected component set whose grid area is greater than a first preset area and less than a second preset area as elements in the third grid set.

[0133] The detection data processing module 301 uses an obstacle detection algorithm to extract target obstacle point cloud data from environmental point cloud data to generate a target obstacle set. This includes: segmenting the environmental point cloud data to generate ground point cloud data and non-ground point cloud data; clustering the non-ground point cloud data to generate a target obstacle set to be determined; and extracting target obstacle point cloud data from the target obstacle set to be determined using a target obstacle screening and projection algorithm to generate the target obstacle set.

[0134] The detection data processing module 301 marks elements in the third grid set that satisfy the connection conditions between elements in the third grid set and target obstacles in the target obstacle set as elements in the first black obstacle grid set. This includes: calculating the first center coordinates of the target obstacle based on the target obstacle point cloud data; mapping the first center coordinates to the obstacle detection grid map to generate the second center coordinates; searching for the edge grids of the target obstacle using the second center coordinates as the base point and marking the edge grids as grids in the target obstacle edge grid set; and marking elements in the third grid set that are connected to any grid in the target obstacle edge grid set as the first black obstacle grid set.

[0135] The detection data processing module 301 searches for the edge grids of the target obstacle using the second center coordinates as the base point and marks the edge grids as grids in the target obstacle edge grid set. This includes searching for five edge grids of the target obstacle along the horizontal left, horizontal right, vertical down, lower left 45 degrees, and lower right 45 degrees directions using the second center coordinates as the base point and marking the five edge grids as grids in the target obstacle edge grid set.

[0136] Before searching for the edge grid of the target obstacle using the second center coordinate as the base point and marking the edge grid as a grid in the target obstacle edge grid set, the detection data processing module 301 also determines whether the second center coordinate is within the quadrilateral area; if so, it searches for the edge grid of the target obstacle using the second center coordinate as the base point and marks the edge grid as a grid in the target obstacle edge grid set; if not, it returns the first center coordinate of the target obstacle calculated based on the target obstacle point cloud data.

[0137] After marking the elements in the third grid set that meet the connection conditions between the elements in the third grid set and the target obstacles in the target obstacle set as elements in the first black obstacle grid set, the detection data processing module 301 further filters the grids in the first black obstacle grid set whose grid ordinate values ​​are not less than the second center ordinate and whose grid coordinates are less than a preset distance value, and marks them as grids in the second black obstacle grid set.

[0138] After filtering out grids in the first black obstacle grid set whose ordinate values ​​are not less than the second center ordinate and whose distance from the second center ordinate is less than a preset distance, and marking them as grids in the second black obstacle grid set, the detection data processing module 301 further calculates the grid coordinates symmetrical about the second center ordinate in the second black obstacle grid set; marks grids belonging to the third grid set that are symmetrical about the second center ordinate in the second black obstacle grid set as grids in the third black obstacle grid set; and combines the second and third black obstacle grid sets to generate a fourth black obstacle grid set.

[0139] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, the detection data processing module can be a separate processing element, or it can be integrated into a chip in the above device. Alternatively, it can be stored as program code in the memory of the above device, and called and executed by a processing element of the device. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0140] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). As another example, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a System-on-a-Chip (SOC).

[0141] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The aforementioned computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The aforementioned computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the aforementioned computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, Bluetooth, microwave, etc.) means. The aforementioned computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0142] Example 3

[0143] Example 3 provides a computer program product containing instructions, such as... Figure 7 As shown, when the computer program product is run on a computer, the computer performs any of the black obstacle detection methods provided in Embodiment 1.

[0144] Example 4

[0145] Embodiment 4 of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the black obstacle detection methods provided in Embodiment 1.

[0146] Example 5

[0147] Embodiment 5 of the present invention provides a computer device, Figure 8 This is a schematic diagram of the computer device structure provided in Embodiment 5 of the present invention. Figure 7 As shown, it includes a memory 400, a processor 500, and a computer program stored in the memory. The processor executes the computer program to implement any of the black obstacle detection methods provided in Embodiment 1 above.

[0148] Example 6

[0149] Embodiment 6 of the present invention provides a mobile tool, including the computer device described in Embodiment 5 above.

[0150] The mobile equipment referred to in this application can be vehicle equipment or robotic equipment with the following various functions:

[0151] (1) Passenger transport function, such as family cars, buses, etc.;

[0152] (2) Cargo carrying function, such as ordinary trucks, box trucks, trailers, enclosed trucks, tank trucks, flatbed trucks, container trucks, dump trucks, special structure trucks, etc.

[0153] (3) Tool functions, such as logistics delivery vehicles, automated guided vehicles (AGVs), patrol vehicles, cranes, hoists, excavators, bulldozers, loaders, road rollers, loaders, off-road engineering vehicles, armored engineering vehicles, sewage treatment vehicles, sanitation vehicles, vacuum trucks, floor scrubbers, water sprinkler trucks, sweeping robots, food delivery robots, shopping guide robots, lawnmowers, golf carts, etc.

[0154] (4) Entertainment functions, such as recreational vehicles, amusement park equipment, balance bikes, etc.;

[0155] (5) Special rescue functions, such as fire trucks, ambulances, power repair vehicles, and engineering emergency rescue vehicles.

[0156] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0157] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0158] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting black obstacles, characterized in that, The method includes: The obstacle detection area is processed into a raster image to generate an obstacle detection raster map; Acquire an obstacle environment depth image, and determine the quadrilateral region within the obstacle detection grid corresponding to the obstacle environment depth image based on the camera's internal and external parameters; Acquire environmental perception data and generate environmental point cloud data based on the environmental perception data; The environmental point cloud data is mapped to a quadrilateral region. The grid cells with environmental point cloud data in the quadrilateral region are marked as grid cells in the first grid cell set, and the grid cells without environmental point cloud data are marked as grid cells in the second grid cell set. Perform connectivity detection on the second grid set, and mark the elements in the second grid set that satisfy the connectivity constraints as elements in the third grid set; The environmental point cloud data is processed using an obstacle detection algorithm to extract target obstacle point cloud data and generate a target obstacle set. Elements in the third grid set that satisfy the connection condition between the elements in the third grid set and the target obstacles in the target obstacle set are marked as elements in the first black obstacle grid set.

2. The method for detecting black obstacles according to claim 1, characterized in that, The step of determining the quadrilateral region within the obstacle detection grid corresponding to the obstacle environment depth image based on the camera's intrinsic and extrinsic parameters includes: Determine the four edge points of the obstacle environment depth image; Calculate the four projection points of the four edge points on the obstacle detection grid based on the camera's intrinsic and extrinsic parameters; Connect the four projection points to form the quadrilateral region.

3. The method for detecting black obstacles according to claim 1, characterized in that, The step of performing connected component detection on the second grid set and marking elements in the second grid set that satisfy the connectivity constraints as elements in the third grid set includes: Perform connected component detection on the second grid to generate a set of connected components; Traverse the elements in the connected component set, and mark the elements in the connected component set whose raster area is greater than the first preset area and less than the second preset area as elements in the third raster set.

4. The method for detecting black obstacles according to claim 1, characterized in that, The step of extracting target obstacle point cloud data from the environmental point cloud data using an obstacle detection algorithm to generate a target obstacle set includes: The environmental point cloud data is segmented to generate ground point cloud data and non-ground point cloud data; The non-ground point cloud data is clustered to generate a set of undetermined target obstacles; The target obstacle set is generated by extracting point cloud data of the target obstacle set by the target obstacle screening and target obstacle projection algorithm.

5. The method for detecting black obstacles according to claim 1, characterized in that, The step of marking elements in the third grid set that satisfy the connection conditions between elements in the third grid set and target obstacles in the target obstacle set as elements in the first black obstacle grid set includes: Calculate the first center coordinates of the target obstacle based on the target obstacle point cloud data; The first center coordinates are mapped to the obstacle detection grid to generate the second center coordinates; Using the second center coordinates as the base point, search for the edge grid of the target obstacle, and mark the edge grid as a grid in the target obstacle edge grid set; Elements in the third grid set that are connected to any grid in the target obstacle edge grid set are marked as elements in the first black obstacle grid set.

6. The method for detecting black obstacles according to claim 5, characterized in that, The step of searching for the edge grid of the target obstacle using the second center coordinates as the base point and marking the edge grid as a grid in the target obstacle edge grid set includes: Using the second center coordinates as the base point, search for five edge grids of the target obstacle along the horizontal left, horizontal right, vertical down, lower left 45 degrees, and lower right 45 degrees directions, and mark the five edge grids as grids in the target obstacle edge grid set.

7. The method for detecting black obstacles according to claim 5, characterized in that, Before searching for the edge grid of the target obstacle using the second center coordinates as the base point and marking the edge grid as a grid in the target obstacle edge grid set, the method further includes: Determine whether the second center coordinates are within the quadrilateral region; If so, search for the edge grid of the target obstacle using the second center coordinate as the base point, and mark the edge grid as a grid in the target obstacle edge grid set; If not, return the first center coordinates of the target obstacle calculated based on the target obstacle point cloud data.

8. The method for detecting black obstacles according to claim 5, characterized in that, After marking the elements in the third grid set that satisfy the connection conditions between the elements in the third grid set and the target obstacles in the target obstacle set as elements in the first black obstacle grid set, the method further includes: Grids in the first black obstacle grid set whose grid coordinates are not less than the second center coordinate and whose distance from the grid coordinates to the second center coordinates is less than a preset distance are selected and marked as grids in the second black obstacle grid set.

9. The method for detecting black obstacles according to claim 8, characterized in that, After selecting grid cells in the first black obstacle grid set whose grid coordinates are not less than the second center grid coordinate and whose distance from the second center grid coordinate is less than a preset distance, and marking them as grid cells in the second black obstacle grid set, the method further includes: Calculate the grid coordinates symmetric about the second center coordinates in the second black obstacle grid set; The grids that belong to the third grid set and are symmetrical about the second center coordinates to the grids in the second black obstacle grid set are marked as grids in the third black obstacle grid set. Combine the second and third black obstacle grid sets to generate a fourth black obstacle grid set.

10. A black obstacle detection device, characterized in that, The device includes: The detection data processing module is used to perform raster image processing on the obstacle detection area to generate an obstacle detection raster map. The data acquisition module is used to acquire depth images of the obstacle environment; The detection data processing module is also used to determine the quadrilateral region in the obstacle detection grid corresponding to the obstacle environment depth image based on the internal and external parameters of the camera. The data acquisition module is also used to acquire environmental perception data; The detection data processing module is further configured to generate environmental point cloud data based on the environmental perception data; map the environmental point cloud data onto a quadrilateral region, where grids containing environmental point cloud data are marked as grids in a first grid set, and grids without environmental point cloud data are marked as grids in a second grid set; perform connected component detection on the second grid set, and mark elements in the second grid set that satisfy the connectivity constraints as elements in a third grid set; extract target obstacle point cloud data from the environmental point cloud data using an obstacle detection algorithm to generate a target obstacle set; and mark elements in the third grid set that satisfy the connection conditions between elements in the third grid set and target obstacles in the target obstacle set as elements in a first black obstacle grid set.

11. A computer program product, characterized in that, When the computer program product is run on a computer, the computer performs the black obstacle detection method as described in any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the black obstacle detection method as described in claims 1-9.

13. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the black obstacle detection method according to claims 1-9.

14. A mobile tool, characterized in that, Includes the computer device as described in claim 13.

Images