A grid manhole cover detection method, device, equipment and storage medium

Abstract

The present invention discloses a grid manhole cover detection method, device, equipment and storage medium. The method comprises: when a mobile robot moves, real-time acquisition of position information and point cloud data of the mobile robot is carried out, each point cloud data is segmented to obtain ground point cloud data and underground point cloud data, the position information is located in a navigation map stored in the mobile robot, and a region within a preset range centered on the position information is rasterized to obtain raster data, the ground point cloud data is projected onto the raster data to obtain a grid into which the ground point cloud data falls, and the grid is marked as a ground grid, the underground point cloud data is projected onto the raster data, if the grids into which each underground point cloud data falls are all ground grids, it is determined that a grid manhole cover exists in the region within the preset range, the mobile robot is instructed to move according to a previously planned navigation route of the navigation map, so as to realize real-time and rapid accurate judgment of the grid manhole covers on the road, and the mobile robot is instructed not to detour the grid manhole covers.

Description

Technical Field

[0001] The technical field of road condition identification of the present invention particularly relates to a grid manhole cover detection method, device, equipment and storage medium. Background Art

[0002] In the field of mobile robots and autonomous driving, road condition information perception is particularly important. Among them, the presence of sewer manhole covers on the road is relatively common, and most of the sewer manhole covers are grid manhole covers.

[0003] Under normal circumstances, when constructing manhole covers, in order to ensure road safety, the manhole covers will be flush with the road surface, and mobile robots can pass through the surface of the manhole covers normally. However, since mobile robots currently determine road conditions and plan navigation routes by collecting road images and laser point clouds, and then identify road conditions based on the collected information, this method can easily regard manhole covers as obstacles and judge them as impassable or detourable, especially grid manhole covers with multiple drainage holes are particularly prone to misjudgment.

[0004] Currently, in order to accurately perceive sewer manhole covers, the mainstream approach is to rely on prior maps, that is, to store road condition information in the map in advance. Figure 1 Generally, it is a two-dimensional vector map or a three-dimensional point cloud map. When making the map, the information of the manhole cover needs to be marked on the map, which is a lot of work. In addition, the prior maps are not timely and cannot cope with temporary changes, such as the removal of the manhole cover due to road construction. Summary of the invention

[0005] The present invention provides a grid manhole cover detection method, device, equipment and storage medium to solve the problem that it is difficult for a mobile robot to accurately identify grid manhole covers in a road. Without relying on a priori maps with road conditions marked in advance and image recognition technology, rapid and accurate judgment of grid manhole covers can be achieved.

[0006] According to one aspect of the present invention, a method for detecting a grid manhole cover is provided, the method being applied to a mobile robot, the method comprising:

[0007] During the movement of the mobile robot, the position information and point cloud data of the mobile robot are acquired in real time;

[0008] Segment each point cloud data to obtain ground point cloud data and underground point cloud data;

[0009] The position information is located in a navigation map stored in the mobile robot, and a region within a preset range centered on the position information is rasterized to obtain raster data, wherein the side length of each grid in the raster data is greater than a preset side length, and the preset side length is determined according to the size of the drainage hole of the grid manhole cover;

[0010] Projecting the ground point cloud data into the grid data to obtain the grid into which the ground point cloud data falls, and marking it as a ground grid;

[0011] The underground point cloud data are projected into the grid data. If the grids into which each underground point cloud data falls are the ground grids, it is determined that there is a grid manhole cover in the area of ​​the preset range of the mobile robot, and the mobile robot is instructed to move according to the previously planned navigation route of the navigation map.

[0012] According to one aspect of the present invention, a grid manhole cover detection device is provided, the device is applied to a mobile robot, and the device comprises:

[0013] An acquisition module, used to acquire the position information and point cloud data of the mobile robot in real time during the movement of the mobile robot;

[0014] A segmentation module is used to segment each point cloud data to obtain ground point cloud data and underground point cloud data;

[0015] A rasterization processing module, used for locating the position information in the navigation map stored in the mobile robot, and performing rasterization processing on an area within a preset range centered on the position information to obtain raster data, wherein the side length of each grid in the raster data is greater than a preset side length, and the preset side length is determined according to the size of the drainage hole of the grid manhole cover;

[0016] A ground grid marking module, used for projecting the ground point cloud data into the grid data to obtain the grid into which the ground point cloud data falls, and marking it as a ground grid;

[0017] A grid manhole cover determination module is used to project the underground point cloud data into the grid data. If the grids into which each underground point cloud data falls are the ground grids, it is determined that there is a grid manhole cover in the area of ​​the preset range of the mobile robot, and the mobile robot is instructed to move according to the previously planned navigation route of the navigation map.

[0018] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0019] at least one processor; and

[0020] a memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a grid manhole cover detection method described in any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a grid manhole cover detection method described in any embodiment of the present invention when executed.

[0023] The technical solution of the embodiment of the present invention provides a grid manhole cover detection method, which is applied to a mobile robot. During the movement of the mobile robot, the position information and point cloud data of the mobile robot are obtained in real time, each point cloud data is segmented to obtain ground point cloud data and underground point cloud data, the position information is located in a navigation map stored in the mobile robot, and an area within a preset range centered on the position information is rasterized to obtain raster data, wherein the side length of each grid in the raster data is greater than the preset side length, and the preset side length is determined according to the size of the drainage hole of the grid manhole cover, so that the grids at the corresponding positions of the drainage holes will also have ground The ground point cloud data is projected into the grid data to obtain the grid into which the ground point cloud data falls, which is marked as the ground grid, and the underground point cloud data is projected into the grid data. If the grids into which the underground point cloud data falls are all ground grids, it is determined that there is a grid manhole cover in the area within the preset range of the mobile robot. The embodiment of the present invention can detect the grid manhole covers on the road in real time without relying on a map that marks the road conditions in advance. The size of each grid is set by using the structural characteristics of the grid manhole cover, so that an accurate judgment can be made on the grid manhole covers on the road quickly. When there is a grid manhole cover in the moving route of the mobile robot, it can be indicated that there is no need to detour.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 is a flow chart of a grid manhole cover detection method provided according to Embodiment 1 of the present invention;

[0027] Figure 2 This is an initial schematic diagram of a rasterization process provided according to the first embodiment of the present invention;

[0028] Figure 3 is a schematic diagram of a grid manhole cover provided according to Embodiment 1 of the present invention;

[0029] Figure 4 is a schematic diagram of grid data projection of a grid manhole cover provided according to the first embodiment of the present invention;

[0030] Figure 5 is a schematic diagram of raster data of a sunken area provided according to the first embodiment of the present invention;

[0031] Figure 6 is a ground grid marking schematic diagram provided according to the first embodiment of the present invention;

[0032] Figure 7 is a structural schematic diagram of a grid manhole cover detection device provided according to Embodiment 2 of the present invention;

[0033] Figure 8 It is a structural schematic diagram of an electronic device for implementing a grid manhole cover detection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] Embodiment 1

[0037] Figure 1 A flowchart of a grid manhole cover detection method is provided for Embodiment 1 of the present invention, and the method is applied to a mobile robot.

[0038] At present, in order to solve the problem mentioned in the background technology that the pre-marking of special road conditions in the prior map requires a lot of manual work and cannot adapt well to the changes in the actual scene, the mobile robot's recognition of special road conditions on the road usually involves collecting images around the mobile robot during driving, comparing and identifying the collected images with pre-set images of special road conditions, and instructing the mobile robot when the image is identified to contain special road conditions, or sending corresponding driving instructions to the mobile robot according to the special road conditions, such as if there is a pothole ahead, the driving instruction can control the vehicle to slow down, turn right to the right lane, etc. This method requires training of pre-set special road condition images, and when comparing and identifying the images, it is necessary to extract features from the images, and then finally obtain the comparison results through a multi-layer neural network. The calculation process is complex and takes a long time.

[0039] In order to improve the efficiency of identifying road depressions, we can abandon the image recognition method and directly use point cloud data for judgment. However, when the mobile robot is walking and using multi-line laser radar to collect point cloud data, when it encounters a grid manhole cover, the laser radar will pass through the drainage area of ​​the grid manhole cover and illuminate below the grid manhole cover. Therefore, there will be point cloud data below the ground at the grid manhole cover. The mobile robot will regard it as a sunken obstacle and bypass it or trigger the anti-fall protection, resulting in the inability to pass normally.

[0040] A grid manhole cover detection method proposed in this embodiment directly processes point cloud data, eliminating the process of image processing, reducing the recognition time while ensuring the accuracy of recognition, so that when a mobile robot faces a grid manhole cover, it can pass smoothly even if the grid manhole cover has many drainage holes.

[0041] The method may be performed by a grid manhole cover detection device, which may be implemented in the form of hardware and / or software.

[0042] like Figure 1 As shown, the method comprises the following steps:

[0043] S110, during the movement of the mobile robot, obtaining the position information and point cloud data of the mobile robot in real time.

[0044] During the movement of the mobile robot, various types of sensors can be used to sense the external environment in real time to determine the position information and point cloud data of the mobile robot. For example, position information can be sensed by a position sensor (such as GPS) to determine the position information of the mobile robot in real time; laser radar can be configured in the mobile robot to collect point cloud data in real time. In the specific implementation, a multi-line laser radar can be used.

[0045] S120, segmenting each point cloud data to obtain ground point cloud data and underground point cloud data.

[0046] After the point cloud data is acquired in real time, the point cloud data can be segmented. After segmentation, it can be determined whether each point cloud data belongs to ground point cloud data or underground point cloud data.

[0047] When segmenting each point cloud data, in order to accurately determine whether it is ground point cloud data or underground point cloud data in each point cloud data, the point cloud data can be segmented on the ground. There are many ways to segment the ground, such as plane grid method, point cloud normal vector method, model fitting method, facet grid method, etc.

[0048] In one implementation, point cloud data obtained during the smooth movement of the mobile robot on the current road can be randomly sampled, and the reference height value of the ground point cloud can be determined based on the coordinates of the randomly sampled point cloud data. The reference height value can be used to set a height range, such as a ground point cloud height range and an underground point cloud height range, and the height value (i.e., the Z-axis value) in the coordinates of each point cloud data can be compared with the set height range to determine whether each point cloud data is ground point cloud data or underground point cloud data.

[0049] In one embodiment, S120 includes the following steps:

[0050] Perform plane fitting based on each point cloud data to obtain fitting ground points;

[0051] Determine the height of each point cloud data and the height of the fitted ground point, and compare the height of each point cloud data with the fitted ground point;

[0052] The point cloud data that is located below the fitted ground point and exceeds a certain distance threshold is determined as underground point cloud data, and the point cloud data that is located within a specified height distance range of the fitted ground point is determined as ground point cloud data.

[0053] The fitted ground points can be obtained by performing plane fitting on each point cloud data. Currently, the plane fitting method is relatively mature and will not be explained in detail here, and the specific plane fitting method is not limited.

[0054] After obtaining the fitted ground points on the plane through plane fitting, the height of each point cloud data can be compared with the fitted ground points. The point cloud coordinates of each point cloud data and the fitted ground points can be determined first, and their respective heights can be determined through the point cloud coordinates.

[0055] The point cloud data located below the fitting ground point and exceeding a certain distance threshold is determined as underground point cloud data, the point cloud data located within the specified height distance range of the fitting ground point is determined as ground point cloud data, and the point cloud data located above the fitting ground point and exceeding the specified height distance range is determined as above-ground point cloud data.

[0056] Through the ground segmentation operation, the ground segmentation results obtained may include ground point cloud data, underground point cloud data, and ground point cloud data. When inspecting the grid manhole cover, the mobile robot may misjudge it as a depression due to the existence of underground point cloud data. Therefore, the point cloud data determined as ground point cloud data in the ground segmentation results may be discarded. After the discarding, the remaining ground point cloud data and underground point cloud data.

[0057] S130, locate the position information in the navigation map stored in the mobile robot, and rasterize the area within a preset range centered on the position information to obtain raster data, wherein the side length of each grid in the raster data is greater than the preset side length, and the preset side length is determined according to the size of the drainage hole of the grid manhole cover.

[0058] During the movement of the mobile robot, various types of sensors in the mobile robot can sense the surrounding environment and combine the information to construct a dynamic map, and determine the navigation map used for navigation movement from the constructed dynamic map.

[0059] When rasterizing an area within a preset range centered on the position information, the resulting raster data can be located in a horizontal coordinate system (two-dimensional coordinate system). In the concept of analytic geometry, a plane can be infinitely extended. In an abstract plane (i.e., a horizontal coordinate system), only the area around the mobile robot, or even only the area within a certain angle range and a certain distance range in the direction of the mobile robot's advance, needs to be detected. Figure 2 An initial schematic diagram of rasterization processing is shown in FIG. The entire road plane area where the mobile robot is located can be regarded as the entire plane in the horizontal coordinate system, A is the mobile robot, and the area framed by B is regarded as the area of ​​a preset range centered on the position information of the mobile robot, and the raster data obtained after rasterization processing. For example, the area with a side length of 5 meters centered on the mobile robot is rasterized to obtain multiple grids with a side length of 0.05 meters, that is, grids.

[0060] When performing rasterization processing, the entire plane in the entire horizontal coordinate system can be first divided into multiple rectangular areas of the same size, each rectangular area is a grid, and the combination of all grids can be regarded as a grid. Of course, the grids and grids are not presented to the user as a visual effect of area division, but are only used to describe the process of grid manhole cover detection.

[0061] The size of the grid and the size of each grid can be determined based on information such as the type of mobile robot, driving speed, sensor category, navigation accuracy, usage scenario, etc., and are empirical values.

[0062] refer to Figure 3 A schematic diagram of a grid manhole cover. A grid manhole cover is a manhole cover that contains several drainage holes surrounded by crossbeams. It is a common sewer manhole cover on the road. In terms of the current mobile robot recognition technology for road conditions, compared with a manhole cover with fewer and smaller drainage holes, a grid manhole cover is more likely to be misjudged as a sunken obstacle by a mobile robot.

[0063] In this embodiment, what is detected is the grid manhole cover, and the side length of each grid in the grid data is greater than the preset side length, and the preset side length is determined according to the size of the drainage hole of the grid manhole cover.

[0064] For different types of grating manhole cover drainage holes, since it is necessary to ensure the efficiency of drainage while also trying to ensure the personal and property safety of pedestrians, the sizes of the grating manhole cover drainage holes have different size standards. The preset length can be set according to the size standard of the previous manhole cover drainage hole used in the section of road frequently traveled by the mobile robot.

[0065] The side length of each grid is greater than the preset side length, which means that when the grid manhole cover is matched with the grid data, for each grid, even if the grid completely includes the entire width of the drain hole, there will still be some positions that include the part of the beam on the ground that is used to surround the drain hole or other parts on the ground. This ensures that when there is a grid manhole cover, each grid will have ground point cloud data. The crossbeam of the grid manhole cover can be referred to Figure 3 a in.

[0066] In one embodiment, the preset side length is equal to the width of the drain hole of the grid manhole cover.

[0067] After determining the size standard of the grid manhole cover drainage hole used in the application scenario of the mobile robot, the side length of the grid can be set according to the width of the grid manhole cover drainage hole. Under the premise of ensuring that the grid manhole cover can not be misjudged as a sunken obstacle, the probability of other types of manhole covers being misjudged will be lower. The preset side length is set to be equal to the width of the grid manhole cover drainage hole, that is, the side length of each grid in the grid data is greater than the width of the grid manhole cover drainage hole. This can maximize the side length of each grid in the raster data as much as possible, so that the computational efficiency of the entire process will not be slowed down due to the small side length of the grid, and it can also ensure that when there is a grid manhole cover, each grid will have ground point cloud data, ensuring the accuracy of detecting the grid manhole cover.

[0068] S140, projecting the ground point cloud data into the grid data to obtain the grid into which the ground point cloud data falls, and marking it as the ground grid.

[0069] When the mobile robot moves according to the navigation map, the point cloud data collected by the laser radar will also change accordingly with the movement of the mobile robot. The position of each point cloud data on the navigation map, that is, the coordinates of the point cloud data, can be determined by determining the real-time posture (position and direction) of the mobile robot on the navigation map. After determining the coordinates of the point cloud data, the point cloud data can be projected into the raster data according to certain conversion rules. For example, the projection coordinates of the point cloud data projected into the raster data are calculated by using the method of calculating the projection point coordinates of the spatial point to the plane in spatial geometry. After determining the projection coordinates of the point cloud data projected into the raster data, they can be matched with the position of each grid in the raster data, and the grid into which the ground point cloud data falls is marked as the ground grid.

[0070] In one embodiment, the grid data is in a two-dimensional coordinate system, and projecting the ground point cloud data into the grid data in S140 includes:

[0071] Get the two-dimensional coordinate system where the raster data is located;

[0072] Obtain the horizontal axis coordinate value and the vertical axis coordinate value of the ground point cloud data, and mark the current point cloud data in the two-dimensional coordinate system according to the horizontal axis coordinate value and the vertical axis coordinate value.

[0073] Since the point cloud data is located in a three-dimensional coordinate system, and the grid data is located in a two-dimensional coordinate system, each point cloud data has three coordinate values ​​of x, y, and z. After obtaining the two-dimensional coordinate system where the grid data is located, the unit length of the horizontal coordinate and the vertical coordinate of the two-dimensional coordinate system can be determined first, and the horizontal coordinate and the vertical coordinate of the two-dimensional coordinate system can be directly projected to the two-dimensional coordinate system where the grid data is located along the direction of the z axis in the three-dimensional coordinate system, that is, the horizontal axis coordinate value x and the vertical axis coordinate value y are taken from the coordinates of the point cloud data, so as to determine that the coordinates of the point cloud data projected to the two-dimensional coordinate system where the grid data is located are (x, y). The coordinates when projected in the two-dimensional coordinate system can also be determined according to a certain coordinate conversion strategy. For example, the horizontal axis coordinate value and the vertical axis coordinate value of the point cloud data can be transformed according to a certain ratio. For example, when the horizontal axis coordinate value of the point cloud data is 5, the vertical axis coordinate value is 10, and the specified ratio is 5, when marking the current point cloud data in the two-dimensional coordinate system, the horizontal coordinate in the two-dimensional coordinate system can be 5÷5=1, and the vertical coordinate in the two-dimensional coordinate system can be 10÷5=2.

[0074] S150, projecting the underground point cloud data into the grid data. If the grids into which the underground point cloud data fall are all ground grids, it is determined that there is a grid manhole cover in the area of ​​the preset range of the mobile robot, and the mobile robot is instructed to move according to the previously planned navigation route of the navigation map.

[0075] After obtaining the raster data in which some or all of the grids are marked as ground grids, project the underground point cloud data into the raster data. Figure 4 A grid data projection diagram of a grid manhole cover. Since the side length of the grid is greater than the width of the drainage hole, each grid will inevitably contain some beams or other non-drainage holes of the manhole cover. Therefore, for the area corresponding to the ground where the grid manhole cover is located in the grid data, that is, Figure 4 Each grid shown in the figure is marked as a ground grid because of the presence of ground point cloud data. The ground grid is equivalent to Figure 4 Grid C in medium grey, Figure 4 The black dot D in the lower middle represents the underground point cloud data. Figure 4 The white point E in the figure represents the projection of underground point cloud data into raster data.

[0076] When all the grids that the underground point cloud data fall into are ground grids, it means that these underground point cloud data are invalid obstacle points, and it can be determined that there are grid manhole covers in the preset range of the mobile robot. This embodiment uses the characteristic that the drainage hole of the grid manhole cover is close to the crossbeam. In the case of underground point cloud data, the ground corresponding to the grid manhole cover in the grid data is all marked as ground grids, then it can be considered that there are grid manhole covers that do not affect normal passage.

[0077] After determining that there is a manhole cover in the area of ​​the preset range of the mobile robot, regardless of whether the manhole cover exists in the previously planned navigation route, the mobile robot can be instructed to continue moving directly according to the previously planned navigation route, and the manhole cover will not affect the normal movement of the mobile robot.

[0078] In one embodiment, the following steps are also included:

[0079] If the grid where each underground point cloud data falls is not a ground grid, the grid where the underground point cloud data falls is regarded as the underground grid;

[0080] The position of the underground grid is located in the navigation map. If the position is on a previously planned navigation route of the navigation map, the mobile robot is instructed to detour to avoid the position.

[0081] refer to Figure 5 A schematic diagram of raster data of a sunken area. Figure 5 The medium gray grid C is the ground grid. Figure 5 The white grid F in the middle is blank before the underground point cloud data is projected onto the grid data. When the grid into which each underground point cloud data falls is not a ground grid but a blank grid, the blank grid into which the underground point cloud data falls is marked as an underground grid G. The underground point cloud data falling into the underground grid can be considered as a valid obstacle point H.

[0082] The position of the underground grid in the grid data can be first determined in a two-dimensional coordinate system, and then located in a navigation map according to the position of the underground grid in the grid data. The located position of the underground grid can be confirmed as a sunken area in the navigation map.

[0083] The determined sunken area only indicates that there is a sunken situation within a certain area centered on the mobile robot. The sunken area may not be located on the previously planned navigation route of the navigation map. Only when the sunken area is located on the previously planned navigation route of the navigation map will it affect the normal driving of the mobile robot, and the mobile robot will be instructed to bypass the location of the sunken area.

[0084] In another implementation, the number of connected underground grids and the area occupied by the connected underground grids located in the navigation map can be determined first, and then combined with the mobile robot's own situation to determine whether it needs to be regarded as an obstacle. For example, if the area of ​​the sunken area is much smaller than the contact surface between the mobile robot's wheels and the ground and has no effect on the movement of the mobile robot, the mobile robot can be instructed to move normally according to the navigation route.

[0085] In one embodiment, the following steps are also included:

[0086] During the movement of the mobile robot, a real-time determined area is obtained according to the position information, and raster data is determined according to the area;

[0087] Determine whether there is an overlapping area between the previously obtained raster data and the raster data determined in real time;

[0088] If so, the ground grid and the underground grid in the overlapping area are retained in the real-time determined grid data.

[0089] Since the laser radar beams are relatively sparse, the grid data at a single moment is difficult to reflect the accurate and complete ground conditions. Figure 6 A schematic diagram of ground grid marking, collecting point cloud data with mobile robot A as the center, Figure 6 All the dots in represent the ground point cloud I collected by the laser radar at a certain moment. Figure 6 It can be seen that each grid where the ground point cloud I is located will be marked in gray, that is, marked as ground grid C. However, for the white grid F in the figure, there is a possibility that it actually belongs to the ground grid, but because the laser radar beam is relatively sparse and the corresponding ground point cloud I cannot be collected, the grid is blank and not marked.

[0090] In order to make up for the shortcomings of laser radar in collecting point cloud data and obtain raster data that is more in line with the actual situation, when determining the raster data of the current position, the real-time determined area can be obtained according to the position information, and rasterization processing can be performed on the area to obtain the real-time determined raster data.

[0091] It can be determined whether there is an overlapping area between the previously obtained raster data and the real-time determined raster data. If there is an overlapping area, the ground raster and underground raster in the overlapping area can be used to determine the raster data based on the partially marked raster. If there is no overlapping area, the raster data can only be determined in all unmarked raster.

[0092] The raster data obtained previously may be the raster data of the previous moment. Since the raster data indicates an area range, there is a high probability that there is an overlapping area between the current real-time determined raster data and the raster data of the previous moment. After determining the overlapping area, the ground raster and the underground raster in the overlapping area are retained in the real-time determined raster data. When the ground raster and the underground raster in the overlapping area are retained, after the raster data is projected onto the ground point cloud data at the current moment, the number of unmarked raster grids will be greatly reduced, and the marking of the raster data will be closer to the actual situation.

[0093] It should be noted that retaining the ground grid and the underground grid in the overlapping area does not mean that there must be an underground grid in the overlapping area. It means that if there is an underground grid in advance, it will be retained.

[0094] In one embodiment, the following steps are also included:

[0095] According to the point cloud data acquired in real time, unmarked grids in the real-time determined grid data are marked.

[0096] In the raster data, there may be ground grids and underground grids retained from overlapping areas, and there may also be unmarked grids. The unmarked grids may be grids in non-overlapping areas of the area determined in real time based on the position information, or grids in overlapping areas. When ground point cloud data falls into an unmarked grid of the raster data, the unmarked grid of the raster data is marked as a ground grid. When underground point cloud data falls into an unmarked grid of the raster data, the unmarked grid of the raster data is marked as an underground grid.

[0097] During the movement, the unmarked grids in the overlapping area are marked, which can be understood as correcting and patching a large number of unmarked grids generated due to the sparse point cloud collected by the lidar, so as to obtain grid data that is closer to the actual ground and underground conditions.

[0098] An embodiment of the present invention provides a grid manhole cover detection method, which is applied to a mobile robot. During the movement of the mobile robot, the position information and point cloud data of the mobile robot are obtained in real time, each point cloud data is segmented to obtain ground point cloud data and underground point cloud data, the position information is located in a navigation map stored in the mobile robot, and a region within a preset range centered on the position information is rasterized to obtain raster data, wherein the side length of each grid in the raster data is greater than the preset side length, and the preset side length is determined according to the size of the drainage hole of the grid manhole cover, so that the grids at the positions corresponding to the drainage holes will also have the ground positions. If a grid manhole cover exists, the ground point cloud data is projected into the grid data to obtain the grid into which the ground point cloud data falls, which is marked as the ground grid; the underground point cloud data is projected into the grid data; if the grids into which each underground point cloud data falls are all ground grids, it is determined that a grid manhole cover exists in an area within a preset range of the mobile robot. The embodiment of the present invention can realize real-time detection of grid manhole covers on the road surface without relying on a map that has marked the road conditions in advance, and the size of each grid is set by utilizing the structural characteristics of the grid manhole cover, so that an accurate judgment can be made quickly on the grid manhole covers on the road. When a grid manhole cover exists in the moving route of the mobile robot, it can be indicated that no detour is required.

[0099] Embodiment 2

[0100] Figure 7 A grid manhole cover detection device provided in the second embodiment of the present invention is applied to a mobile robot, and the device includes:

[0101] An acquisition module 710 is used to acquire the position information and point cloud data of the mobile robot in real time during the movement of the mobile robot;

[0102] The segmentation module 720 is used to segment each point cloud data to obtain ground point cloud data and underground point cloud data;

[0103] A rasterization processing module 730 is used to locate the position information in the navigation map stored in the mobile robot, and perform rasterization processing on an area within a preset range centered on the position information to obtain raster data, wherein the side length of each grid in the raster data is greater than a preset side length, and the preset side length is determined according to the size of the drainage hole of the grid manhole cover;

[0104] A ground grid marking module 740 is used to project the ground point cloud data into the grid data to obtain the grid into which the ground point cloud data falls, and mark it as a ground grid;

[0105] The grid manhole cover determination module 750 is used to project the underground point cloud data into the grid data. If the grids into which each underground point cloud data falls are the ground grids, it is determined that there is a grid manhole cover in the area of ​​the preset range of the mobile robot, and the mobile robot is instructed to move according to the previously planned navigation route of the navigation map.

[0106] In one embodiment, the segmentation module 720 includes the following submodules:

[0107] The plane fitting submodule is used to perform plane fitting based on each point cloud data to obtain the fitted ground points;

[0108] A comparison submodule, used to determine the height of each point cloud data and the height of the fitted ground point, and compare the height of each point cloud data with the fitted ground point;

[0109] The execution submodule is used to determine the point cloud data located below the fitting ground point and exceeding a certain distance threshold as underground point cloud data, and determine the point cloud data located within a specified height distance range of the fitting ground point as ground point cloud data.

[0110] In one embodiment, the real-time grid data is in a two-dimensional coordinate system;

[0111] The ground grid marking module 740 includes the following submodules:

[0112] A two-dimensional coordinate system acquisition submodule, used to acquire the two-dimensional coordinate system where the raster data is located;

[0113] The point cloud data marking submodule is used to obtain the horizontal axis coordinate value and the vertical axis coordinate value of the ground point cloud data, and mark the current point cloud data in the two-dimensional coordinate system according to the horizontal axis coordinate value and the vertical axis coordinate value.

[0114] In one embodiment, the device further comprises the following modules:

[0115] An underground grid determination module, configured to, when the grid into which each of the underground point cloud data falls is not the ground grid, regard the grid into which the underground point cloud data falls as the underground grid;

[0116] A detour module is used to locate the position of the underground grid in the navigation map, and if the position is on a previously planned navigation route of the navigation map, instruct the mobile robot to detour to avoid the position.

[0117] In one embodiment, the preset side length is equal to the width of the drain hole of the grid manhole cover.

[0118] In one embodiment, the device further comprises the following modules:

[0119] A raster data determination module is used to obtain a real-time determined area according to the position information during the movement of the mobile robot, and determine raster data according to the area;

[0120] A judgment module, used for judging whether there is an overlapping area between the previously obtained raster data and the raster data determined in real time;

[0121] The retaining module is used for retaining the ground grid and the underground grid in the overlapping area in the real-time determined grid data when there is an overlapping area.

[0122] In one embodiment, the device is further used for:

[0123] According to the point cloud data acquired in real time, unmarked grids in the real-time determined grid data are marked.

[0124] A grid manhole cover detection device provided in an embodiment of the present invention can implement a grid manhole cover detection method provided in the first embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0125] Embodiment 3

[0126] Figure 8A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0127] like Figure 8 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, ROM 12 and RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0128] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0129] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a grid manhole cover detection method.

[0130] In some embodiments, a grid manhole cover detection method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of a grid manhole cover detection method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform a grid manhole cover detection method in any other suitable manner (for example, by means of firmware).

[0131] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0132] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0133] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0134] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0135] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0136] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0137] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0138] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting a grid manhole cover, characterized in that: The method is applied to a mobile robot, and the method comprises: During the movement of the mobile robot, the position information and point cloud data of the mobile robot are acquired in real time; Segment each point cloud data to obtain ground point cloud data and underground point cloud data; The position information is located in a navigation map stored in the mobile robot, and a region within a preset range centered on the position information is rasterized to obtain raster data, wherein the side length of each grid in the raster data is greater than a preset side length, and the preset side length is determined according to the size of the drainage hole of the grid manhole cover; Projecting the ground point cloud data into the grid data to obtain the grid into which the ground point cloud data falls, and marking it as a ground grid; The underground point cloud data are projected into the grid data. If the grids into which each underground point cloud data falls are the ground grids, it is determined that there is a grid manhole cover in the area of ​​the preset range of the mobile robot, and the mobile robot is instructed to move according to the previously planned navigation route of the navigation map.

2. The method according to claim 1, characterized in that The step of segmenting each point cloud data to obtain ground point cloud data and underground point cloud data includes: Perform plane fitting based on each point cloud data to obtain fitting ground points; Determine the height of each point cloud data and the height of the fitted ground point, and compare the height of each point cloud data with the fitted ground point; The point cloud data located below the fitting ground point and exceeding a certain distance threshold are determined as underground point cloud data, and the point cloud data located within a specified height distance range of the fitting ground point are determined as ground point cloud data.

3. The method according to claim 2, characterized in that The grid data is located in a two-dimensional coordinate system; The projecting the ground point cloud data into the raster data comprises: Obtain the two-dimensional coordinate system where the raster data is located; The horizontal axis coordinate value and the vertical axis coordinate value of the ground point cloud data are obtained, and the current point cloud data is marked in the two-dimensional coordinate system according to the horizontal axis coordinate value and the vertical axis coordinate value.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: If the grid into which each of the underground point cloud data falls is not the ground grid, the grid into which the underground point cloud data falls is used as the underground grid; The position of the underground grid is located in the navigation map, and if the position is on a previously planned navigation route of the navigation map, the mobile robot is instructed to detour to avoid the position.

5. The method according to any one of claims 1 to 3, characterized in that: The preset side length is equal to the width of the drain hole of the grid manhole cover.

6. The method according to claim 4, characterized in that The method further comprises: During the movement of the mobile robot, a region determined in real time is obtained according to the position information, and raster data is determined according to the region; Determine whether there is an overlapping area between the previously obtained raster data and the raster data determined in real time; If so, the ground grid and the underground grid in the overlapping area are retained in the real-time determined grid data.

7. The method according to claim 6, characterized in that The method further comprises: According to the point cloud data acquired in real time, unmarked grids in the real-time determined grid data are marked.

8. A grid manhole cover detection device, characterized in that: The device is applied to a mobile robot, and comprises: An acquisition module, used to acquire the position information and point cloud data of the mobile robot in real time during the movement of the mobile robot; A segmentation module is used to segment each point cloud data to obtain ground point cloud data and underground point cloud data; A rasterization processing module, used for locating the position information in the navigation map stored in the mobile robot, and performing rasterization processing on an area within a preset range centered on the position information to obtain raster data, wherein the side length of each grid in the raster data is greater than a preset side length, and the preset side length is determined according to the size of the drainage hole of the grid manhole cover; A ground grid marking module, used for projecting the ground point cloud data into the grid data to obtain the grid into which the ground point cloud data falls, and marking it as a ground grid; A grid manhole cover determination module is used to project the underground point cloud data into the grid data. If the grids into which each underground point cloud data falls are the ground grids, it is determined that there is a grid manhole cover in the area of ​​the preset range of the mobile robot, and the mobile robot is instructed to move according to the previously planned navigation route of the navigation map.

9. An electronic device, characterized in that: The electronic device comprises: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a grid manhole cover detection method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a grid manhole cover detection method according to any one of claims 1 to 7 when executed.

Images