Indoor mobile robot ground tiny obstacle detection method and device

Abstract

The invention provides a ground tiny obstacle detection method and device for an indoor mobile robot, and the method comprises the steps: carrying out the external parameter calibration, time synchronization and interpolation fusion of multi-line sensor data, so as to obtain ground point cloud data; removing the unreasonable point cloud data, and carrying out modeling to obtain a ground background; screening ground suspicious obstacle points, performing neighborhood curvature clustering, and obtaining a plurality of suspicious obstacle clusters; all the subareas of the ground and the suspicious obstacle clusters are incorporated into a graph optimization framework, and global fitting of the ground curved surface is obtained; performing connectivity and layering processing on the suspicious obstacle cluster through a multi-scale morphological operation method and a region growth correction method; judging the stability of the suspicious obstacle cluster by adopting a time sliding window, and judging the state of the suspicious obstacle cluster by utilizing speed so as to filter dynamic interference; performing attitude or texture verification on the suspicious obstacle cluster to determine an obstacle cluster; and inputting the determined obstacle cluster into a navigation map of the indoor mobile robot.

Description

Technical Field

[0001] This application relates to the technical field of obstacle detection for indoor mobile robots, and particularly to a method and device for detecting minute obstacles on the ground of an indoor mobile robot. Background Art

[0002] With the increasingly widespread application of service robots in indoor environments such as medical care, home assistance, warehousing logistics, and public security, when mobile robots perform autonomous navigation and task planning, they often need to face indoor scenarios that are narrow, densely populated with obstacles, and have diverse terrains.

[0003] However, due to the complex and ever-changing indoor environment and the diverse forms of ground obstacles, it is difficult for indoor mobile robots to detect minute obstacles.

[0004] Therefore, this application provides a method and device for detecting minute obstacles on the ground of an indoor mobile robot. Summary of the Invention

[0005] Embodiments of this application provide a method and device for detecting minute obstacles on the ground of an indoor mobile robot to efficiently detect minute obstacles on the indoor ground.

[0006] In a first aspect, embodiments of this application provide a method for detecting minute obstacles on the ground of an indoor mobile robot, including:

[0007] Performing extrinsic parameter calibration, time synchronization, and interpolation fusion on multi-line sensor data to obtain ground point cloud data;

[0008] Removing unreasonable point cloud data and performing modeling to obtain a ground background;

[0009] Screening ground suspicious obstacle points, performing neighborhood curvature clustering, and obtaining several suspicious obstacle clusters;

[0010] Incorporating each partition of the ground and the suspicious obstacle clusters into a graph optimization framework to obtain a global fitting of the ground surface;

[0011] Performing connectivity and layering processing on the suspicious obstacle clusters through a multi-scale morphological operation method and a region growth correction method;

[0012] Using a time sliding window to judge the stability of the suspicious obstacle clusters and using speed to determine the states of the suspicious obstacle clusters to filter dynamic interference;

[0013] Performing attitude or texture verification on the suspicious obstacle clusters to determine obstacle clusters;

[0014] Inputting the determined obstacle clusters into the navigation map of the indoor mobile robot.

[0015] In a feasible implementation manner, the external parameter calibration of the multi-line sensor data includes:

[0016] Converting the laser points in the lidar L of the indoor mobile robot at time t i in it to the world coordinate system W according to formula (1);

[0017] Among them, the formula (1) specifically refers to:

[0018]

[0019] Among them, refers to the coordinates of the laser point in the robot body coordinate system, refers to the external parameters of the lidar, refers to the coordinates of the laser point at time t; refers to the coordinates of the laser point in the world coordinate system, T B→W (t)p refers to the transformation matrix that converts the coordinates of the laser point at time t in the robot body coordinate system to the coordinates in the world coordinate system.

[0020] The time synchronization includes:

[0021] Using linear interpolation or based on the IMU timestamp to perform time synchronization on each line of laser points, so that when the indoor mobile robot turns or accelerates / decelerates, the laser data of each line is merged into a unified frame at the same moment; and / or, complete the external parameter calibration of the laser-image coordinates;

[0022] The interpolation fusion includes:

[0023] If the adjacent lidar beams l i and l j are close enough in scanning angle θ but have a large difference in range resolution, perform distance or coordinate interpolation to generate a denser near-ground point cloud P dense .

[0024] In a feasible implementation manner, the screening of suspicious ground obstacle points includes:

[0025] Filter the point cloud data higher than the specified height in the ground point cloud data, and retain the target point cloud data within the specified height range;

[0026] Project the target point cloud data onto a 2D grid map, and calculate the maximum height and minimum height in each grid;

[0027] According to the RANSAC or piecewise plane fitting method, obtain several ground segments to obtain the ground background.

[0028] In a feasible implementation, a height difference residual calculation method, a normal vector deviation detection, and a double curvature measure method are used to screen out suspicious ground obstacle points;

[0029] Among them, the height difference residual calculation method includes:

[0030] Determine the point coordinates and judge whether the point coordinates exceed the specified threshold. If they exceed the specified threshold, the point is determined as a suspicious obstacle point;

[0031] If the height difference residual calculation method cannot determine whether the point is a suspicious obstacle point, calculate the angle between the normal vector and the reference plane normal in the local neighborhood. If the angle is greater than the specified angle, or the minimum eigenvalue of the covariance of the point shows a significant anomaly, the point is determined as a suspicious obstacle point.

[0032] In a feasible implementation, the method of incorporating each ground partition and suspicious obstacle points into the graph optimization framework to obtain a global fitting of the ground surface and identifying large residual points as obstacle points includes:

[0033] Regard the suspicious obstacle points and ground segments obtained in the time series as graph nodes, and establish a factor connection graph including neighborhood smoothing constraints, time consistency constraints, and suspicious obstacle and surface residual constraints.

[0034] In a feasible implementation, the method of performing connectivity and layering processing on the suspicious obstacle clusters through multi-scale morphological operation methods and region growth correction methods includes:

[0035] Map the suspicious obstacle clusters obtained after graph optimization back to the 2D grid for small-scale erosion to eliminate isolated points and then perform dilation to merge adjacent obstacles;

[0036] Perform morphological operations at a larger scale. If multiple fragmented obstacles are separated by < threshold, they are merged into a large cluster to eliminate duplicate detection phenomena;

[0037] Perform 8-connected or 16-connected region growth on the merged large cluster. If the curvature or height difference in the local neighborhood is similar to that of the cluster, it is incorporated into the cluster; if not, the original cluster boundary is retained, and finally a set of suspicious obstacle clusters that are smooth and free of isolated noise is formed.

[0038] In a feasible implementation, the method of using a time sliding window to judge the stability of the suspicious obstacle clusters and using speed to determine the state of the suspicious obstacle clusters to filter out dynamic interference includes:

[0039] Set the sliding window size K = 3 - 5 frames; if obstacle detection results of ≥ K / 2 exist at the same spatial position in the past K frames, increase its confidence;

[0040] If the moving speed of the suspicious obstacle cluster between two frames is greater than the specified speed, or it is determined to be a dynamic object based on the tracking state of the Kalman filter, then the suspicious obstacle cluster is determined to be a dynamic object.

[0041] In a feasible implementation, the attitude or texture verification of the suspicious obstacle cluster includes:

[0042] Project the suspicious obstacle cluster in the image coordinates, and compare its color and texture with the ground background respectively. If the difference degree is greater than the specified value, then increase the confidence of the suspicious obstacle cluster and determine it as the obstacle cluster.

[0043] In a feasible implementation, the inputting the determined obstacle cluster into the navigation map of the indoor mobile robot includes:

[0044] Input the verified obstacle cluster into the local or global occupancy grid map.

[0045] In a second aspect, an embodiment of the present application provides a device for detecting tiny ground obstacles of an indoor mobile robot, which applies the method for detecting tiny ground obstacles of an indoor mobile robot as described in the first aspect, and includes:

[0046] A controller;

[0047] A point cloud data acquisition module, electrically connected to the controller, and used for performing external parameter calibration, time synchronization, and interpolation fusion on the multi-line sensor data;

[0048] A ground background acquisition module, electrically connected to the controller, and used for removing unreasonable point cloud data and performing modeling to obtain the ground background;

[0049] An obstacle point screening module, electrically connected to the controller, screening ground suspicious obstacle points, performing neighborhood curvature clustering, and obtaining several suspicious obstacle clusters;

[0050] A suspicious obstacle cluster generation module, electrically connected to the controller, and used for incorporating each ground partition and suspicious obstacle points into the graph optimization framework to obtain a global fitting of the ground surface;

[0051] A suspicious obstacle cluster processing module, electrically connected to the controller, and used for performing connectivity and layering processing on the suspicious obstacle cluster through a multi-scale morphological operation method and a region growth correction method; judging the stability of the suspicious obstacle cluster by using a time sliding window and determining the state of the suspicious obstacle cluster by using speed to filter dynamic interference;

[0052] A suspicious obstacle cluster verification module, electrically connected to the controller, and used for performing attitude or texture verification on the suspicious obstacle cluster to determine the obstacle cluster;

[0053] An obstacle cluster input module, electrically connected to the controller, is configured to input the determined obstacle clusters into the navigation map of the indoor mobile robot.

[0054] An embodiment of the present application provides a method and device for detecting tiny ground obstacles of an indoor mobile robot, including: performing extrinsic calibration, time synchronization, and interpolation fusion on multi-line sensor data to obtain ground point cloud data; removing unreasonable point cloud data and performing modeling to obtain a ground background; screening ground suspicious obstacle points, performing neighborhood curvature clustering, and obtaining several suspicious obstacle clusters; incorporating each ground partition and the suspicious obstacle clusters into a graph optimization framework to obtain a global fit of the ground surface; performing connectivity and layering processing on the suspicious obstacle clusters through a multi-scale morphological operation method and a region growth correction method; using a time sliding window to determine the stability of the suspicious obstacle clusters and using speed to determine the state of the suspicious obstacle clusters to filter dynamic interference; performing attitude or texture verification on the suspicious obstacle clusters to determine obstacle clusters; and inputting the determined obstacle clusters into the navigation map of the indoor mobile robot, thereby achieving efficient detection of tiny ground obstacles. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The drawings described herein are used to provide a further understanding of the present invention, and constitute a part of the present invention. The schematic embodiments and descriptions thereof of the present invention are used to explain the present application and do not constitute an improper limitation to the present invention.

[0056] In the drawings:

[0057] Figure 1 is a schematic diagram of a method for detecting tiny ground obstacles of an indoor mobile robot provided by an embodiment of the present application;

[0058] Figure 2 is a point cloud schematic diagram of tiny ground obstacles provided by an embodiment of the present application;

[0059] Figure 3 is a schematic diagram of a device for detecting tiny ground obstacles of an indoor mobile robot provided by an embodiment of the present application.

[0060] DESCRIPTION OF REFERENCE NUMERALS

[0061] 100 - Controller; 200 - Map construction module; 300 - Pose evaluation module; 400 - Information gain evaluation module; 500 - Path planning module; 600 - Adjustment module. DETAILED DESCRIPTION

[0062] To enable those skilled in the art to better understand the technical solutions in this application, the following will clearly and completely describe the technical solutions in the embodiments of this application in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of this application.

[0063] In the description of the embodiments of this application, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In the description of this application, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0064] In this application, unless otherwise clearly defined and limited, terms such as "installed", "connected", "connected to", "fixed" and the like shall be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or integrated; it can be a mechanical connection; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the communication inside two elements or the interaction relationship between two elements, unless otherwise clearly limited. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0065] In this application, unless otherwise clearly defined and limited, the first feature being "on" or "under" the second feature can be that the first and second features are in direct contact, or the first and second features are indirectly in contact through an intermediate medium. Moreover, the first feature being "above", "over" and "on top of" the second feature can be that the first feature is directly above or obliquely above the second feature, or merely indicates that the first feature has a higher horizontal height than the second feature. The first feature being "under", "beneath" and "underneath" the second feature can be that the first feature is directly below or obliquely below the second feature, or merely indicates that the first feature has a lower horizontal height than the second feature.

[0066] As service robots are increasingly widely used in indoor environments such as medical care, home assistance, warehousing logistics, and public security, when mobile robots perform autonomous navigation and task planning, they often need to face indoor scenarios that are narrow, densely populated with obstacles, and have variable terrains.

[0067] However, due to the complex and ever-changing indoor environment and the diverse forms of ground obstacles, it is difficult for indoor mobile robots to detect tiny obstacles. For example, ground-hugging obstacles such as small protrusions (e.g., threshold differences, carpet edges), loose cables, and fine debris are often overlooked; simple horizontal scanning of lidar or vision algorithms seriously miss detecting objects with extremely low height and colors close to the ground; local high reflectivity or shadows cause sensor noise, making the detection results unstable.

[0068] To solve the problem of the difficulty in detecting tiny obstacles by indoor mobile robots as described above, this application provides a method and device for detecting tiny ground obstacles of indoor mobile robots. The following will detail the solution provided in the embodiments of this application with reference to the accompanying drawings of the specification.

[0069] Figure 1 It is a schematic diagram of a method for detecting tiny ground obstacles of an indoor mobile robot provided in an embodiment of this application.

[0070] Referring to Figure 1 as shown, the embodiments of this application provide a method for detecting tiny ground obstacles of an indoor mobile robot, which includes:

[0071] S100: Perform extrinsic parameter calibration, time synchronization, and interpolation fusion on multi-line sensor data to obtain ground point cloud data.

[0072] There are M lidars installed around the chassis of the indoor mobile robot. The installation height of each lidar is 5 - 15 cm, and each lidar has a different inclination angle thus forming an up-and-down micro-scanning angle combination to cover the ground-hugging area. It can be understood that by setting multiple lidars around the chassis of the indoor mobile robot, multiple radar inclination angle combinations can capture ground-hugging micro-protrusions from 3D directions, reducing the detection blind area and avoiding missing obstacles.

[0073] In addition, use a laser target or checkerboard calibration method to accurately calibrate the extrinsic parameters of the lidar with the robot body coordinate system {B}.

[0074] Then, convert the laser points i in the lidar L of the indoor mobile robot at time t to the world coordinate system W according to formula (1);

[0075] wherein, the formula (1) specifically refers to:

[0076]

[0077] wherein, refers to the coordinates of the laser point in the robot body coordinate system, refers to the external parameters of the laser radar. refers to the coordinates of the laser point at time t; Refers to the coordinates of the laser point in the world coordinate system, T B→W (t)p refers to the transformation matrix that converts the coordinates of the laser point at time t in the robot body coordinate system into the coordinates of the world coordinate system, which can be dynamically obtained from the odometer of the indoor mobile robot or the mileage estimate of the simultaneous localization and mapping system.

[0078] Then, the time synchronization of each line laser point is performed using linear interpolation or based on the IMU timestamp, so that when the indoor mobile robot turns or accelerates or decelerates, the laser data of each line is merged into a unified frame at the same time. In some examples, the laser-image coordinate extrinsic calibration can also be completed, and the result is stored in the camera's extrinsic transformation matrix.

[0079] In addition, among multiple laser radar beams, if adjacent laser radar beams l i With l j When the scanning angle θ is close enough but the range resolution is different, distance or coordinate interpolation can be performed to generate a denser near-ground point cloud P dense , thereby improving the resolution of micro-bumps on the ground and reducing the noise impact of a single beam.

[0080] This solution installs lidars at multiple inclination angles (a small number of degrees up / down) around the robot chassis and densifies the data in time and space using interpolation fusion, thereby significantly reducing the ground blind spot and making it easier to capture extremely low and small obstacles. In narrow indoor places, this multi-line combination can achieve a large field of view coverage near the ground, providing a reliable perception basis for threshold differences and cables that are easily missed, without significantly increasing sensor costs or system complexity.

[0081] S200: Eliminate the unreasonable point cloud data and perform modeling to obtain the ground background.

[0082] After obtaining the ground point cloud data in step S100, filter out the points above the specified height and retain the points at a height near the ground, which are recorded as a set For example, according to actual needs, points with a height greater than 30 cm may be filtered out. |z|>Z max , where Z max = 30cm. For points at a height near the ground Divide into sets in, is the main ROI range of z∈[-10cm,+10cm], These points are too high or too low, and these points can be processed as special cases later. For example, these points can represent slopes or potholes.

[0083] Then project all the points in onto a 2D grid, and calculate z for each grid (u, v). min , z max . Exemplarily, the resolution δ of this 2D grid is approximately 2 - 5 cm, and z min , z max represent the lowest point and the highest point respectively. The z value can help identify the height of obstacles.

[0084] Then, according to the RANSAC or piecewise plane fitting method, roughly obtain several ground segments {Π 1 ,..., Π k}}, and each segment Π i is described by (n i , d i ) or quadratic surface parameters (A i , B i ,...) to obtain the ground background. Points with excessive undulation or unable to be classified into any Π i in the 2D grid are listed as suspicious obstacle points.

[0085] S300: Screen the suspicious ground obstacle points.

[0086] Exemplarily, in this step, it is necessary to use the height difference residual calculation method, normal vector deviation detection, and bi - curvature measure method to screen the suspicious ground obstacle points.

[0087] Among them, the height difference residual calculation method specifically includes:

[0088] Determine the point coordinates. For point p(x p , y p ), if the plane / surface Π i it belongs to satisfies

[0089] then point p is classified into the suspicious obstacle set

[0090] Among them, represents the z value obtained by plane / surface fitting, z p - represents the height value of point p, and δ h represents the threshold.

[0091] If the height difference residual calculation method cannot determine whether this point is a suspicious obstacle point, the included angle Δθ between the normal vector n(p) calculated in the local neighborhood and the reference plane normal n i can be calculated;

[0092] If Δθ > θ 0 (such as 15°) or the minimum eigenvalue of the point cloud covariance shows a significant anomaly, then

[0093] Finally, the set of suspected obstacle points is determined as

[0094] When using the double curvature measure method to screen for suspected ground obstacle points, on a 3D surface, the principal curvature κ 1 , κ 2 can be defined as the maximum / minimum curvature; in the discrete implementation, the neighborhood covariance Σ p of the 3D point cloud is used for eigenvalue decomposition λ 1 ≥ λ 2 ≥ λ 3 ≥ 0:

[0095]

[0096] If κ max - κ min > κ 0 or κ max > κ th , it means that there is an obvious raised structure on the ground.

[0097] For the point cloud within the same local area (window Ω), if most points satisfy κ max < κ flat it indicates that the ground is flat; if there is obvious unevenness, it is judged as a micro-protrusion. Among them, κ flat represents the curvature value in the flat direction of the surface. Specifically, it represents the minimum curvature value in a certain direction and is used to describe the geometric characteristics of the surface.

[0098] Based on the height difference residual calculation method, the solution of this application also introduces the principal curvature of the surface and the eigenvalues of the local three-dimensional covariance, which has higher geometric discrimination for small bumps close to the ground and reduces the probability of confusion with slight undulations / grooves.

[0099] That is, on the basis of the common height difference residual calculation, the double principal curvatures (maximum and minimum curvatures) and the local covariance eigenvalue analysis are additionally introduced to enhance the geometric discrimination of extremely small protrusions. Combining the normal vector deviation and the curvature mutation can effectively identify small obstacles with a height of only 2 - 5 cm but with significant geometric differences from the ground; it can also reduce false alarms caused by ground micro-convex and micro-concave noises.

[0100] Next, the points in O cand are further subjected to neighborhood curvature clustering to form several suspected obstacle clusters {C j}, and each cluster satisfies conditions such as a significant increase in curvature.

[0101] S400: Incorporate each ground partition and the suspected obstacle clusters into the graph optimization framework to obtain a global fit of the ground surface.

[0102] The suspected obstacle clusters obtained under the time series t = 1...T and ground segments are regarded as graph nodes.

[0103] Define variables in the graph: the surface coefficient of the ground segment suspicious obstacle positions

[0104] Establish a factor connection graph that satisfies the following constraints:

[0105] Neighborhood smoothing: adjacent segments should be smoothly connected;

[0106] Temporal consistency: the same segment changes smoothly at times t and t + 1;

[0107] suspicious obstacles and surface residuals > δ h .

[0108] Use the multi-frame non-linear optimization method to optimize each partition of the ground and the suspicious obstacle clusters.

[0109] Among them, the multi-frame non-linear optimization formula is:

[0110]

[0111] Among them, ρ is the surface fitting residual with a robust kernel, and Res measures the deviation of the obstacle from the ground surface (if > the threshold, the obstacle mark is retained). The globally consistent ground surface estimate is obtained by iteration (such as LM / Trust-Region).

[0112] Compared with the common single-plane / local RANSAC, the solution of this application regards the ground fitting as a graph optimization problem with a robust kernel in a large range. Through the multi-partition state and neighborhood smoothing, the overall consistency and local flexibility of the ground are achieved. Theoretically, it can handle the diversity of the ground surface and significantly reduce the missed detection of a small number of ground protrusions. Using graph optimization to split the ground into several interconnected plane / surface regions with neighborhood smoothing and temporal consistency factors, different from traditional local methods such as only doing single-plane fitting or simple RANSAC once. Through the joint fitting of the robust kernel and multi-frame data, the uneven ground conditions (such as slopes, local protrusions, etc.) can be globally processed, reducing the misjudgment of a small number of high-noise points or small ground obstacle points close to the ground, and having stronger overall consistency and local flexibility.

[0113] S500: Through the multi-scale morphological operation method and the region growth correction method, perform connectivity and layering processing on the suspicious obstacle clusters.

[0114] Map the suspicious obstacle clusters obtained after graph optimization back to the 2D grid with a resolution of δ 1 for small-scale erosion E(C, r 1 ) to eliminate isolated points; then perform dilation to merge adjacent obstacles.

[0115] Perform a morphological operation at a larger scale (resolution δ 2 > δ 1 ). If multiple fragment obstacles are separated by < threshold, they are merged into a large cluster C k ; to eliminate the phenomenon of duplicate detection.

[0116] For the merged large cluster C k Perform 8-connected or 16-connected region growing: If the curvature / height difference in the local neighborhood is similar to that of the cluster, it is incorporated into the cluster; if not, the original cluster boundary is retained, and finally a set of smaller obstacle clusters {C 1 ,..., C m} that are smoother and free of isolated noise is formed.

[0117] It can be understood that in addition to one-time erosion-dilation, this application adds hierarchical morphological operations (small resolution δ 1 and large resolution δ 2 ) for suspicious obstacle clusters, combines region growing to determine connected clusters, thereby helping to distinguish true small obstacle clusters from scattered noise, and at the same time merging adjacent fragments to reduce false alarms and duplicate detections.

[0118] It can be understood that for the detected suspicious obstacle clusters, first perform multi-scale morphological processing such as multi-round erosion-dilation in the 2D grid, and then combine time sliding window and Kalman fusion for dynamic / static classification. On the one hand, it can suppress isolated noise points and merge fragment obstacles; on the other hand, it can improve the detection robustness in the time dimension and avoid false markings caused by robot jolts or pedestrian interference.

[0119] S600: Use a time sliding window to judge the stability of the suspicious obstacle cluster, and use speed to judge the state of the suspicious obstacle cluster to filter dynamic interference.

[0120] When using a time sliding window to judge the stability of a suspicious obstacle cluster, set the window size K = 3 - 5 frames; if there are obstacle detection results ≥ K / 2 at the same spatial position in the past K frames, increase its confidence; otherwise, it may be noise or pedestrian foot interference.

[0121] If the moving speed of the suspicious obstacle cluster is too large between two frames (e.g., > 1m / s), or it is judged to be a dynamic object (person, pet, etc.) according to the Kalman filter tracking state ; such are not regarded as ground small obstacles (which can be dynamically avoided by the navigation module) and are not written into the navigation map.

[0122] It is understandable that, based on the original single-frame detection, this application introduces multi-frame fusion to improve the confidence level and distinguish between static micro-obstacles and dynamic objects. In addition, the Kalman filter performs temporal smoothing on the obstacle coordinate estimation, which can eliminate high-frequency noise and sudden false alarms, and can significantly enhance the detection stability in a complex indoor environment.

[0123] S700: Perform pose or texture verification on the suspicious obstacle cluster to determine the obstacle cluster.

[0124] Exemplarily, performing pose or texture verification on the suspicious obstacle cluster includes:

[0125] Project the suspicious obstacle cluster in the image coordinates p img = Π(p W ), and compare its color and texture with the ground background respectively. If the difference degree is greater than the specified value (ΔI > I th ), then increase the confidence level of the said suspicious obstacle cluster and determine it as the obstacle cluster. In addition, when the indoor mobile robot makes a sharp turn or jolts severely, the ground-based lidar may shake severely; compensate for the pose change of the laser points through the IMU acceleration a and angular velocity ω to ensure the spatio-temporal consistency of the point cloud.

[0126] The solution of this application provides a fusion determination with the camera texture information, which can improve the determination efficiency in an environment with abnormal color / texture, can further reduce the negative impact of illumination and material differences on detection, and perform real-time pose correction of the IMU for laser jitter, improving the comprehensive accuracy of positioning and detection. It greatly expands the applicability of this method when implemented in a real scenario.

[0127] S800: Input the determined obstacle cluster into the navigation map of the indoor mobile robot.

[0128] Specifically, input the verified obstacle cluster into the local or global occupancy grid map.

[0129] Record the finally confirmed ground obstacle cluster {C k} as Occ, mark it as occupied in the local / global occupancy grid map; or set the value d obs = 0 in the ESDF (signed distance field);

[0130] If the obstacle is in front of the current planned path d p < δ p , trigger the local replanning strategy of the indoor mobile robot.

[0131] During the movement of the indoor mobile robot, if a large height difference > 10 cm is detected, or a suspected step / pit is detected, it is regarded as an abnormal scene, and human-machine interaction or shutdown is prompted;

[0132] If it is found that a certain line lidar completely lacks data or is extremely different from other lines, a hardware failure is determined and the fault tolerance mode is entered.

[0133] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0134] Based on the same inventive concept, the embodiments of the present application also provide an indoor mobile robot ground micro obstacle detection device for implementing the indoor mobile robot ground micro obstacle detection method described above. The implementation solutions provided by this device to solve problems are similar to those described in the above method. Therefore, the specific limitations in one or more embodiments of the indoor mobile robot ground micro obstacle detection device provided below can refer to the limitations on the indoor mobile robot ground micro obstacle detection method in the above text, and will not be elaborated here.

[0135] Figure 3 It is a schematic diagram of an indoor mobile robot ground micro obstacle detection device provided by an embodiment of the present application.

[0136] Refer to Figure 3As shown, in a second aspect, the embodiments of the present application provide a device for detecting minute ground obstacles of an indoor mobile robot, which applies the method for detecting minute ground obstacles of an indoor mobile robot as described in the first aspect. The device includes a controller, a point cloud data acquisition module, a ground background acquisition module, an obstacle point screening module, a suspicious obstacle cluster processing module, a suspicious obstacle cluster verification module, and an obstacle cluster input module. Among them, the point cloud data acquisition module is electrically connected to the controller and is used for performing external parameter calibration, time synchronization, and interpolation fusion on multi-line sensor data; the ground background acquisition module is electrically connected to the controller and is used for removing unreasonable point cloud data and performing modeling to obtain the ground background; the obstacle point screening module is electrically connected to the controller, screening suspicious ground obstacle points, performing neighborhood curvature clustering, and obtaining several suspicious obstacle clusters; the suspicious obstacle cluster generation module is electrically connected to the controller and is used for incorporating each partition of the ground and the suspicious obstacle points into the graph optimization framework to obtain a global fit of the ground surface; the suspicious obstacle cluster processing module is electrically connected to the controller and is used for performing connectivity and layering processing on the suspicious obstacle clusters through a multi-scale morphological operation method and a region growth correction method; judging the stability of the suspicious obstacle clusters by using a time sliding window and determining the state of the suspicious obstacle clusters by using speed to filter dynamic interference; the suspicious obstacle cluster verification module is electrically connected to the controller and is used for performing attitude or texture verification on the suspicious obstacle clusters to determine the obstacle clusters; the obstacle cluster input module is electrically connected to the controller and is used for inputting the determined obstacle clusters into the navigation map of the indoor mobile robot.

[0137] Each module in the above-mentioned mobile robot autonomous positioning device for an indoor dynamic scene can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor in the computer device in the form of hardware or be independent of it, or can be stored in the memory in the computer device in the form of software, so as to facilitate the processor to call and execute the operations corresponding to the above-mentioned modules.

[0138] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include Read-Only Memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0139] It is easily understandable that those skilled in the art can combine, split, reorganize, etc. the embodiments of the present application based on several embodiments provided in the present application to obtain other embodiments, and none of these embodiments exceed the protection scope of the present application.

[0140] The above specific implementation manners further elaborate on the purpose, technical solutions, and beneficial effects of the embodiments of the present application. It should be understood that the above is only the specific implementation manners of the embodiments of the present application and is not used to limit the protection scope of the embodiments of the present application. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of the embodiments of the present application should be included in the protection scope of the embodiments of the present application.

Claims

1. A method for detecting small obstacles on the ground by an indoor mobile robot, characterized in that: include: Perform external parameter calibration, time synchronization and interpolation fusion on multi-line sensor data to obtain ground point cloud data; Eliminate the unreasonable point cloud data and perform modeling to obtain the ground background; Screen suspicious obstacle points on the ground, perform neighborhood curvature clustering, and obtain several suspicious obstacle clusters; Incorporating all ground partitions and the suspected obstacle clusters into the graph optimization framework to obtain a global fit of the ground surface; Through multi-scale morphological operation method and regional growth correction method, the connectivity and stratification of suspicious obstacle clusters are processed; A time sliding window is used to determine the stability of the suspected obstacle cluster, and a speed is used to determine the state of the suspected obstacle cluster to filter dynamic interference; Performing posture or texture verification on the suspected obstacle cluster to determine the obstacle cluster; The determined obstacle cluster is input into the navigation map of the indoor mobile robot.

2. The method for detecting small obstacles on the ground of an indoor mobile robot according to claim 1, characterized in that: The extrinsic parameter calibration of the multi-line sensor data includes: The laser radar L of the indoor mobile robot at time t i Laser point in Convert to the world coordinate system W according to formula (1); Wherein, the formula (1) specifically refers to: in, Refers to the coordinates of the laser point in the robot body coordinate system, refers to the external parameters of the laser radar. refers to the coordinates of the laser point at time t; Refers to the coordinates of the laser point in the world coordinate system, T B→W (t)p refers to the transformation matrix that converts the coordinates of the laser point at time t in the robot body coordinate system into the coordinates of the world coordinate system. The time synchronization includes: Time synchronization of each line laser point is performed using linear interpolation or based on IMU timestamp, so that when the indoor mobile robot turns or accelerates or decelerates, each line laser data is merged into a unified frame at the same time; and / or, laser-image coordinate external parameter calibration is completed; The interpolation fusion includes: If the adjacent laser radar beam l i With l j When the scanning angle θ is close enough but the range resolution is different, distance or coordinate interpolation is performed to generate a denser near-ground point cloud P dense .

3. The method for detecting small obstacles on the ground of an indoor mobile robot according to claim 1, characterized in that: The screening of suspected ground obstacle points includes: Filtering point cloud data above a specified height in the ground point cloud data, and retaining target point cloud data within a specified height range; Project the target point cloud data onto a 2D grid map, and calculate the maximum and minimum heights in each grid; According to RANSAC or piecewise plane fitting method, several ground segments are obtained to obtain the ground background.

4. The method for detecting small obstacles on the ground of an indoor mobile robot according to claim 1, characterized in that: Use height difference residual calculation method, normal vector deviation detection and hyperbolic curvature measurement method to screen suspicious ground obstacle points; The height difference residual calculation method includes: Determine the point coordinates and judge whether the point coordinates exceed the specified threshold. If they exceed the specified threshold, the point is determined to be a suspicious obstacle point; If the height difference residual calculation method cannot determine whether the point is a suspected obstacle point, the angle between the normal vector and the normal of the reference plane is calculated in the local neighborhood. If the angle is greater than the specified angle, or the minimum eigenvalue of the covariance of the point shows a significant abnormality, the point is determined to be a suspected obstacle point.

5. The method for detecting small obstacles on the ground of an indoor mobile robot according to claim 1, characterized in that: The above-mentioned method incorporates all ground partitions and suspected obstacle points into the graph optimization framework to obtain a global fit of the ground surface, and identifies large residual points as obstacle points, including: The suspicious obstacle points and ground segments obtained in the time series are regarded as graph nodes, and a factor connection graph including neighborhood smoothness constraints, time consistency constraints, and suspicious obstacle and surface residual constraints is established.

6. The method for detecting small obstacles on the ground of an indoor mobile robot according to claim 1, characterized in that: The multi-scale morphological operation method and the regional growth correction method are used to perform connectivity and layering processing on the suspicious obstacle clusters, including: The suspicious obstacle clusters obtained after graph optimization are mapped back to the 2D grid for small-scale erosion, and isolated points are eliminated before dilation and merging of adjacent obstacles. Morphological operations are performed at a larger scale. If multiple debris obstacles are separated by less than a threshold, they are merged into a large cluster to eliminate repeated detections. The merged large clusters are subjected to 8-connected or 16-connected region growth. If the curvature or height difference in the local neighborhood is similar to that of the cluster, it is incorporated into the cluster; if not, the original cluster boundary is retained, and finally a smooth suspicious obstacle cluster set without isolated noise is formed.

7. The method for detecting small obstacles on the ground of an indoor mobile robot according to claim 1, characterized in that: The method of using a time sliding window to judge the stability of the suspected obstacle cluster and using speed to judge the state of the suspected obstacle cluster to filter dynamic interference includes: Assume that the sliding window size K = 3 to 5 frames; if the same spatial position has ≥ K / 2 obstacle detection results in the past K frames, then increase its confidence; If the moving speed of the suspicious obstacle cluster between two frames is greater than the specified speed, or it is judged to be a dynamic object according to the Kalman filter tracking state, then the suspicious obstacle cluster is determined to be a dynamic object.

8. The method for detecting small obstacles on the ground of an indoor mobile robot according to claim 1, characterized in that: The performing posture or texture verification on the suspicious obstacle cluster includes: The suspicious obstacle cluster is projected in the image coordinates, and its color and texture are compared with the ground background respectively. If the difference is greater than a specified value, the confidence of the suspicious obstacle cluster is increased and it is determined to be the obstacle cluster.

9. The method for detecting small obstacles on the ground of an indoor mobile robot according to claim 1, characterized in that: The step of inputting the determined obstacle cluster into the navigation map of the indoor mobile robot comprises: The verified obstacle cluster is input into a local or global occupancy grid map.

10. A ground micro-obstacle detection device for an indoor mobile robot, characterized in that: The method for detecting small obstacles on the ground of an indoor mobile robot according to any one of claims 1 to 9 is applied, comprising: Controller; The point cloud data acquisition module is electrically connected to the controller and is used to perform external parameter calibration, time synchronization and interpolation fusion on the multi-line sensor data; A ground background acquisition module is electrically connected to the controller and is used to remove unreasonable point cloud data and perform modeling to obtain the ground background; The obstacle point screening module is electrically connected to the controller, screens suspicious obstacle points on the ground, performs neighborhood curvature clustering, and obtains a number of suspicious obstacle clusters; The obstacle cluster generation module is electrically connected to the controller and is used to incorporate all ground partitions and suspicious obstacle points into the graph optimization framework to obtain a global fit of the ground surface; The suspicious obstacle cluster processing module is electrically connected to the controller and is used to perform connectivity and layering processing on the suspicious obstacle clusters through a multi-scale morphological operation method and a regional growth correction method; a time sliding window is used to determine the stability of the suspicious obstacle clusters, and a speed is used to determine the state of the suspicious obstacle clusters to filter dynamic interference; A suspicious obstacle cluster verification module, electrically connected to the controller, for performing posture or texture verification on the suspicious obstacle cluster to determine the obstacle cluster; The obstacle cluster input module is electrically connected to the controller and is used to input the determined obstacle cluster into the navigation map of the indoor mobile robot.

Images