Obstacle avoidance method and system for mobile robot and mobile robot

Abstract

The invention provides a technical scheme for obstacle avoidance of a mobile robot, and the method comprises the steps: obtaining a depth map of an environment in the field of view of the mobile robot through a depth camera, and determining first depth point cloud data under a depth camera coordinate system based on the depth map, converting the first depth point cloud data into third depth point cloud data under a mobile robot coordinate system, and preprocessing the third depth point cloud data to obtain fifth depth point cloud data; filtering ground point cloud data in the mobile robot to obtain a point cloud height map, performing leveling processing and down-sampling processing on the point cloud height map to obtain a point cloud memory map, and when an obstacle is detected according to the point cloud memory map, determining an obstacle avoidance strategy according to the distance and angle of the obstacle and the current motion state of the mobile robot; and executing the obstacle avoidance strategy to realize obstacle avoidance. The data processing amount can be effectively reduced, the obstacle detection efficiency and accuracy are improved, the obstacle avoidance real-time performance and efficiency are improved, and the passing rate in a narrow space can be improved.

Description

Technical Field

[0001] The present application relates to the field of mobile robot technology, and in particular to a technology for mobile robot obstacle avoidance. Background Art

[0002] With the rapid development of mobile robot technology, mobile robots are increasingly being used in home / commercial services, medical care, rescue, and other fields. Obstacle avoidance technology, as one of the key technologies applied to mobile robots, directly affects the navigation efficiency and accuracy of mobile robots. Existing mobile robot obstacle avoidance methods generally use visualization methods, grid methods, topology methods, or artificial potential field methods. However, these methods for mobile robot obstacle avoidance still have many limitations when facing complex dynamic environments. For example, although the grid method is widely used, the grid size affects the storage capacity of environmental information and the calculation time. The smaller the grid, the better the obstacle avoidance effect, but the calculation amount increases significantly.

[0003] As high-resolution sensors such as multi-line lidar and depth cameras begin to be equipped on mobile robots, they are able to cope with more complex environments. A common method is to process the environmental data obtained by lidar and / or depth cameras into point cloud data and fuse it into the cost map to generate an expansion radius for the mobile robot to avoid obstacles. This method can meet the needs in most scenarios, but in narrow spaces such as aisles, which are usually only slightly wider than the mobile robot, mobile robots generally lack backward observation and cannot turn around in narrow spaces. The mobile robot is more likely to trigger the retreat mechanism, which will seriously reduce the success rate of the mobile robot's obstacle avoidance in narrow spaces and affect the mobile robot's ability to pass through narrow spaces.

[0004] In addition, due to the weak GPS signal in indoor environments, mobile robots providing services indoors cannot be accurately positioned through GPS. They can only perceive the environment and estimate their own posture through the mobile robot's own sensors. Without a priori Simultaneous Localization and Mapping (SLAM) map, it becomes very difficult for mobile robots to judge dynamic obstacles, especially in narrow spaces.

[0005] Therefore, how to improve the success rate of obstacle avoidance of mobile robots in narrow spaces has become a technical problem that needs to be solved urgently. Summary of the Invention

[0006] The purpose of this application is to provide a method, system and mobile robot for mobile robot obstacle avoidance, so as to at least partially solve the technical problem in the prior art that the success rate of mobile robots in avoiding obstacles in narrow spaces is not high.

[0007] According to one aspect of the present application, a method for obstacle avoidance of a mobile robot is provided, wherein the method comprises:

[0008] Acquire a depth map of an environment within a field of view of the mobile robot through a depth camera, determine first depth point cloud data in the depth camera coordinate system based on the depth map, convert the first depth point cloud data into third depth point cloud data in the mobile robot coordinate system, and pre-process the third depth point cloud data to obtain fifth depth point cloud data;

[0009] filtering ground point cloud data in the fifth depth point cloud data to obtain a point cloud height map, flattening and downsampling the point cloud height map to obtain a point cloud memory map, and determining an obstacle avoidance strategy based on a distance and angle of the obstacle relative to the mobile robot and a current motion state of the mobile robot when an obstacle is detected based on the point cloud memory map;

[0010] The mobile robot is controlled to execute the obstacle avoidance strategy to achieve obstacle avoidance.

[0011] Optionally, filtering the ground point cloud data to obtain a point cloud height map includes:

[0012] Based on the ground segmentation algorithm, the ground point cloud is screened out and filtered out to obtain the point cloud height map.

[0013] Optionally, the performing flattening and downsampling processing on the point cloud height map includes:

[0014] The point cloud height map is flattened to convert the point cloud height map into a two-dimensional point cloud plane map, and then a number of point clouds in each preset angle direction are sampled and retained.

[0015] Optionally, the point cloud memory map is updated based on preset rules, including:

[0016] Generate a new point cloud based on the preset time interval, and remember the location of the point cloud on the map based on the last point cloud.

[0017] If it is within a preset radius threshold centered on the mobile robot but not within the current field of view of the mobile robot, then retain it;

[0018] If the point is within the preset radius threshold and within the current field of view of the mobile robot, the corresponding point cloud on the previously generated point cloud memory map is replaced;

[0019] If it is outside the preset radius threshold, the fading process is performed according to the preset memory time threshold.

[0020] Optionally, the determining and executing an obstacle avoidance strategy based on the distance and angle of the obstacle relative to the mobile robot and the current motion state of the mobile robot to achieve obstacle avoidance includes:

[0021] If the distance between the obstacle and the mobile robot is greater than a preset obstacle avoidance radius, the obstacle avoidance strategy is determined to be a long-distance obstacle avoidance movement, and the long-distance obstacle avoidance movement is executed in combination with the current motion state of the mobile robot;

[0022] If the distance between the obstacle and the mobile robot is less than the preset obstacle avoidance radius, the angle of the obstacle relative to the mobile robot is further determined. If it is greater than the preset angle, the obstacle avoidance strategy is determined to be lateral obstacle avoidance movement. Combined with the current motion state of the mobile robot, the lateral obstacle avoidance movement is executed.

[0023] Optionally, before preprocessing the third depth point cloud data to obtain fifth depth point cloud data, the method further includes:

[0024] synchronously acquiring a depth map of an environment within the field of view of the mobile robot through a laser radar, determining second depth point cloud data in the laser radar coordinate system based on the depth map, and converting the second depth point cloud data into fourth depth point cloud data in the mobile robot coordinate system;

[0025] Fusing the third depth point cloud data and the fourth depth point cloud data to obtain fused depth point cloud data;

[0026] The preprocessing of the third depth point cloud data to obtain fifth depth point cloud data includes:

[0027] The fused depth point cloud data is preprocessed to obtain fifth depth point cloud data.

[0028] According to another aspect of the present application, a system for obstacle avoidance of a mobile robot is provided, wherein the system comprises:

[0029] A perception module, comprising a depth camera and an inertial measurement unit, configured to determine first depth point cloud data in a coordinate system of the depth camera based on a depth map of an environment within a field of view of the mobile robot acquired by the depth camera, convert the first depth point cloud data into third depth point cloud data in a coordinate system of the mobile robot, and pre-process the third depth point cloud data to obtain fifth depth point cloud data, wherein the inertial measurement unit is configured to obtain a current motion state of the mobile robot;

[0030] a planning module, configured to filter ground point cloud data in the fifth depth point cloud data to obtain a point cloud height map, flatten and downsample the point cloud height map to obtain a point cloud memory map, and, when an obstacle is detected based on the point cloud memory map, determine an obstacle avoidance strategy based on a distance and angle of the obstacle relative to the mobile robot and a current motion state of the mobile robot;

[0031] The control module is used to control the mobile robot to execute the obstacle avoidance strategy to achieve obstacle avoidance.

[0032] Optionally, the perception module further includes:

[0033] a laser radar, configured to synchronously acquire a depth map of the environment within the field of view of the mobile robot with the depth camera, and determine second depth point cloud data in the laser radar coordinate system based on the depth map, and convert the second depth point cloud data into fourth depth point cloud data in the mobile robot coordinate system;

[0034] The third depth point cloud data and the fourth depth point cloud data are fused through the perception module to obtain fused depth point cloud data, and the fused depth point cloud data are preprocessed to obtain fifth depth point cloud data.

[0035] Compared with the prior art, the present application provides a method, system and mobile robot for obstacle avoidance for a mobile robot, the method comprising: obtaining a depth map of the environment within the field of view of the mobile robot through a depth camera, and determining first depth point cloud data in the depth camera coordinate system based on the depth map, and converting the first depth point cloud data into third depth point cloud data in the mobile robot coordinate system, pre-processing the third depth point cloud data to obtain fifth depth point cloud data; filtering the ground point cloud data in the fifth depth point cloud data to obtain a point cloud height map, and flattening and down-sampling the point cloud height map to obtain a point cloud memory map, and when it is determined that an obstacle is detected according to the point cloud memory map, determining an obstacle avoidance strategy according to the distance and angle of the obstacle relative to the mobile robot and the current motion state of the mobile robot; and controlling the mobile robot to execute the obstacle avoidance strategy to achieve obstacle avoidance. This application does not need to integrate the depth point cloud into the cost map. It only needs to process the depth point cloud data of the depth camera, accurately and completely retain the necessary environmental data, and detect obstacles based on the obtained point cloud memory map. It can effectively reduce the amount of data processing and improve the efficiency and accuracy of the mobile robot's obstacle detection, thereby improving the real-time and efficiency of obstacle avoidance, which is particularly beneficial for improving the pass rate in narrow spaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0037] Figure 1 A schematic diagram of a method for mobile robot obstacle avoidance according to one aspect of the present application is shown;

[0038] Figure 2 A schematic diagram of a system for mobile robot obstacle avoidance according to another aspect of the present application is shown;

[0039] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION

[0040] The present application is described in further detail below with reference to the accompanying drawings.

[0041] In a typical configuration of each embodiment of the present application, the execution subject of the method, each trusted party of the system and / or each module of the device may include one or more processors (CPU), input / output interface, network interface and memory.

[0042] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0043] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0044] In order to further illustrate the technical means adopted by this application and the effects achieved, the technical solution of this application is clearly and completely described below in combination with the accompanying drawings and preferred embodiments.

[0045] Figure 1 A schematic diagram of a method for obstacle avoidance for a mobile robot according to one aspect of the present application is shown, wherein the method of one embodiment includes:

[0046] S101 acquires a depth map of an environment within a field of view of a mobile robot through a depth camera, determines first depth point cloud data in a coordinate system of the depth camera based on the depth map, converts the first depth point cloud data into third depth point cloud data in a coordinate system of the mobile robot, and pre-processes the third depth point cloud data to obtain fifth depth point cloud data;

[0047] S102: filtering the ground point cloud data in the fifth depth point cloud data to obtain a point cloud height map, and performing flattening and downsampling on the point cloud height map to obtain a point cloud memory map. When an obstacle is detected according to the point cloud memory map, determining an obstacle avoidance strategy based on a distance and angle of the obstacle relative to the mobile robot and a current motion state of the mobile robot;

[0048] S103 controls the mobile robot to execute the obstacle avoidance strategy to achieve obstacle avoidance.

[0049] The method for obstacle avoidance for a mobile robot claimed for protection in this application is implemented and / or executed through a system 100, and the system 100 is deployed in a mobile robot that has the necessary hardware and software environment and can autonomously perform tasks, wherein the mobile robot can accept user input instructions to perform tasks, or start and run pre-programmed program instructions to perform tasks, and can also autonomously formulate and execute tasks based on artificial intelligence technology to complete corresponding services.

[0050] In this embodiment, in step S101, the system 100 can obtain a depth map of the environment within the field of view of the mobile robot through a depth camera, and based on the depth map, determine the first depth point cloud data in the depth camera coordinate system, and convert the first depth point cloud data into third depth point cloud data in the mobile robot coordinate system, and pre-process the third depth point cloud data to obtain fifth depth point cloud data.

[0051] Depth cameras, also known as RGB-D cameras, are based on physical measurement methods and use infrared structured light or ToF (Time of Flight) technology to measure the distance between objects and the camera by actively emitting light to the object and receiving the returned light.

[0052] Among them, the system 100 can obtain a depth map (or initial depth data) of the environment within the field of view of the mobile robot through a depth camera installed on the mobile robot. Then, according to the intrinsic parameter matrix of the depth camera, the obtained depth map can be converted into first depth point cloud data in the coordinate system of the depth camera, and then according to the extrinsic parameter matrix of the depth camera, the first depth point cloud data can be converted into third depth point cloud data in the coordinate system of the mobile robot, and then the third depth point cloud data is pre-processed to obtain fifth depth point cloud data, wherein the point cloud is related to the objects in the environment. For example, assuming that the intrinsic parameter matrix of the depth camera is K and the pixel coordinates of the initial point cloud are P uv , Z represents the depth value of the point cloud, the coordinate in the depth camera coordinate system is P, and the coordinate in the mobile robot coordinate system is P r , the external parameter matrix of the depth camera relative to the mobile robot may include the rotation matrix R and the translation matrix T, that is, the posture of the depth camera relative to the mobile robot can be described by the rotation matrix R and the translation matrix T, then the relevant coordinates have the following relationship:

[0053] ZP uv =KP (1)

[0054] P=RP r +T (2)

[0055] Mobile robots are typically equipped with a single depth camera, whose field of view is typically the same as the depth camera's field of view. The purpose of this application is to provide a technical solution for obstacle avoidance for mobile robots, which only requires detecting obstacles in the direction of travel of the mobile robot. The depth camera's facing direction is typically the same as the direction of travel of the mobile robot, so it is only necessary to obtain a depth map of the environment within the mobile robot's field of view.

[0056] Continuing with this embodiment, in step S102, the system 100 may filter the ground point cloud data in the fifth depth point cloud data to obtain a point cloud height map, and flatten and downsample the point cloud height map to obtain a point cloud memory map. When it is determined that an obstacle is detected based on the point cloud memory map, an obstacle avoidance strategy is determined based on the distance and angle of the obstacle relative to the mobile robot and the current motion state of the mobile robot.

[0057] Depending on the actual application needs, the system 100 may process the obtained fifth depth point cloud data, filtering out invalid point cloud data and ground point cloud data (because ground point cloud data is unlikely to be an obstacle), to obtain a point cloud height map after filtering the point cloud data. The point cloud height map is then flattened and downsampled to obtain a point cloud memory map. When the system 100 determines that an obstacle has been detected based on the obtained point cloud memory map, it may determine an obstacle avoidance strategy based on the distance and angle of the obstacle relative to the mobile robot, as well as the current motion state of the mobile robot.

[0058] Continuing with this embodiment, in step S103 , the system 100 may control the mobile robot to execute the obstacle avoidance strategy to achieve obstacle avoidance.

[0059] Among them, the system 100 issues corresponding program instructions according to the determined obstacle avoidance strategy, controls the mobile robot to execute the obstacle avoidance strategy, and moves according to the traveling mode determined by the obstacle avoidance strategy to achieve obstacle avoidance.

[0060] Through the above embodiment, there is no need to fuse the depth point cloud into the cost map. It is only necessary to process the depth point cloud data of the depth camera, accurately and completely retain the necessary environmental data, and detect obstacles based on the obtained point cloud memory map. This can effectively reduce the amount of data processing, improve the efficiency and accuracy of the mobile robot's obstacle detection, and thus improve the real-time and efficiency of obstacle avoidance, which is particularly beneficial for improving the pass rate in narrow spaces.

[0061] Optionally, in step S102, filtering the ground point cloud data to obtain a point cloud height map includes:

[0062] Based on the ground segmentation algorithm, the ground point cloud is screened out and filtered out to obtain the point cloud height map.

[0063] The system 100 may use a ground segmentation algorithm to process the fifth depth point cloud data, and filter out invalid point cloud data and ground point cloud data.

[0064] For example, first, the points that may be on the ground can be selected according to the fifth depth point cloud data value. i , if z is satisfied min <z i <z max , then the point cloud z i is the possible ground point cloud, which is cached and can be used to solve the extreme mean and fit the ground plane. Then, the least squares method can be used to fit a plane equation:

[0065] z=ax+by+c (3)

[0066] It can be assumed that the error of the above plane equation is:

[0067] ∈=z-(ax+by+c) (4)

[0068] Minimize its residual sum of squares:

[0069]

[0070] Construct the normal equation system Ax=b, where

[0071]

[0072] The best-fitting plane equation can then be solved using LDLT decomposition, a matrix decomposition technique in linear algebra that decomposes a symmetric positive definite matrix into a lower triangular matrix L, a diagonal matrix D, and the transposed matrix of L.

[0073] Corresponding to the point cloud z in the fifth depth point cloud data i Point cloud on the fitting plane

[0074]

[0075] Computable point cloud z i The distance from the fitting plane is: The squared residual is:

[0076]

[0077] The mean of all squared residuals:

[0078]

[0079] Therefore, the ground point cloud in the fifth depth point cloud data can be extracted by the residual square threshold:

[0080]

[0081] Among them, std_dev_ratio_near is the residual square threshold set according to the application scenario.

[0082] The extracted ground point clouds are filtered out from the fifth depth point cloud data, and the remaining ones are considered as non-ground point clouds. The ground height of each non-ground point cloud can be determined according to the distance between it and the fitted ground. According to the ground height of each non-ground point cloud, a point cloud height map can be obtained.

[0083] Optionally, in step S102, the flattening and downsampling processing of the point cloud height map includes:

[0084] The point cloud height map is flattened to convert the point cloud height map into a two-dimensional point cloud plane map, and then a number of point clouds in each preset angle direction are sampled and retained.

[0085] Among them, the point cloud in the point cloud height map is three-dimensional. In order to facilitate subsequent processing, the point cloud height map can be flattened to convert the three-dimensional point cloud into a two-dimensional point cloud to obtain a two-dimensional point cloud plane map, wherein the pixel value of each pixel in the two-dimensional point cloud plane map is the depth value of the point cloud.

[0086] Among them, the mobile robot has a certain shape, and its ground projection has a maximum diameter. If the distance between two depth point clouds is smaller than the diameter of the mobile robot, the mobile robot cannot pass. Therefore, sparse point clouds can meet the obstacle avoidance needs of mobile robots. Therefore, the point cloud on the two-dimensional point cloud plane can be downsampled in each preset angle direction, and the point cloud on the two-dimensional point cloud plane can be thinned, and a number of point clouds in each preset angle direction are retained, and the retained point clouds are composed of a point cloud memory map. For example, in combination with the actual application scenario, retaining 3 point clouds in each degree angle direction can meet the obstacle avoidance needs of the mobile robot, and the three point cloud points closest to the mobile robot can be downsampled (i.e., the three point cloud points with the smallest depth values), and the set of 3 point cloud points in each degree angle direction retained by the downsampling is used as the point cloud memory map.

[0087] As the mobile robot moves and avoids obstacles, the depth camera will obtain a depth map of the environment within its field of view in real time based on a preset frequency, process it to obtain a new two-dimensional point cloud, and dynamically update the point cloud memory map.

[0088] Optionally, the point cloud memory map is updated based on preset rules, including:

[0089] Generate a new point cloud based on the preset time interval, and remember the location of the point cloud on the map based on the last point cloud.

[0090] If it is within a preset radius threshold centered on the mobile robot but not within the current field of view of the mobile robot, then retain it;

[0091] If the point is within the preset radius threshold and within the current field of view of the mobile robot, the corresponding point cloud on the previously generated point cloud memory map is replaced;

[0092] If it is outside the preset radius threshold, the fading process is performed according to the preset memory time threshold.

[0093] Among them, according to the preset time interval, a new depth map is obtained by the depth camera, and after coordinate conversion processing and filtering of the ground point cloud, as well as flattening and downsampling processing, a new point cloud for each preset angle direction is generated. According to the position of the corresponding point cloud in the same preset angle direction on the last point cloud memory map, it can be updated according to the projection diameter of the mobile robot's shape and the radius memory mechanism based on the center point of the mobile robot at the current position (referring to the origin of the mobile robot coordinate system): if it is within the preset radius threshold centered on the mobile robot at the current position but is not within the current field of view of the mobile robot, the point cloud is retained and not updated until the point When the angle direction corresponding to the cloud returns to the field of view and a new point cloud appears within the preset radius threshold, it is replaced by the new point cloud, or exceeds the preset radius threshold and is faded according to the preset memory time threshold; if it is within the preset radius threshold centered on the current position of the mobile robot and within the current field of view of the mobile robot, the corresponding point cloud on the previously generated point cloud memory map is replaced; if it is outside the preset radius threshold centered on the current position of the mobile robot, it is faded according to the preset memory time threshold (if there is a new point cloud in the same angle direction and the update conditions are met, it will be updated to the new point cloud memory map). Among them, for the preset radius threshold centered on the mobile robot, the larger the radius threshold, the longer the processing time and the greater the computing power consumption. However, if the preset radius threshold is too small, the detection effect may be poor, which will affect the obstacle avoidance of the mobile robot. It is necessary to preset a suitable radius threshold based on the complexity of the actual application scenario. Usually, when the application scenario is more complex, the preset radius threshold can be larger.

[0094] Through this optional embodiment, the point cloud memory map can be dynamically updated to better adapt to complex dynamic environments, improve the initiative and safety of the mobile robot, and better reduce the risk of collision with dynamic obstacles.

[0095] Optionally, in step S102, determining and executing an obstacle avoidance strategy based on the distance and angle of the obstacle relative to the mobile robot and the current motion state of the mobile robot to achieve obstacle avoidance includes:

[0096] If the distance between the obstacle and the mobile robot is greater than a preset obstacle avoidance radius, the obstacle avoidance strategy is determined to be a long-distance obstacle avoidance movement, and the long-distance obstacle avoidance movement is executed in combination with the current motion state of the mobile robot;

[0097] If the distance between the obstacle and the mobile robot is less than the preset obstacle avoidance radius, the angle of the obstacle relative to the mobile robot is further determined. If it is greater than the preset angle, the obstacle avoidance strategy is determined to be lateral obstacle avoidance movement. Combined with the current motion state of the mobile robot, the lateral obstacle avoidance movement is executed.

[0098] When the system 100 determines that an obstacle has been detected based on the point cloud memory map, it can determine the distance between the obstacle and the current position of the mobile robot and judge the distance. If the distance is greater than the preset obstacle avoidance radius, it means that the obstacle will not affect the current movement of the mobile robot or the impact is very small. The obstacle avoidance strategy can be determined as long-distance obstacle avoidance movement. The long-distance obstacle avoidance movement can be further executed based on the current motion state (including posture, velocity, acceleration, etc.) of the current position of the mobile robot without taking any special obstacle avoidance action. If the distance is less than the preset obstacle avoidance radius, the angle of the obstacle relative to the current position of the mobile robot can be determined. If the angle is greater than the preset angle, it means that the obstacle has little impact on the mobile robot and only appropriate obstacle avoidance action is required. The obstacle avoidance strategy can be determined as lateral obstacle avoidance movement. Based on the current motion state of the mobile robot, the lateral obstacle avoidance movement can be executed. If the above distance is less than the preset obstacle avoidance radius and the above angle is less than the preset angle, the mobile robot does not need to determine the obstacle avoidance strategy and continues to move normally.

[0099] The preset obstacle avoidance radius is the minimum safe distance that the mobile robot needs to maintain during movement to avoid collision with obstacles. The preset angle is used to indicate the field of view of the mobile robot during movement.

[0100] Optionally, in step S101, before preprocessing the third depth point cloud data to obtain fifth depth point cloud data, the method further includes:

[0101] synchronously acquiring a depth map of an environment within the field of view of the mobile robot through a laser radar, determining second depth point cloud data in the laser radar coordinate system based on the depth map, and converting the second depth point cloud data into fourth depth point cloud data in the mobile robot coordinate system;

[0102] Fusing the third depth point cloud data and the fourth depth point cloud data to obtain fused depth point cloud data;

[0103] The preprocessing of the third depth point cloud data to obtain fifth depth point cloud data includes:

[0104] The fused depth point cloud data is preprocessed to obtain fifth depth point cloud data.

[0105] Among them, the depth point cloud obtained by the depth camera has a relatively close depth distance and cannot accurately obtain depth information at a longer distance. Using only the depth point cloud data of the depth camera may affect the accuracy and stability of obstacle detection in some specific application scenarios. The depth point cloud at a longer depth distance can be obtained by the lidar, and the depth point clouds of the two can complement each other.

[0106] In step S101, the system 100 can also synchronously obtain a depth map of the environment within the field of view of the mobile robot through the installed laser radar. The obtained depth map can then be converted into second depth point cloud data in the laser radar coordinate system based on the internal parameter matrix of the laser radar. The second depth point cloud data can then be converted into fourth depth point cloud data in the mobile robot coordinate system based on the external parameter matrix of the laser radar. The third depth point cloud data and the fourth point cloud data are then fused, and the fused depth point cloud data is pre-processed to obtain fifth depth point cloud data. The fusion processing method can be a simple combination to obtain two sets of depth point cloud data in the mobile robot coordinate system.

[0107] Through this optional embodiment, the depth point cloud data of the depth camera and the depth point cloud data of the lidar are used synchronously to form a complementary relationship. The two sets of depth point cloud data are fused and processed to obtain a point cloud memory map. Obstacles are detected based on the point cloud map, and the detection accuracy and stability are better.

[0108] Figure 2 A schematic diagram of a system for mobile robot obstacle avoidance according to another aspect of the present application is shown, wherein the system in one embodiment includes:

[0109] The perception module 210 includes a depth camera 211 and an inertial measurement unit 212, which is used to determine first depth point cloud data in the depth camera coordinate system based on the depth map of the environment within the field of view of the mobile robot obtained by the depth camera 211, and convert the first depth point cloud data into third depth point cloud data in the mobile robot coordinate system, and pre-process the third depth point cloud data to obtain fifth depth point cloud data. The inertial measurement unit 212 is used to obtain the current motion state of the mobile robot;

[0110] a planning module 220 configured to filter the ground point cloud data in the fifth depth point cloud data to obtain a point cloud height map, flatten and downsample the point cloud height map to obtain a point cloud memory map, and, when an obstacle is detected based on the point cloud memory map, determine an obstacle avoidance strategy based on a distance and angle of the obstacle relative to the mobile robot and a current motion state of the mobile robot;

[0111] The control module 230 is used to control the mobile robot to execute the obstacle avoidance strategy to achieve obstacle avoidance.

[0112] The system in this system embodiment has the same software and hardware environment as the system 100 in the aforementioned method embodiment and / or optional embodiment.

[0113] In this embodiment, the system 100 includes a perception module 210, a planning module 220, and a control module 230. The perception module 210 includes a depth camera 211 and an inertial measurement unit 212. The depth camera 211 can obtain a depth map of the environment within the field of view of the mobile robot. Then, based on the intrinsic parameter matrix of the depth camera 211, the obtained depth map can be converted into first depth point cloud data in the depth camera coordinate system. Then, based on the extrinsic parameter matrix of the depth camera 211, the first depth point cloud data can be converted into third depth point cloud data in the mobile robot coordinate system. The third depth point cloud data is then pre-processed to obtain fifth depth point cloud data, wherein the point cloud is related to objects in the environment. The inertial measurement unit 212 is used to obtain the current motion state of the mobile robot, such as the current position, posture, velocity, acceleration, etc. of the mobile robot, which can represent information of the current motion state.

[0114] Continuing with this embodiment, based on actual application needs, the system 100 can process the obtained fifth depth point cloud data through the planning module 220, filtering out invalid point cloud data and ground point cloud data (because point cloud data on the ground is unlikely to be an obstacle), and obtain a point cloud height map after filtering the point cloud data. The point cloud height map is then flattened and downsampled to obtain a point cloud memory map. When an obstacle is detected based on the obtained point cloud memory map, an obstacle avoidance strategy can be determined based on the distance and angle of the obstacle relative to the mobile robot, as well as the current motion state of the mobile robot.

[0115] Continuing with this embodiment, the system 100 can issue corresponding program instructions through the control module 230 according to the obstacle avoidance strategy determined by the planning module 220, control the mobile robot to execute the obstacle avoidance strategy, and move according to the travel mode determined by the obstacle avoidance strategy to achieve obstacle avoidance.

[0116] Optionally, the perception module 210 further includes:

[0117] The laser radar 213 is configured to synchronously acquire a depth map of the environment within the field of view of the mobile robot with the depth camera 211, and determine second depth point cloud data in the laser radar coordinate system based on the depth map, and convert the second depth point cloud data into fourth depth point cloud data in the mobile robot coordinate system;

[0118] The third depth point cloud data and the fourth depth point cloud data are fused by the perception module 210 to obtain fused depth point cloud data, and the fused depth point cloud data is preprocessed to obtain fifth depth point cloud data.

[0119] Generally speaking, the depth point cloud obtained by a depth camera has a relatively close depth distance and cannot accurately obtain depth information at a longer distance. Using only the depth point cloud data of the depth camera may affect the accuracy and stability of obstacle detection in some specific application scenarios. The depth point cloud at a longer distance can be obtained by using a lidar, and the depth point clouds of the two can complement each other. In this optional embodiment, the perception module 210 of the system 100 also includes a laser radar 213, which can synchronously obtain a depth map of the environment within the field of view of the mobile robot through the depth camera 211 and the laser radar 213. Then, the perception module 210 can convert the respective depth maps into first depth point cloud data in the depth camera coordinate system and second depth point cloud data in the laser radar coordinate system according to the intrinsic parameter matrix of the depth camera 211 and the intrinsic parameter matrix of the laser radar 213, respectively. Then, according to the extrinsic parameter matrix of the depth camera 211 and the extrinsic parameter matrix of the laser radar 213, the first depth point cloud data is converted into third depth point cloud data in the mobile robot coordinate system and the second depth point cloud data is converted into fourth depth point cloud data in the mobile robot coordinate system, respectively. Then, the third depth point cloud data and the fourth point cloud data are fused, and the fused depth point cloud data are preprocessed to obtain fifth depth point cloud data.

[0120] In the above system embodiments and / or optional embodiments, the functions and / or method steps that can be implemented by the various components of the system are the same as those in the aforementioned related method embodiments and are not mentioned here.

[0121] According to another aspect of the present application, a computer-readable medium is provided, which stores computer-readable instructions. The computer-readable instructions can be executed by a processor to implement part or all of the aforementioned method embodiments and / or optional embodiments.

[0122] It should be noted that, in the present application, each method embodiment and / or optional embodiment does not strictly limit the order in which each step is performed, as long as each method embodiment and / or optional embodiment can solve the defects existing in the prior art, achieve the purpose of the invention of the present application, and obtain beneficial effects. In the present application, each method embodiment and / or optional embodiment can be implemented in software and / or a combination of software and hardware. The software program involved in the present application can be executed by a processor to implement the steps or functions of the above-mentioned embodiments. Similarly, the software program of the present application (including related data structures) can be stored in a computer-readable recording medium.

[0123] In addition, part or all of the present application may be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the method and / or technical solution according to the present application through the operation of the computer. The program instructions for invoking the method of the present application may be stored in a fixed or removable recording medium, and / or transmitted via a data stream in a broadcast or other signal-carrying medium, and / or stored in a working memory of a computer device that operates according to the program instructions.

[0124] According to another aspect of the present application, a mobile robot is provided, comprising: a memory for storing computer program instructions and a processor for executing the program instructions, wherein, when the computer program instructions are executed by the processor, the mobile robot is triggered to run part or all of the methods and / or technical solutions of the aforementioned embodiments.

[0125] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be encompassed therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

[0126] In this application, if the terms "upper", "lower", "left", "right", "front", "back", "top", "bottom", "inside", "outside", "middle", "vertical", "horizontal", "lateral", "longitudinal" and the like are used, the orientation and / or positional relationship indicated is based on the orientation and / or positional relationship shown in the accompanying drawings. These terms are mainly used to better describe the present application and its embodiments, and are not used to limit the indicated devices, elements or components to have a specific orientation, or to be constructed and operated in a specific orientation. Moreover, in addition to being used to indicate orientation or positional relationships, some of the above terms can also be used to express other meanings. For example, the term "upper" can also be used to indicate a certain dependency or connection relationship in some cases. For those skilled in the art, the specific meanings of these terms in this application can be understood according to the specific circumstances.

[0127] Furthermore, the terms "installed," "disposed," "equipped with," "connected," and "connected" should be interpreted broadly. For example, they can refer to fixed connections, removable connections, or integral structures; mechanical connections or electrical connections; direct connections or indirect connections through an intermediary; or internal communication between two devices, elements, or components. Those skilled in the art will understand the specific meanings of these terms in this application based on the specific circumstances.

[0128] In addition, the terms "first," "second," etc., are primarily used to distinguish different devices, units, modules, elements, circuits, or components (the specific types and configurations of which may or may not be the same), and are not intended to indicate or imply the relative importance, order, and / or quantity of the indicated devices, units, modules, elements, circuits, or components. Unless otherwise specified, "plurality" means two or more.

[0129] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim can also be implemented by one unit or device through software and / or hardware.

Claims

1. A method for mobile robot to avoid obstacles, characterized in that: The method comprises: Acquire a depth map of an environment within a field of view of the mobile robot through a depth camera, determine first depth point cloud data in the depth camera coordinate system based on the depth map, convert the first depth point cloud data into third depth point cloud data in the mobile robot coordinate system, and pre-process the third depth point cloud data to obtain fifth depth point cloud data; filtering ground point cloud data in the fifth depth point cloud data to obtain a point cloud height map, flattening and downsampling the point cloud height map to obtain a point cloud memory map, and determining an obstacle avoidance strategy based on a distance and angle of the obstacle relative to the mobile robot and a current motion state of the mobile robot when an obstacle is detected based on the point cloud memory map; The mobile robot is controlled to execute the obstacle avoidance strategy to achieve obstacle avoidance.

2. The method according to claim 1, characterized in that The filtering of the ground point cloud data to obtain a point cloud height map includes: Based on the ground segmentation algorithm, the ground point cloud is screened out and filtered out to obtain the point cloud height map.

3. The method according to claim 1, characterized in that The flattening and downsampling processing of the point cloud height map includes: The point cloud height map is flattened to convert the point cloud height map into a two-dimensional point cloud plane map, and then a number of point clouds in each preset angle direction are sampled and retained.

4. The method according to claim 3, characterized in that The point cloud memory map is updated based on preset rules, including: Generate a new point cloud based on the preset time interval, and remember the location of the point cloud on the map based on the last point cloud. If it is within a preset radius threshold centered on the mobile robot but not within the current field of view of the mobile robot, then retain it; If the point is within the preset radius threshold and within the current field of view of the mobile robot, the corresponding point cloud on the previously generated point cloud memory map is replaced; If it is outside the preset radius threshold, the fading process is performed according to the preset memory time threshold.

5. The method according to claim 1, wherein The determining and executing an obstacle avoidance strategy based on the distance and angle of the obstacle relative to the mobile robot and the current motion state of the mobile robot to achieve obstacle avoidance includes: If the distance between the obstacle and the mobile robot is greater than a preset obstacle avoidance radius, the obstacle avoidance strategy is determined to be a long-distance obstacle avoidance movement, and the long-distance obstacle avoidance movement is executed in combination with the current motion state of the mobile robot; If the distance between the obstacle and the mobile robot is less than the preset obstacle avoidance radius, the angle of the obstacle relative to the mobile robot is further determined. If it is greater than the preset angle, the obstacle avoidance strategy is determined to be lateral obstacle avoidance movement. Combined with the current motion state of the mobile robot, the lateral obstacle avoidance movement is executed.

6. The method according to claim 1, characterized in that Before preprocessing the third depth point cloud data to obtain fifth depth point cloud data, the method further includes: synchronously acquiring a depth map of an environment within the field of view of the mobile robot through a laser radar, determining second depth point cloud data in the laser radar coordinate system based on the depth map, and converting the second depth point cloud data into fourth depth point cloud data in the mobile robot coordinate system; Fusing the third depth point cloud data and the fourth depth point cloud data to obtain fused depth point cloud data; The preprocessing of the third depth point cloud data to obtain fifth depth point cloud data includes: The fused depth point cloud data is preprocessed to obtain fifth depth point cloud data.

7. A system for mobile robot obstacle avoidance, characterized in that: The system comprises: A perception module, comprising a depth camera and an inertial measurement unit, configured to determine first depth point cloud data in a coordinate system of the depth camera based on a depth map of an environment within a field of view of the mobile robot acquired by the depth camera, convert the first depth point cloud data into third depth point cloud data in a coordinate system of the mobile robot, and pre-process the third depth point cloud data to obtain fifth depth point cloud data, wherein the inertial measurement unit is configured to obtain a current motion state of the mobile robot; a planning module, configured to filter ground point cloud data in the fifth depth point cloud data to obtain a point cloud height map, flatten and downsample the point cloud height map to obtain a point cloud memory map, and, when an obstacle is detected based on the point cloud memory map, determine an obstacle avoidance strategy based on a distance and angle of the obstacle relative to the mobile robot and a current motion state of the mobile robot; The control module is used to control the mobile robot to execute the obstacle avoidance strategy to achieve obstacle avoidance.

8. The system according to claim 7, characterized in that The perception module also includes: a laser radar, configured to synchronously acquire a depth map of the environment within the field of view of the mobile robot with the depth camera, and determine second depth point cloud data in the laser radar coordinate system based on the depth map, and convert the second depth point cloud data into fourth depth point cloud data in the mobile robot coordinate system; The third depth point cloud data and the fourth depth point cloud data are fused through the perception module to obtain fused depth point cloud data, and the fused depth point cloud data are preprocessed to obtain fifth depth point cloud data.

9. A computer-readable medium, characterized in that Computer-readable instructions are stored thereon, and the computer-readable instructions are executed by a processor to implement part or all of the method according to any one of claims 1 to 6.

10. A mobile robot, characterized in that: The mobile robot comprises: one or more processors; and A memory storing computer-readable instructions, wherein when the computer-readable instructions are executed, the processor is caused to perform part or all of the operations of the method according to any one of claims 1 to 6.

Images