Obstacle avoidance method, device and equipment of robot and storage medium

Abstract

The invention provides an obstacle avoidance method capable of improving the obstacle avoidance capability of a robot, and the method comprises the steps: obtaining point cloud data and image data in a detection range of a sensor, and generating a map based on the point cloud data and the image data; when an obstacle is detected in the detection range of the sensor, increasing a pixel value for a map area where the obstacle is located at a pixel growth rate a / t1; when the map area where the obstacle is located is located within the detection range of the sensor, the pixel value of the map area where the obstacle is located is reduced at a first pixel attenuation rate b / t1, when the map area where the obstacle is located is located outside the detection range of the sensor, the pixel value of the map area where the obstacle is located is reduced at a second pixel attenuation rate c / t2, and b / t1 is larger than c / t2; when the pixel value of the map area where the obstacle is located is larger than or equal to the pixel threshold value d, the obstacle information is sent to the planning and control module.

Description

Technical Field

[0001] This application belongs to the field of robotics, and specifically relates to an obstacle avoidance method, apparatus, device, and storage medium for a robot. Background Technology

[0002] Because the number of sensors on a robot is limited, even if the robot can detect obstacles normally, it will not be able to detect obstacles when they enter the robot's blind spot, which can easily lead to a collision between the robot and the obstacle. Summary of the Invention

[0003] In view of the above-mentioned deficiencies or defects in the prior art, the present invention provides a robot obstacle avoidance method, device, storage medium and electronic device that can improve the robot's obstacle avoidance ability.

[0004] The robot obstacle avoidance method provided by this invention includes: acquiring point cloud data and image data within the sensor detection range, and generating a map based on the point cloud data and image data; when an obstacle is detected within the sensor detection range, increasing the pixel value of the map area where the obstacle is located at a pixel growth rate a / t1; when the map area where the obstacle is located is within the sensor detection range, decreasing the pixel value of the map area where the obstacle is located at a first pixel decay rate b / t1; when the map area where the obstacle is located is outside the sensor detection range, decreasing the pixel value of the map area where the obstacle is located at a second pixel decay rate c / t2, where b / t1 is greater than c / t2, and t1 is the pixel decay interval time when the obstacle is within the sensor detection range, and t2 is the pixel decay interval time when the obstacle is outside the sensor detection range; when the pixel value of the map area where the obstacle is located is greater than or equal to a pixel threshold d, sending the obstacle information to the planning and control module.

[0005] Optionally, the robot obstacle avoidance method provided by the present invention includes: establishing an obstacle location map based on point cloud data acquired by sensors, establishing an obstacle category map based on image data acquired by sensors, wherein the obstacle location map records the location information of the obstacles, and the obstacle category map records the category information of the obstacles; and assigning corresponding pixel growth rate, first pixel decay rate, second pixel decay rate, and pixel threshold to the obstacles on the obstacle location map according to the category of the obstacles.

[0006] Optionally, the robot obstacle avoidance method provided by the present invention includes: obtaining the current operating mode of the robot; and assigning different pixel growth rates, first pixel decay rates, second pixel decay rates, and pixel thresholds to obstacles detected in the operating mode according to the operating mode.

[0007] Optionally, the robot obstacle avoidance method provided by the present invention includes: acquiring the current operating mode of the robot; and assigning different expansion coefficients to obstacles detected in the operating mode according to the operating mode.

[0008] Furthermore, the operation modes include normal, welt, and elevator. The normal mode includes point-to-point navigation scenarios and bow-shaped cleaning scenarios. The elevator mode includes elevator scenarios. The welt mode includes robot edge-following scenarios. The expansion coefficient of the same obstacle in the normal mode is smaller than that in the elevator mode, but larger than that in the welt mode.

[0009] The robot obstacle avoidance device provided by the present invention includes: a map generation module, which acquires point cloud data and image data within the detection range of a sensor, and generates a map based on the point cloud data and image data;

[0010] The system includes a pixel increment module, which increments the pixel value of the map area where the obstacle is located at a pixel increment rate a / t1 when an obstacle is detected within the sensor's detection range; a pixel decrement module, which decrements the pixel value of the map area where the obstacle is located at a first pixel decrement rate b / t1 when the map area is within the sensor's detection range, and decrements the pixel value of the map area where the obstacle is located at a second pixel decrement rate c / t2 when the map area is outside the sensor's detection range, where b / t1 is greater than c / t2, and t1 is the pixel decrement interval time when the obstacle is within the sensor's detection range, and t2 is the pixel decrement interval time when the obstacle is outside the sensor's detection range; and a generation module, which sends obstacle information to the planning and control module when the pixel value of the map area where the obstacle is located is greater than or equal to a pixel threshold d.

[0011] Optionally, the robot obstacle avoidance device provided by the present invention includes: a first acquisition module for acquiring the current operating mode of the robot; and a first assignment module for assigning different pixel growth rates, first pixel decay rates, second pixel decay rates, and pixel thresholds to obstacles detected in the operating mode according to the operating mode.

[0012] Optionally, the robot obstacle avoidance device provided by the present invention includes: a second acquisition module for acquiring the current operating mode of the robot; and a second assignment module for assigning different expansion coefficients to obstacles detected in the operating mode according to the operating mode.

[0013] The electronic device provided by the present invention includes: a processor and a memory, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement the above-described robot obstacle avoidance method.

[0014] The present invention provides a computer-readable storage medium, wherein a computer program is stored therein, and when the computer program is executed, it implements the above-described robot obstacle avoidance method. Attached Figure Description

[0015] The accompanying drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the following detailed description to explain the embodiments of this application, but do not constitute a limitation on the embodiments of this application. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without any inventive effort. In the drawings:

[0016] Figure 1 This is a diagram illustrating an obstacle avoidance scenario where the obstacle is within the sensor's detection range.

[0017] Figure 2 This is a scenario diagram showing obstacle avoidance when the obstacle is outside the sensor's detection range.

[0018] Figure 3 Flowchart of obstacle avoidance method for robots. Detailed Implementation

[0019] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustrative and explanatory purposes only and are not intended to limit the invention. Figure 1 and 2 As shown in the figure, label 1 represents the robot, label 2 represents the detection range of the robot's sensors, and label 3 represents the obstacle.

[0020] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. The robot obstacle avoidance method provided in this embodiment includes the following steps:

[0021] Acquire point cloud data and image data within the sensor's detection range, and generate a map based on the point cloud data and image data;

[0022] When an obstacle is detected within the sensor's detection range, the pixel value of the map area where the obstacle is located increases at a pixel growth rate a / t1.

[0023] When the map area where the obstacle is located is within the sensor's detection range, the pixel value of the map area where the obstacle is located decreases at a first pixel decay rate b / t1. When the map area where the obstacle is located is outside the sensor's detection range, the pixel value of the map area where the obstacle is located decreases at a second pixel decay rate c / t2, where b / t1 is greater than c / t2. Here, t1 is the decay interval time when the obstacle is within the sensor's detection range, and t2 is the decay interval time when the obstacle is outside the sensor's detection range.

[0024] When the pixel value of the map area where the obstacle is located is greater than or equal to the pixel threshold d, the obstacle information is sent to the planning and control module. After receiving the obstacle information, the planning and control module performs corresponding obstacle avoidance processing.

[0025] The obstacle detection method provided in this embodiment sets a higher first pixel attenuation rate for obstacles within the sensor's detection range and a lower second pixel attenuation rate for obstacles outside the sensor's detection range. This allows obstacles to remain outside the sensor's detection range for a longer period, preventing the robot from colliding with them.

[0026] The sensors include depth cameras, LiDAR, and line lasers mounted on the robot. Point cloud data of obstacles is acquired using the depth cameras, LiDAR, and line lasers, while image data of obstacles can also be acquired using the depth cameras. In this embodiment, the robot is equipped with a total of three depth cameras: two located at the front (one forward-facing and one top-facing), and one at the rear (a rear-facing camera). These three depth cameras collect point cloud data around the robot.

[0027] An obstacle will be recorded as soon as it appears in an image frame, but only after the pixel value of the map area where the obstacle is located reaches the pixel threshold will the outline and position information of the obstacle be sent to the planning and control module for subsequent obstacle avoidance processing.

[0028] The first preset time is equal to the second preset time, or the first preset time is less than the second preset time. In this embodiment, the first preset time is, for example, 0.1 seconds, which is equivalent to the interval between the robot acquiring one frame of point cloud data and image data; the second preset time is, for example, 1 second, which means that the pixel value of the grid where the obstacle is located is attenuated once every 10 frames. Generally speaking, the attenuation rate of the obstacle within the sensor's detection range is much greater than the attenuation rate outside the sensor's detection range. If the robot attenuates the pixel value of the grid within the detection range at the same interval, this attenuation value is usually an integer value, that is, b is an integer. However, if the pixel value of the grid outside the detection range is attenuated at the same interval, this attenuation value is likely to be a small value, requiring floating-point operations, increasing the computational power requirements of the processor, and increasing the cost of the robot. This embodiment controls the attenuation value to an integer, that is, c is an integer, by increasing the attenuation interval time of the pixel value of the grid outside the detection range, thus eliminating the need for floating-point operations and reducing the cost of the robot.

[0029] When the pixel value of a grid cell in an obstacle area on the map decreases to zero, the pixel value will not decrease further. Simultaneously, the obstacle category map is reset, effectively eliminating the obstacle's category. For certain special static obstacles, such as doorstops, cliffs, and niches, which are essentially permanent, they do not cause any attenuation to the pixel values ​​of the grid cells in their respective areas, regardless of whether they are within or outside the sensor's detection range.

[0030] d can be set arbitrarily, but once the value of d is determined, a and b need to be set based on d. The settings of a, b, and d must satisfy d greater than (ab), that is, to ensure that obstacles can be identified as quickly as possible after the sensor collects two frames of point cloud data. The magnitude by which d is greater than (ab) is determined by the robot's field of vision and its travel speed. In other words, the product of the robot's speed and the time to detect the obstacle must be less than the robot's field of vision to prevent the robot from colliding with the obstacle.

[0031] The setting of b also depends on the acceptable time for dynamic obstacles to disappear. For example, after detecting a person as an obstacle, how long is expected for the obstacle to disappear from the map after the person leaves?

[0032] The value 'c' is set based on the longest time an obstacle remains outside the sensor's detection range while still within the robot's area of ​​activity. Taking a cleaning robot as an example, this maximum time is determined by the robot's obstacle-avoidance scenario—that is, the scenario where the robot encounters and avoids obstacles while moving around the cleaning area. When moving along edges, the cleaning robot's speed is already slow to avoid hitting walls, and because it needs to avoid obstacles, it will linger near obstacles for a relatively long time. By keeping other obstacles outside the sensor's detection range on the map, collisions with other obstacles are prevented when the robot is moving along edges and avoiding obstacles.

[0033] Taking a shoe cabinet as an obstacle, a is 40, b is 10, c is 20, d is 50, t1 is 0.1 seconds, and t2 is 1 second. The logic for determining the existence of the shoe cabinet is as follows.

[0034] At time t0 (0.1s), the object appears in the blue area of ​​the diagram. The pixel value of this area's grid is 40-10=30. Since the pixel value of this area's grid is less than 50, the object is not considered an obstacle. At time t1 (0.2s), the object is still in its original position. The pixel value of this area's grid is 30+40-10=60. Since the pixel value of this area's grid is greater than 50, the object is considered an obstacle. When the cleaning robot passes through this area, it will either avoid the obstacle or stop there.

[0035] From time t1 to t10 (0.1s to 1s), the object is stationary, so the pixel value of the grid in this area continuously increases to a peak of 245 and will not increase further because the maximum pixel value of a single grid is 255. The object moves away at time t11, and the pixel value of the grid in this area becomes 245 - 10 = 235. Since the object disappears, the pixel value of this area decreases by 10 with each passing frame. At time t30 (3s), 19 frames later, the pixel value of the grid in this area is 235 - 19 * 10 = 45, which is below the object threshold of 50, indicating that there is no obstacle in this area.

[0036] The map includes an obstacle location map and an obstacle category map. Obstacles on the two maps are labeled with numbers. One map records the obstacle category, represented by numbers; the other map records the obstacle pixel growth rate, first pixel decay rate, second pixel decay rate, and pixel threshold, also represented by numbers. If the two sets of numerical information are placed on the same map, mathematical processing is not possible.

[0037] The steps for creating the two maps include: creating an obstacle location map based on point cloud data acquired by the sensor, and creating an obstacle category map based on image data acquired by the sensor. The obstacle location map records the coordinates and outline information of the obstacles, and the obstacle category map records the category information of the obstacles. Based on the category of the obstacles, the obstacles on the obstacle location map are assigned corresponding pixel growth rate, first pixel decay rate, second pixel decay rate, and pixel threshold.

[0038] The obstacle categories are obtained by performing object detection and semantic segmentation on the image data. Object detection can identify the categories of most obstacles and circle them with boxes; semantic segmentation is used to accurately identify carpets to make up for the shortcomings of object detection.

[0039] For example, in edge-fitting scenarios, with t1 being 0.1s and t2 being 1s, the pixel values ​​of a bicycle are 5 and c is 2; the pixel values ​​of a person are 10 and c is 20; and the pixel values ​​of a door closer are 1 and c is 0. This means that once a door is detected, its pixel value will not be attenuated in the blind zone. Similar to doors are cliffs and niches.

[0040] The pixel growth rate, first pixel decay rate, second pixel decay rate, and pixel threshold of the obstacle are also related to the robot's operating mode, specifically including the following steps: obtaining the robot's current operating mode; assigning different pixel growth rates, first pixel decay rates, and second pixel decay rates to the obstacles detected in the operating mode according to the operating mode.

[0041] Taking a cleaning robot as an example, its operating modes include normal, welt, and elevator. The normal mode includes point-to-point navigation and bow-shaped cleaning scenarios. The elevator mode includes elevator-riding scenarios, and the welt mode includes edge-tracing scenarios. When performing a cross-floor cleaning scenario, the cleaning robot first enters the point-to-point navigation scenario, moving from its current location to the elevator waiting point. Then, it enters the elevator-riding scenario, including moving from the waiting point outside the elevator to the elevator boarding point inside (entering the elevator) and from the boarding point to the waiting point (exiting the elevator). Next, the robot enters the edge-tracing and bow-shaped cleaning scenarios, first cleaning the area along the edges (circling the area), then cleaning the interior of the area in a bow-shaped pattern.

[0042] Taking a bicycle as an example, t1 is 0.1s, t2 is 1s, b is 5 and c is 2 in the edge-closed scene; b is 5 and c is 5 in the bow-shaped cleaning scene; b is 20 and c is 20 in the elevator scene. In these three cases, a is 250 and d is 50, that is, as long as a bicycle is found in a frame of point image, the bicycle is determined to be an obstacle.

[0043] The expansion coefficient of obstacles is also related to the robot's operating mode, specifically including the following steps: obtaining the robot's current operating mode; assigning different expansion coefficients to obstacles detected in the operating mode according to the operating mode.

[0044] The expansion coefficient of the same obstacle in normal mode is smaller than that in elevator mode, but larger than that in welt mode. For example, a bicycle expands by 6cm in a close-to-the-edge scene; it expands by 8cm in a bow-shaped cleanup scene; and it expands by 20cm in a ladder scene. For instance, if the original outline of a bicycle is 130cm x 10cm, its expanded size in a ladder scene is 170cm x 50cm.

[0045] This embodiment also provides an obstacle avoidance device, including: a map generation module, which acquires point cloud data and image data within the sensor detection range, and generates a map based on the point cloud data and image data;

[0046] The system includes a pixel increment module, which increments the pixel value of the map area where the obstacle is located at a pixel increment rate a / t1 when an obstacle is detected within the sensor's detection range; a pixel decrement module, which decrements the pixel value of the map area where the obstacle is located at a first pixel decrement rate b / t1 when the map area is within the sensor's detection range, and decrements the pixel value of the map area where the obstacle is located at a second pixel decrement rate c / t2 when the map area is outside the sensor's detection range, where b / t1 is greater than c / t2, and t1 is the pixel decrement interval time when the obstacle is within the sensor's detection range, and t2 is the pixel decrement interval time when the obstacle is outside the sensor's detection range; and a transmission module, which sends the obstacle information to the planning and control module when the pixel value of the map area where the obstacle is located is greater than or equal to a pixel threshold d.

[0047] The robot obstacle avoidance device provided in this embodiment further includes: a first acquisition module for acquiring the current operating mode of the robot; and a first assignment module for assigning different pixel growth rates, first pixel decay rates, second pixel decay rates, and pixel thresholds to obstacles detected in the operating mode according to the operating mode.

[0048] The robot obstacle avoidance device provided in this embodiment further includes: a second acquisition module for acquiring the current operating mode of the robot; and a second assignment module for assigning different expansion coefficients to obstacles detected in the operating mode according to the operating mode.

[0049] The operation modes include normal, welt, and elevator. The normal mode includes point-to-point navigation and bow-shaped cleaning scenarios. The elevator mode includes elevator scenarios. The welt mode includes robot edge-following scenarios. The expansion coefficient of the same obstacle in the normal mode is smaller than that in the elevator mode, but larger than that in the welt mode.

[0050] This embodiment also provides an electronic device, including a processor and a memory. The memory stores instructions, which are loaded and executed by the processor to implement the obstacle avoidance method of the robot described above. The electronic device also includes a communication interface for communicating with external devices and exchanging data. If the memory, processor, and communication interface are implemented independently, they can be interconnected via a bus to communicate with each other. The bus can be categorized as an address bus, data bus, control bus, etc. If the memory, processor, and communication interface are integrated on a single chip, they can communicate with each other through an internal interface.

[0051] This embodiment provides a computer-readable storage medium storing a computer program, which, when executed, implements the obstacle avoidance method of the robot described above.

[0052] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. It is worth noting that the processor can be a processor supporting the Advanced Reduced Instruction Set Computing (RISC) machine (ARM) architecture.

[0053] The memory may include read-only memory and random access memory, and may also include non-volatile random access memory. The memory may be volatile or non-volatile, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0054] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A robot obstacle avoidance method, characterized by, The method comprises: acquiring point cloud data and image data within a sensor detection range, and generating a map based on the point cloud data and the image data; when an obstacle is detected within the sensor detection range, increasing pixel values of a map area where the obstacle is located at a pixel growth rate a / t1; when the map area where the obstacle is located is within the sensor detection range, decreasing the pixel values of the map area where the obstacle is located at a first pixel decay rate b / t1, and when the map area where the obstacle is located is outside the sensor detection range, decreasing the pixel values of the map area where the obstacle is located at a second pixel decay rate c / t2, b / t1 being greater than c / t2, wherein t1 is a pixel decay interval time of the obstacle within the sensor detection range, and t2 is a pixel decay interval time of the obstacle outside the sensor detection range; when the pixel values of the map area where the obstacle is located are greater than or equal to a pixel threshold value d, sending obstacle information to a planning and control module.

2. The robot obstacle avoidance method of claim 1, wherein, The method comprises: establishing an obstacle position map according to point cloud data acquired by a sensor, and establishing an obstacle category map according to image data acquired by the sensor, the obstacle position map recording position information of an obstacle, and the obstacle category map recording category information of the obstacle; according to the category of the obstacle, assigning the obstacle on the obstacle position map with a corresponding pixel growth rate, a first pixel decay rate, a second pixel decay rate, and a pixel threshold value.

3. The robot obstacle avoidance method of claim 1, wherein, The method comprises: acquiring a current working mode of the robot; according to the working mode, assigning an obstacle detected in the working mode with different pixel growth rates, first pixel decay rates, second pixel decay rates, and pixel threshold values.

4. The robot obstacle avoidance method of claim 1 or 2, wherein, The method comprises: acquiring a current working mode of the robot; according to the working mode, assigning an obstacle detected in the working mode with different inflation coefficients.

5. The robot obstacle avoidance method of claim 4, wherein, The working modes include normal, welt, and elevator, the normal mode includes a point-to-point navigation scene and a bow-shaped cleaning scene, the elevator mode includes a lift taking scene, and the welt mode includes a robot edge sticking scene; an inflation coefficient of the same obstacle in the normal mode is smaller than that in the elevator mode and larger than that in the welt mode.

6. A robot obstacle avoidance device, characterized by, The method comprises: a map generation module acquires point cloud data and image data within a sensor detection range, and generates a map based on the point cloud data and the image data; a pixel increase module, when an obstacle is detected within the sensor detection range, increases pixel values of a map area where the obstacle is located at a pixel growth rate a / t1; a pixel decay module, when the map area where the obstacle is located is within the sensor detection range, decreases the pixel values of the map area where the obstacle is located at a first pixel decay rate b / t1, and when the map area where the obstacle is located is outside the sensor detection range, decreases the pixel values of the map area where the obstacle is located at a second pixel decay rate c / t2, b / t1 being greater than c / t2, wherein t1 is a pixel decay interval time of the obstacle within the sensor detection range, and t2 is a pixel decay interval time of the obstacle outside the sensor detection range; and and The sending module sends the obstacle information to the planning and control module when a pixel value of a map area where the obstacle is located is greater than or equal to a pixel threshold d.

7. The robotic barrier avoidance device of claim 6, wherein, Comprising The first obtaining module obtains a current working mode of the robot; And The first assigning module assigns different pixel growth rates, first pixel decay rates, second pixel decay rates and pixel thresholds to the obstacles detected in the working mode according to the working mode.

8. The robotic barrier avoidance device of claim 6, wherein, Comprising The second obtaining module obtains a current working mode of the robot; And The second assigning module assigns different inflation coefficients to the obstacles detected in the working mode according to the working mode.

9. An electronic device, comprising: Comprising: A processor and a memory, the memory storing instructions, the instructions being loaded and executed by the processor to implement the robot obstacle avoidance method according to any one of claims 1 to 5. 10.A computer readable storage medium, the computer readable storage medium storing a computer program, the computer program being executed to implement the robot obstacle avoidance method according to any one of claims 1 to 5.

Images