Obstacle detection method and device, self-propelled robot, and storage medium

Abstract

To provide a method and apparatus for detecting an obstacle, a self-propelled robot, and a storage medium.SOLUTION: This obstacle detection method includes a step (502) that converts the obstacle information collected by an image acquisition device into depth information, a step (504) that converts the depth information into a point cloud map and determines the coordinate data of each reference point on the obstacle, a step (506) that selects the effective analysis range in the height direction from the point cloud map, and a step (508) that analyzes whether the obstacle exists within the effective analysis range based on the coordinate data of each reference point. The method for detecting an obstacle can detect any object in a traveling path of a self-propelled robot, thus breaking through the limitations of existing methods on the type of the obstacle, and simultaneously compensating for defects in the detection height of a laser device, further improving the capabilities of the self-propelled robot in detecting an obstacle, and increasing the obstacle avoidance performance of the self-propelled robot.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] <Related Applications> This application claims priority to Chinese Patent Application No. 202010872327.3 filed on August 26, 2020, the entire disclosure of which is incorporated herein by reference.

[0002] The present disclosure relates to the technical field of self-propelled robots, and in particular to an obstacle detection method and device, a self-propelled robot, and a storage medium. [Background technology]

[0003] With the development of artificial intelligence technology, various intelligent robots have appeared, such as cleaning robots, mopping robots, vacuum cleaners, weeders, etc. These cleaning robots can automatically identify surrounding obstacles during operation and perform evasive actions against the obstacles, which can not only liberate labor and reduce labor costs, but also improve cleaning efficiency.

[0004] However, the image collecting device of the existing cleaning robot can only identify a limited type of obstacle, and usually, the captured obstacle image is compared with images stored in a past database, and if the comparison is successful, the captured object is identified as a specific type of obstacle such as a shoe. Furthermore, the cleaning robot may use a laser device to identify obstacles. However, the laser device is usually disposed on the upper surface of the cleaning robot, and the laser device transmits and receives signals horizontally to identify obstacles around the cleaning robot, and since the cleaning robot itself has a height and the laser device also has a height, the laser device can only detect obstacles higher than the height of the cleaning robot, and may erroneously come into contact with a low-height obstacle, resulting in a poor usability. Summary of the Invention

[0005] In view of this, embodiments of the present disclosure provide an obstacle detection method and apparatus, a self-propelled robot, and a storage medium, so that the self-propelled robot can avoid any obstacles within a certain height range.

[0006] An embodiment of the present disclosure provides an obstacle detection method applicable to a self-propelled robot, the method including the steps of converting obstacle information collected by an image collection device into depth information, converting the depth information into a point cloud map and determining coordinate data of each reference point of the obstacle, selecting an effective analysis range in the height direction from the point cloud map, and analyzing whether an obstacle exists within the effective analysis range based on the coordinate data of each reference point within the effective analysis range.

[0007] Optionally, the step of analyzing whether an obstacle exists within the effective analysis range based on the coordinate data of each of the reference points within the effective analysis range includes a step of converting the 3D coordinate data of each of the reference points within the effective analysis range into 2D coordinate data, and a step of analyzing whether an obstacle exists within the effective analysis range based on the 2D coordinate data of each of the reference points within the effective analysis range.

[0008] Optionally, the step of analyzing whether an obstacle exists within the effective analysis range based on the 2D coordinate data of each of the reference points within the effective analysis range includes the steps of obtaining a probability map formed during the traveling process of the self-propelled robot, filtering noise point coordinate data in the 2D coordinate data of each of the reference points based on the probability map, and analyzing whether an obstacle exists within the effective analysis range based on the filtered 2D coordinate data of each of the reference points.

[0009] Optionally, the step of filtering noise point coordinate data in the 2D coordinate data of each of the reference points based on the probability map includes the steps of: acquiring position coordinate data of an obstacle in the probability map when the probability value of the obstacle in the probability map is greater than a preset probability threshold; and comparing the position coordinate data of the obstacle with the 2D coordinate data of each of the reference points, and filtering the 2D coordinate data when a distance between the position coordinate data of the obstacle and the 2D coordinate data of each of the reference points is greater than a predetermined value.

[0010] Optionally, the step of analyzing whether an obstacle exists within the effective analysis range based on the 2D or 3D coordinate data of each of the reference points within the effective analysis range includes a step of clustering each of the reference points within the effective analysis range and treating all reference points whose distance from the coordinate data of adjacent reference points is less than a second predetermined value as one region, and a step of determining the size of each region and determining that an obstacle exists within the effective analysis range if the size is equal to or greater than a third predetermined value.

[0011] Optionally, the size of the region includes at least one of the area / volume of the region, the diameter of the region, and the minimum distance across the geometric center of the region.

[0012] Optionally, after analyzing whether an obstacle exists within the effective analysis range, the method further includes a step of determining whether a size range of the obstacle is less than a predetermined threshold when an obstacle exists within the effective analysis range, and a step of controlling the self-propelled robot to reduce a side brush rotation speed within a predetermined distance range from the obstacle when the size range of the obstacle is less than a predetermined threshold.

[0013] An embodiment of the present disclosure provides an obstacle detection method applicable to a self-propelled robot, which includes the steps of converting obstacle information collected by an image collection device into depth information, converting the depth information into a point cloud map and selecting an effective analysis range from the point cloud map, and when it is determined that an obstacle exists within the effective analysis range, performing navigation according to the size of the obstacle.

[0014] Optionally, the step of navigating according to the size of the obstacle specifically includes a step of controlling the self-propelled robot to reduce a side brush rotation speed within a preset distance range from the obstacle when the size of the obstacle is less than a preset threshold.

[0015] Optionally, the step of navigating according to the size of the obstacle specifically includes a step of determining an obstacle avoidance distance according to the size of the obstacle or the range in which the obstacle is located, and controlling the self-propelled robot to start execution of an obstacle avoidance policy at the obstacle avoidance distance.

[0016] Optionally, after converting the depth information into a point cloud map, further comprising: determining coordinate data for each of the reference points on the obstacle; A step of clustering the reference points within the effective analysis range, and classifying all reference points whose distance from the coordinate data of adjacent reference points is less than a second predetermined value into one region; The method further includes determining the size of each area, and determining that an obstacle exists within the effective analysis range if the size of each area is equal to or greater than a third predetermined value.

[0017] Optionally, the size of the region includes at least one of the area / volume of the region, the diameter of the region, and the minimum distance across the geometric center of the region, the height of the region.

[0018] Optionally, the step of performing navigation according to the size of the obstacle specifically includes the step of executing an obstacle avoidance policy if it is determined that the height of the obstacle is greater than a preset height.

[0019] An embodiment of the present disclosure provides an obstacle detection device applicable to a self-propelled robot, which includes: a conversion unit used to convert obstacle information collected by an image collection device into depth information; a conversion unit used to convert the depth information into a point cloud map and determine coordinate data of each reference point on the obstacle; a determination unit used to select an effective analysis range in a height direction from the point cloud map; and an analysis unit used to analyze whether an obstacle exists within the effective analysis range based on the coordinate data of each reference point within the effective analysis range.

[0020] An embodiment of the present disclosure provides a self-propelled robot comprising a processor and a memory, wherein the memory stores computer program commands executable by the processor, and when the computer program commands are executed by the processor, steps of the method described in any one of the above are performed.

[0021] An embodiment of the present disclosure provides a non-transitory computer-readable storage medium having computer program instructions stored thereon, the computer program instructions being invoked and executed by a processor to perform the steps of the method according to any one of the above. Effect of the Invention

[0022] Compared with the prior art, the present invention has at least the following technical advantages:

[0023] The embodiments of the present disclosure provide an obstacle detection method and device, a self-propelled robot, and a storage medium, and the obstacle detection method uses a depth image within a certain height range obtained by the self-propelled robot itself to perform processes such as coordinate transformation and data clustering on the depth image to accurately determine whether an obstacle exists in the current depth image. This method can detect all objects in the driving path of the self-propelled robot, breaking through the limitations on the types of obstacles in conventional methods, while improving the detection height defects of the laser device, further improving the obstacle detection ability of the self-propelled robot, and improving the obstacle avoidance performance of the self-propelled robot.

[0024] In order to more clearly describe the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly describe the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can obtain other drawings based on these drawings without creative labor. [Brief description of the drawings]

[0025] [Figure 1] FIG. 2 is a schematic diagram of an application scenario provided by an embodiment of the present disclosure. [Diagram 2] FIG. 1 is a perspective view of a self-propelled robotic structure provided by an embodiment of the present disclosure. [Diagram 3] FIG. 2 is a top view of a self-propelled robotic structure provided by an embodiment of the present disclosure. [Figure 4] FIG. 2 is a bottom view of a self-propelled robotic structure provided by an embodiment of the present disclosure. [Diagram 5] 1 is a flowchart of a self-propelled robot obstacle detection method provided by an embodiment of the present disclosure. [Figure 6] FIG. 2 is a structural diagram of the binocular distance measuring principle provided by an embodiment of the present disclosure. [Figure 7] FIG. 2 is a schematic diagram of a binocular ranging geometry provided by an embodiment of the present disclosure. [Figure 8]FIG. 2 is a schematic diagram of a plane coordinate system provided by an embodiment of the present disclosure. [Figure 9] FIG. 2 is a schematic diagram of a three-dimensional coordinate system provided by an embodiment of the present disclosure. [Figure 10] FIG. 2 is a structural schematic diagram of a clustering method provided by an embodiment of the present disclosure; [Figure 11] 1 is a flowchart of a self-propelled robot obstacle detection method provided by another embodiment of the present disclosure. [Figure 12] 1 is a structural schematic diagram of an obstacle detection device provided by an embodiment of the present disclosure; [Figure 13] FIG. 2 is a structural schematic diagram of an obstacle detection device provided according to another embodiment of the present disclosure. [Figure 14] FIG. 2 is a schematic diagram of the electronic structure of a robot provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] In order to make the objectives, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are only some embodiments of the present disclosure, but not all of the embodiments. Based on the embodiments of the present disclosure, other embodiments obtained by those skilled in the art without creative labor are all included in the protection scope of the present disclosure.

[0027] The embodiments of the present disclosure provide possible application scenarios. In some embodiments, the application scenarios include an automatic cleaning device 100, such as a self-propelled robot, a mopping robot, a vacuum cleaner, a weeder, etc. In this embodiment, as shown in FIG. 1, a self-propelled robot for home use is taken as an example. During the operation of the self-propelled robot, an image collector provided at the tip of the self-propelled robot captures a forward field of view image in real time, and the presence or absence of an obstacle 200 is determined by analyzing the field of view image, and the travel path of the self-propelled robot is controlled according to the recognition result. In this embodiment, the robot may be equipped with one or more image collectors for capturing images during the travel path, and the robot may be controlled by a touch-sensitive display or a mobile terminal for receiving an operation command input by a user. The self-propelled robot may be equipped with various sensors, such as sensing devices such as a buffer, cliff sensor, ultrasonic sensor, infrared sensor, geomagnetic sensor, accelerometer, gyroscope, or odograph, and the robot may be equipped with a wireless communication module such as a WIFI module or a Bluetooth (registered trademark) module, and may be connected to an intelligent terminal or server to receive operation commands from the intelligent terminal or server via the wireless communication module.

[0028] 2, the automatic cleaning apparatus 100 may move over a ground surface relative to various combinations of movement along three mutually perpendicular axes defined by the body 110: a front-to-rear axis X, a lateral axis Y, and a central longitudinal axis Z. A driving direction forward along the front-to-rear axis X is designated as "forward," and a driving direction backward along the front-to-rear axis X is designated as "reverse." The direction of the lateral axis Y is a direction extending between the right and left wheels of the robot substantially along an axis defined by the center point of the drive wheel module 141.

[0029] The automatic cleaning device 100 can rotate around the Y-axis. When the forward moving part of the automatic cleaning device 100 tilts upward and the backward moving part tilts downward, it is called "pitch up", and when the forward moving part of the automatic cleaning device 100 tilts downward and the backward moving part tilts upward, it is called "pitch down". Furthermore, the robot 100 may rotate around the Z-axis. When the automatic cleaning device 100 tilts to the right of the X-axis in the forward moving direction of the automatic cleaning device 100, it is called "clockwise", and when the automatic cleaning device 100 tilts to the left of the X-axis, it is called "counterclockwise".

[0030] As shown in FIG. 3, the automatic cleaning device 100 includes a device body 110, a sensing system 120, a control system, a driving system 140, a cleaning system, an energy system, and a man-machine interaction system 180.

[0031] The device body 110 has a forward moving portion 111 and a rearward moving portion 112 and may have a generally circular shape (both front and rear are circular) or may be other shapes such as, but not limited to, a generally D-shaped shape with a rectangular front and a circular rear, a rectangular shape with a rectangular front and a rectangular rear, or a square shape.

[0032] 3, the sensing system 120 includes a positioning device 121 on the device body 110, a collision sensor and a proximity sensor provided in a buffer 122 of the forward portion 111 of the device body 110, a cliff sensor provided at the bottom of the device body, and sensing devices such as a geomagnetic sensor, an accelerometer, a gyroscope (Gyro), and an odograph (ODO, odograph) provided inside the device body, and is configured to provide various position information and motion state information of the device to the control system 130. The positioning device 121 includes, but is not limited to, a camera, a laser direct structuring (LDS), and the like.

[0033] As shown in FIG. 3, the forward moving part 111 of the equipment body 110 is equipped with a buffer 122. During the cleaning process, when the driving wheel module 141 pushes the robot to move on the ground, the buffer 122 uses a sensor system, such as an infrared sensor, installed thereon to detect one or more events in the traveling path of the automatic cleaning device 100. Based on the events detected by the buffer 122, such as an obstacle, a wall, etc., the automatic cleaning device 100 controls the driving wheel module 141 so that the automatic cleaning device 100 responds to the event, for example, moves away from the obstacle.

[0034] The control system 130 is disposed on a circuit board in the device body 110 and includes a computing processor such as a central processing unit or an application processor that communicates with a non-transitory memory such as a hard disk, a flash memory, or a random access memory. The application processor uses a localization algorithm such as Simultaneous Localization And Mapping (SLAM) according to obstacle information fed back from a laser distance measuring device to draw an instant map of the environment in which the robot is located. In addition, the control system 130 refers to distance information and speed information fed back from sensing devices such as sensors, cliff sensors, geomagnetic sensors, accelerometers, gyroscopes, and odographs provided on the buffer 122 to comprehensively judge the current working state, position, and current attitude of the self-mobile robot, such as crossing a threshold, moving on a carpet, localization on a cliff, getting stuck at the top or bottom, a dust box full, and picking up, and further provides specific next-operation policies for different cases, so that the robot's work can better meet the owner's requirements and provide a better user experience.

[0035] As shown in FIG. 4, the drive system 140 can operate the robot 100 to move across the ground based on a drive command having distance and angle information (e.g., x, y and θ components). The drive system 140 includes a drive wheel module 141, which can simultaneously control the left and right wheels, and preferably includes a left drive wheel module and a right drive wheel module, respectively, for more precise control of the equipment motion. The left drive wheel module and the right drive wheel module are opposed along a horizontal axis defined by the body 110. To enable the robot to move more stably or strongly on the ground, the robot includes one or more driven wheels 142, which include, but are not limited to, universal wheels. The drive wheel module includes a running wheel, a drive motor and a control circuit for controlling the drive motor, and the drive wheel module may further connect a circuit for measuring the drive current, an odograph. The drive wheel module 141 is detachably connected to the body 110 for easy disassembly and repair. The drive wheels are rotatably mounted and movably secured to the robot body 110 and may include an offset drop suspension system that receives a spring offset that is downwardly offset away from the robot body 110. The spring offset allows the drive wheels to maintain contact and traction with the ground with a constant force while the cleaning element of the automatic cleaning apparatus 100 also contacts the ground 10 with a constant pressure.

[0036] The cleaning system may be a dry cleaning system and / or a wet cleaning system. As a dry cleaning system, the main cleaning function is obtained from the cleaning system 151 formed by the roller brush, the dust box, the fan, the air outlet and the connecting parts between them. The roller brush that interferes with the ground sweeps the dirt on the ground, rolls it to the front of the suction port between the roller brush and the dust box, and then the dirt is sucked into the dust box by the suction gas generated by the fan that passes through the dust box. The dry cleaning system may further include a side brush 152 having a rotation axis that is angled with the ground to move the dirt to the roller brush area of ​​the cleaning system.

[0037] The energy system includes a rechargeable battery such as a nickel-metal hydride battery or a lithium battery. The rechargeable battery is connected to a charging control circuit, a battery pack charging temperature detection circuit, and a battery voltage drop monitoring circuit. The charging control circuit, the battery pack charging temperature detection circuit, and the battery voltage drop monitoring circuit are connected to a one-chip microcomputer control circuit. The host is charged by connecting it to the charging pile via the charging electrode located on the side or bottom of the main body. If dust adheres to the bare charging electrode, the plastic body around the electrode will melt and deform due to the charge accumulation effect during the charging process, and the electrode itself will also deform, making it impossible to continue charging normally.

[0038] The man-machine interaction system 180 includes buttons on the host panel, which are used for the user to select functions, and further includes a display screen and / or an indicator and / or a speaker. The display screen, indicator, and speaker are used to present the current status or function options of the device to the user. The man-machine interaction system 180 may further include a mobile phone client program. In the case of a path navigation type automatic cleaning device, the mobile phone client can display an environment map of the device and the location of the device to the user, providing the user with richer and more personalized function items.

[0039] An embodiment of the present disclosure provides an obstacle detection method that is applied to the above-mentioned self-propelled robot, and involves obtaining a frontal image using an image collection device provided on the self-propelled robot body, analyzing the image, determining whether an obstacle is present in the image, and controlling the travel path of the self-propelled robot according to the recognition result.

[0040] As shown in FIG. 5, the obstacle detection method is applied to a self-propelled robot and specifically includes the following method steps, where the method step numbers are not necessarily in order:

[0041] Step S502: Convert the obstacle information collected by the image collecting device into depth information. The image collecting device can be, but is not limited to, a camera.

[0042] In 3D computer graphics and computer vision, the depth information of an object is an image or image channel that contains information about the distance from the surface of a scenario object to the viewpoint. The grayscale value of each pixel point of the depth image may be used to represent the distance between a point in the scenario and the image acquisition device.

[0043] For example, a method of acquiring depth information is binocular stereo vision, in which two cameras separated by a certain distance simultaneously acquire two images of the same scenario, and a three-dimensional matching algorithm is used to find corresponding pixel points in the two images, and the disparity information is calculated according to the triangulation principle. The disparity information can be transformed and used to display the depth information of objects in the scenario. Based on the three-dimensional matching algorithm, it is also possible to acquire the depth information of the same scenario by taking a group of images of the same scenario from different angles.

[0044] Figure 6 is a diagram of the principle of binocular stereo vision, where OL and OR are the optical centers of the left and right cameras, and their optical axes and respective shooting planes are shown in Figure 6. It is assumed that the internal and external parameters of the binocular cameras are the same, the focal length is f, the distance between the optical centers (baseline) is B, and the binocular cameras are on the same plane, that is, the Y coordinates of their projection centers are equal. The shooting points of the spatial point P(x, y, z) on the binocular cameras at the same time are respectively P 左 , P 右 Let us assume that.

[0045] As shown in FIG. 7, the formula 1 is the law of similarity of triangles.

[0046]

number

[0047] After transforming equation 1, we obtain the following equations 2 and 3.

[0048]

number

[0049]

number

[0050] As can be seen from the above equation, once the camera focal length f, the left and right camera baselines b (obtained using prior information or camera calibration), and the disparity d (the relationship between the pixel point (xl, yl) of the left camera and the corresponding point (xr, yr) of the right camera) are determined, the distance (depth) z between the spatial point P and the self-propelled robot can be calculated.

[0051] Step S504: Convert the depth image into a point cloud map, and determine 3D coordinate data of each reference point of the obstacle. In a broad sense, the reference point refers to a point on the obstacle that can detect the distance, and may be, but is not limited to, a boundary point, a feature point, etc. on the obstacle.

[0052] Each pixel of the depth image satisfies the pixel coordinate system and the image coordinate system, and each point in the point cloud map satisfies the world coordinate system. The specific conversion principle of converting the depth image into the point cloud map is described below.

[0053] During the camera imaging process, there are four coordinate systems: pixel coordinate system, image coordinate system, camera coordinate system and world coordinate system. As shown in Figure 8, the pixel coordinate system has the upper left corner of the image as the origin O0, the horizontal coordinates (u, v) respectively indicate the column number and row number of the pixel in the image, the origin of the image coordinate system is the intersection point of the optical axis of the camera and the image plane, which is generally the center of the image plane, the x axis is parallel to the u axis, and the y axis is parallel to the v axis. Pixel coordinate system O 0-uv Let the coordinates of O1 in be (u0, v0), and dx and dy be the physical dimensions of a unit pixel on the horizontal and vertical axes, then the relationship expressed by the following equation 4 exists between the two coordinate systems.

[0054]

number

[0055] A homogeneous matrix is ​​expressed as in the following equation 5.

[0056]

number

[0057] Furthermore, it is transformed into the following equation 6.

[0058]

number

[0059] FIG. 9 shows the imaging principle of the camera. O is the optical center of the camera, and z c is the optical axis of the camera, and the intersection point of the optical axis and the image plane is O1. Also, in the coordinate system Ox c y c z c is the camera coordinate system, Ow-x w y w z w is the world coordinate system, and the distance between OO1 is the focal length f of the camera.

[0060] The conversion relationship from the camera coordinate system to the image coordinate system is as shown in the following formula 7.

[0061]

number

[0062] A homogeneous matrix is ​​expressed as in the following equation 8.

[0063]

number

[0064] The transformation relationship between the world coordinate system and the camera coordinate system is as shown in the following formula 9.

[0065]

number

[0066] R is the rotation matrix and T is the translation matrix.

[0067] It should be noted that the above explanation of the principle is used only for introductory description and is not to be understood as the only solution for limiting the conversion principle.

[0068] Step S506: Select an effective analysis range in the height direction from the point cloud map.

[0069] Typically, the height of a self-propelled robot is 10 cm or more, so the laser radar device installed on the top of the self-propelled robot can only detect obstacles that are 10 cm or more in height. Since the height at which a self-propelled robot can overcome obstacles is 1 to 2 cm, the selected height range is preferably 3 to 9 cm. In this case, the camera's viewing direction faces the direction of movement of the self-propelled robot when in operation, so a blind zone for the laser radar may be detected. In this case, the ideal target obstacle can be accurately determined by analyzing 3D data points in the height range of 3 to 9 cm of the obtained point cloud map.

[0070] Step S508: Analyze whether an obstacle exists within the effective analysis range based on the coordinate data of each of the reference points within the effective analysis range.

[0071] In a selectable specific embodiment, the step of analyzing whether an obstacle exists within the effective analysis range based on the coordinate data of each of the reference points within the effective analysis range includes the following method steps:

[0072] Step S508-2: If the coordinate data is 3D coordinates, the 3D coordinate data of each of the reference points within the effective analysis range can be converted into 2D coordinate data. The specific conversion process is as described above, and the detailed description is omitted here. By converting the 3D point cloud data into 2D coordinate data, the amount of data processing can be effectively reduced and the data processing efficiency can be improved.

[0073] Step S508-4: Analyze whether an obstacle exists within the effective analysis range based on the 2D coordinate data of each of the reference points within the effective analysis range.

[0074] In a selectable specific embodiment, the step of analyzing whether an obstacle exists within the effective analysis range based on the 2D coordinate data of each of the reference points within the effective analysis range includes the following substeps:

[0075] Step S508-4-2: A probability map formed during the traveling process of the self-propelled robot is obtained.

[0076] The self-propelled robot performs obstacle detection in real time while traveling. When an obstacle is detected, the probability value of the corresponding estimated position on the map is increased, and for parts where no obstacle is detected, the corresponding map area is calculated and the probability value within this area is reduced.

[0077] The probability map contains current estimates of the mean and variance of the system state vector: the former indicates the nominal location of the objects in the map under a global reference system, and the latter indicates the uncertainty of each reference point in the map, and the degree to which these uncertainties are interdependent.

[0078] The probability map uses multiple historical experiences during the self-propelled robot's historical travel trajectory process to determine whether an obstacle exists at that position, mark the position whether an obstacle exists, and at the same time mark the probability value of that position being an obstacle. For example, the probability value of the existence of an obstacle at that position is 80% or 30%. The probability map is stored in the self-propelled robot's own memory, and the probability data and position data are continuously updated as the self-propelled robot moves.

[0079] Step S508-4-4: Based on the probability map, filter the noise point coordinate data in the 2D coordinate data of each of the reference points, and analyze whether an obstacle exists within the effective analysis range based on the filtered 2D coordinate data of each of the reference points.

[0080] As an optional specific embodiment, the step of filtering noise point coordinate data in the 2D coordinate data of each of the reference points based on the probability map includes the following substeps:

[0081] Step S508-4-4-2: When the probability value of the obstacle in the probability map is greater than a preset probability threshold, obtain the position coordinate data of the obstacle in the probability map.

[0082] In the probability map, the selected obstacle target is an obstacle target whose probability value is greater than a preset threshold. For example, if the probability of a certain obstacle target A is 50% and the probability of another obstacle target B is 60%, then if the probability threshold is set to 55%, the coordinate data of obstacle target B will be considered and obstacle target A will be ignored. When the obstacle targets in the probability map to be selected are determined, the coordinate data of each obstacle target can be obtained, for example, as represented as B(x1, y1).

[0083] Step S508-4-4-4: The position coordinate data of the obstacle is compared with the 2D coordinate data of each of the reference points, and if the distance between the position coordinate data of the obstacle and the 2D coordinate data of each of the reference points is greater than a predetermined value, the 2D coordinate data is filtered.

[0084] For example, the coordinate value of the obstacle target B(x1, y1) selected from the probability map is compared with the coordinate value C(x0, y0) of a certain point C in the 2D data obtained by the binocular camera, and if the distance between the obstacle position coordinate data and the 2D coordinate data of each reference point is greater than a predetermined value, the 2D coordinate data is filtered. That is, if the distance between B(x1, y1) and C(x0, y0) is sufficiently far, the 2D coordinate data is filtered, and in this case, the point C(x0, y0) is determined to be noise. Note that the predetermined value may be determined according to experimental data and is not limited here.

[0085] As an optional specific embodiment, the step of analyzing whether an obstacle exists within the effective analysis range based on the 2D coordinate data of each of the reference points within the effective analysis range includes the following steps:

[0086] Within the effective analysis range, the reference points are clustered, and all reference points whose distance from the coordinate data of adjacent reference points is less than a second predetermined value are treated as one region. The size of each region is determined, and if the size is equal to or greater than a third predetermined value, it is determined that an obstacle exists within the effective analysis range.

[0087] 10, for example, within a height distance range of 3 to 9 cm, the distance of the coordinate data between each reference point within this range and adjacent points in the image is calculated, and all reference points within this range with distance values ​​smaller than a second predetermined value are set as one region, for example, regions D, E, F, and G. Points H and K exceed the second predetermined value, so they can be determined as noise points. Note that the second and third predetermined values ​​are set according to the pixel size and are not limited here.

[0088] Next, the size of each region is determined, and if the size is equal to or larger than a third predetermined value, it is determined that an obstacle exists within the effective analysis range. The size includes at least one of the area of ​​each region, the diameter of each region (if the regions form an approximately circular area, the diameter of the region is obtained), and the minimum distance across the geometric center of each region. As shown in Figure 10, it is determined that regions D, E, F, and G belong to the obstacle, each point in the region belongs to a point on the obstacle surface, and points H and K are non-obstacles.

[0089] As another selectable specific embodiment, the step of analyzing whether an obstacle exists within the effective analysis range based on the coordinate data of each of the reference points within the effective analysis range includes the following steps:

[0090] Within the effective analysis range, the reference points are clustered, and all reference points whose distance to the 3D coordinate data of adjacent points is less than a fourth predetermined value are treated as one three-dimensional region. The size of each three-dimensional region is determined, and if the size is greater than or equal to a fifth predetermined value, it is determined that an obstacle exists within the effective analysis range.

[0091] For example, within a height distance range of 3 to 9 cm, the distance between each reference point within the range and the coordinate data of adjacent points within the 3D space range is calculated, and all reference points whose distance value is smaller than a fourth predetermined value are treated as one region. After processing all coordinate points, the size of each three-dimensional region is determined, and if the size is equal to or larger than a fifth predetermined value, it is determined that an obstacle exists within the effective analysis range. The size includes at least one of the volume of each region, the diameter of each region (if the regions form an approximately circular region, the diameter of the region is obtained), the minimum distance across the geometric center of each region, and the height of the region. Note that the fourth and fifth predetermined values ​​are set according to the pixel size and are not limited here.

[0092] In a specific embodiment, after analyzing whether there is an obstacle within the effective analysis range, the method includes the following steps:

[0093] Step S510: When an obstacle is present within the effective analysis range, determine whether the size range of the obstacle is less than a preset threshold, and when the size range of the obstacle is less than the preset threshold, control the self-propelled robot to reduce the side brush rotation speed within a preset distance range from the obstacle.

[0094] According to the steps of the above method, it is possible to judge whether it is an obstacle, and further, the size of the obstacle is determined according to the size of the area, and a preset threshold is set by referring to experimental data. The threshold may be the area or the amount of pixels, but is not limited to a specific threshold. If the threshold is exceeded, it is determined to be a large-sized obstacle, and otherwise it is determined to be a small-sized obstacle. For a small-sized obstacle, the self-propelled robot is controlled not to perform obstacle avoidance, but to reduce the rotation speed of the side brush within a preset distance range from the obstacle, so as not to blow away the small obstacle. For a large-sized obstacle, the self-propelled robot is controlled to perform obstacle avoidance operation within a preset distance range from the obstacle.

[0095] An embodiment of the present disclosure provides an obstacle detection method, which uses a depth image within a certain height range acquired by the self-propelled robot itself to perform processes such as coordinate transformation and data clustering on the depth image to accurately determine whether an obstacle exists in the current depth image. This method detects all objects in the driving path of the self-propelled robot, overcomes the limitations on obstacle types in the conventional method, and at the same time improves the detection height defect of the laser device, further improves the obstacle detection ability of the self-propelled robot, and improves the obstacle avoidance performance of the self-propelled robot.

[0096] As shown in FIG. 11 , another embodiment of the present disclosure provides an obstacle detection method, which is applied to a self-propelled robot, including the following method steps:

[0097] Step S1102: Convert the obstacle information collected by the image collecting device into depth information.

[0098] The process of this step refers to step S502 in the above embodiment, and a detailed description thereof will be omitted here.

[0099] Step S1104: Convert the depth information into a point cloud map, and select a valid analysis range from the point cloud map.

[0100] The process of converting the depth information into a point cloud map refers to step S504 of the above embodiment, and the process of selecting an effective analysis range from the point cloud map refers to step S5011 of the above embodiment, and detailed description thereof will be omitted here.

[0101] Step S1108: Determine whether an obstacle exists within the effective analysis range, and perform navigation according to the size of the obstacle.

[0102] In one embodiment, the step of determining whether an obstacle is present within the valid analysis range comprises the following sub-steps:

[0103] Step S1108-1: Within the valid analysis range, the reference points are clustered.

[0104] Step S1108-2: All reference points whose distance from the coordinate data of the adjacent reference points is smaller than a second predetermined value are treated as one region.

[0105] Step S1108-3: The size of each area is determined.

[0106] Step S1108-4: If the size of the region is equal to or greater than a third predetermined value, it is determined that an obstacle exists within the valid analysis range. Optionally, the size includes at least one of the area / volume of each region, the diameter of each region, and the minimum distance across the geometric center of each region.

[0107] In one embodiment, the step of navigating according to the size of the obstacle includes controlling the self-propelled robot to reduce a side brush rotation speed within a preset distance range from the obstacle when the size range of the obstacle is less than a preset threshold.

[0108] For example, for a small-sized obstacle, a relatively long obstacle avoidance distance can be selected so as not to blow away the obstacle at a close distance. Specifically, the size of the obstacle is determined according to the size of the area, and one preset threshold is set with reference to experimental data. This threshold may be the area or the amount of pixels, but is not limited to a specific threshold. If this threshold is exceeded, it is determined to be a large-sized obstacle, and otherwise it is a small-sized obstacle. For a small-sized obstacle, the self-propelled robot can be controlled to reduce the side brush rotation speed within a preset distance range from the obstacle so as not to perform obstacle avoidance and not to blow away the small-sized obstacle.

[0109] In one embodiment, the step of navigating according to the size of the obstacle further includes determining an obstacle avoidance distance according to the size of the obstacle or the range in which the obstacle is located, and controlling the self-propelled robot to start execution of an obstacle avoidance policy at the obstacle avoidance distance.

[0110] The self-propelled robot is controlled so that it performs obstacle avoidance operations within a preset distance range from an obstacle that is large enough to exceed a preset threshold. Typically, the larger the determined size of the obstacle, the smaller the distance at which obstacle avoidance is performed, and the larger the obstacle, the less susceptible it is to side brushes, allowing the robot to move along the obstacle.

[0111] In one embodiment, the step of navigating according to the size of the obstacle specifically includes executing an obstacle avoidance policy when it is determined that the height of the obstacle is greater than a preset height. In this way, only obstacles whose height exceeds a certain value need to be avoided, and if the height is small, the obstacle can be directly overcome to further reduce the clearance loss area.

[0112] As shown in FIG. 12, the embodiment of the present disclosure provides an obstacle detection device, which is used to implement the steps of the method in the above embodiment. Similar method steps have the same technical effect. Hereinafter, detailed description will be omitted. Specifically, the obstacle detection device includes: A conversion unit 1202 used to convert the obstacle information collected by the image collection device into depth information; a conversion unit 1204 used to convert the depth information into a point cloud map and determine coordinate data of each reference point of the obstacle; A determining unit 1206 for selecting an effective analysis range in the height direction from the point cloud map; and an analysis unit 1208, which is used to analyze whether an obstacle exists within the effective analysis range based on the coordinate data of each of the reference points within the effective analysis range.

[0113] Optionally, the analysis unit 1208 is further used to convert the 3D coordinate data of each of the reference points within the effective analysis range into 2D coordinate data, and analyze whether an obstacle exists within the effective analysis range based on the 2D coordinate data of each of the reference points within the effective analysis range.

[0114] Optionally, the analysis unit 1208 is further used to obtain a probability map formed during the traveling process of the self-propelled robot, filter noise point coordinate data in the 2D coordinate data of each of the reference points based on the probability map, and analyze whether an obstacle exists within the effective analysis range based on the filtered 2D coordinate data of each of the reference points.

[0115] Optionally, the analysis unit 1208 is further used to obtain position coordinate data of an obstacle in the probability map when the probability value of the obstacle in the probability map is greater than a preset probability threshold, compare the position coordinate data of the obstacle with the 2D coordinate data of each reference point, and filter the 2D coordinate data when the distance between the position coordinate data of the obstacle and the 2D coordinate data of each reference point is greater than a predetermined value.

[0116] Optionally, the analysis unit 1208 further clusters the reference points within the valid analysis range, and all reference points whose distance from the coordinate data of adjacent points is less than a second predetermined value are grouped into one region, respectively determine the size of each region, and if the size is equal to or greater than a third predetermined value, determine that an obstacle exists within the valid analysis range, where the size includes at least one of the area / volume of each region, the diameter of each region, and the minimum distance across the geometric center of each region.

[0117] The system further includes a control unit (not shown) that is used to determine whether a size range of an obstacle is less than a predetermined threshold when an obstacle is present within the effective analysis range, and to control the self-propelled robot to reduce a side brush rotation speed within a predetermined distance range from the obstacle when the size range of the obstacle is less than the predetermined threshold, and to control the self-propelled robot to perform an obstacle avoidance operation within a predetermined distance range from the obstacle when the size range of the obstacle is equal to or greater than the predetermined threshold.

[0118] The embodiment of the present disclosure provides an obstacle detection device, which uses a depth image of the self-propelled robot within a certain height range acquired by the binocular camera of the self-propelled robot to perform processes such as coordinate transformation and data clustering on the depth image to more accurately determine whether an obstacle exists in the current depth image. This method detects all objects in the driving path of the self-propelled robot, breaks through the obstacle type limitations of the conventional method, and at the same time improves the detection height defect of the laser device, further improves the obstacle detection ability of the self-propelled robot, and further improves the obstacle avoidance performance of the self-propelled robot.

[0119] As shown in FIG. 13, another embodiment of the present disclosure provides an obstacle detection device, which implements the steps of the method in the above embodiment. Similar method steps have the same technical effect. Hereinafter, detailed description will be omitted. Specifically, the obstacle detection device includes: an image conversion unit 1302, which is used to convert the obstacle information collected by the image collection device into depth information; an image conversion unit 1304, which is used to convert the depth image into a point cloud map and determine the coordinate data of each reference point of the obstacle; a range selection unit 1306 used to select a valid analysis range from the point cloud map; and a navigation execution unit 1308 used to determine whether an obstacle exists within the effective analysis range and to perform navigation according to the size of the obstacle.

[0120] In one embodiment, the determining unit 1308 further clusters the reference points in the effective analysis range, and all the reference points whose distance from the coordinate data of the neighboring points is less than a second predetermined value are grouped into one region, and the size of each region is respectively determined, and when the size of the region is equal to or greater than a third predetermined value, the size is used to determine that an obstacle exists within the effective analysis range. Optionally, the size includes at least one of the following: the area / volume of each region, the diameter of each region, the minimum distance across the geometric center of each region, and the height of the region.

[0121] In one embodiment, the step of navigating according to the size of the obstacle includes controlling the self-propelled robot to reduce a side brush rotation speed within a preset distance range from the obstacle when the size range of the obstacle is less than a preset threshold.

[0122] In one embodiment, the step of navigating according to the size of the obstacle further includes determining an obstacle avoidance distance according to the size or range of the obstacle, and controlling the self-propelled robot to start execution of an obstacle avoidance policy at the obstacle avoidance distance.

[0123] An embodiment of the present disclosure provides a non-transitory computer-readable storage medium, storing computer program instructions, which, when invoked and executed by a processor, perform the steps of the method according to any one of the above.

[0124] An embodiment of the present disclosure provides a robot including a processor and a memory, the memory storing computer program commands executable by the processor, and the processor, when executing the computer program commands, performs the steps of the method of any one of the embodiments above.

[0125] As shown in Fig. 14, the robot may further include a processing unit (e.g., a central processor, a graphics processor, etc.) 1401 that may perform various appropriate operations and processes according to programs stored in a Read Only Memory (ROM) 1402 or loaded from a storage device 1408 into a Random Access Memory (RAM) 1403. The RAM 1403 stores various programs and data required for the operation of the electronic robot 1400. The processing unit 1401, the ROM 1402, and the RAM 1403 are connected to each other via a bus 1404. An input / output (I / O) interface 1405 is also connected to the bus 1404.

[0126] Typically, input devices 1406, such as a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc., output devices 1407, such as a liquid crystal display (LCD), speaker, vibrator, etc., storage devices 1408, such as a hard disk, etc., and communication devices 1409 may be connected to the I / O interface 1405. The communication devices 1409 may enable the electronic robot to communicate wirelessly or wired with other robots to exchange data. Although FIG. 14 shows an electronic robot with various devices, it should be understood that not all devices need to be implemented or provided. Alternatively, more or fewer devices may be implemented or provided.

[0127] In particular, according to an embodiment of the present disclosure, the processes described with reference to the above flowcharts may be implemented as a robot software program. For example, an embodiment of the present disclosure provides a robot software program product, including a computer program stored on a readable medium. The computer program includes program code of the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network using the communication device 1409, or may be installed from the storage device 1408, or may be installed from the ROM 1402. When the computer program is executed by the processing device 1401, the above functions defined in the method of the embodiment of the present disclosure are performed.

[0128] It should be noted that the computer readable medium of the present disclosure may be a computer readable signal medium, or a computer readable storage medium, or any combination of both. The computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination. The computer readable storage medium may more specifically be, but not limited to, an electrical connection having one or more conductive lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium may be any tangible medium that contains or stores a program, which may be used by or in combination with a command execution system, apparatus, or device. In the present disclosure, the computer readable signal medium may include a data signal that contains the computer readable program code and is propagated in baseband or as part of a carrier. Such propagated data signals include, but are not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium that transmits, propagates, or transfers a program used by or in combination with a command execution system, apparatus, or device. The program code contained in the computer-readable medium may be transferred by any suitable medium, including, but not limited to, electrical wires, optical cables, RF (radio frequency), or any suitable combination of the above.

[0129] The computer readable medium may be included in the robot or may exist separately and not be assembled to the robot.

[0130] Computer program code for carrying out the operations of the present disclosure may be written in one or more program design languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may run entirely on the user's computer, partially on the user's computer, as an independent software package, partly on the user's computer and partly on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet by using an Internet Service Provider).

[0131] The flowcharts and block diagrams in the accompanying drawings illustrate possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams represents a module, program segment, or part of code that includes executable commands for implementing one or more specified logical functions. It should also be noted that in some alternative aspects, the functions noted in the blocks may occur in a different order than the order depicted in the drawings. For example, two blocks depicted in succession may in fact be executed substantially in parallel, or may be executed in reverse order depending on the related functionality. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be realized by a dedicated hardware-based system that executes a given function or operation, or by a combination of dedicated hardware and computer commands.

[0132] The above described embodiments of the device are merely illustrative, and the units described as separate components above may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed among multiple network units. According to actual needs, some or all of the modules may be selected to achieve the purpose of the embodiment. Those skilled in the art can understand and implement the present application without any creative work.

[0133] Finally, the above embodiments are used to describe the technical solutions of the present disclosure, and are not intended to limit the present disclosure. Although the present disclosure has been described in detail with reference to the above embodiments, those skilled in the art can modify the technical solutions of each of the above embodiments or replace some of the technical features with equivalents. It should be understood that these modifications or replacements do not depart from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of each of the embodiments of the present disclosure.

Claims

1. An automatic cleaning device including an image collecting device, a laser ranging device, and a control system, The image collection device is configured to collect obstacle information; The laser distance measuring device is configured to feed back obstacle information for drawing an environment in which the automatic cleaning device is located, i.e., an instant map; the control system is configured to: draw the instant map using obstacle information fed back by the laser ranging device; draw a probability map based on historical experience indicating whether an obstacle exists; convert the obstacle information collected by the image collection device into depth information; transform the depth information into a point cloud map; determine coordinate data of each reference point on the obstacle; and determine whether an obstacle exists in an environment in which the automatic cleaning device is located based on the coordinate data of each reference point and the probability map.

2. further comprising an analysis unit; 2. The automatic cleaning device of claim 1, wherein the analysis unit is configured to convert the 3D coordinate data of each of the determined reference points of the obstacle into 2D coordinate data, and determine whether an obstacle exists in an environment in which the automatic cleaning device is located based on the 2D coordinate data of each of the reference points and the probability map.

3. determining whether an obstacle exists in an environment in which the automatic cleaning device is located based on the 2D coordinate data of each of the reference points and the probability map, The automatic cleaning device of claim 1, further comprising: filtering noise point coordinate data in the 2D coordinate data of each of the reference points based on the probability map; and analyzing whether an obstacle exists in an environment in which the automatic cleaning device is located based on the filtered 2D coordinate data of each of the reference points.

4. filtering noise point coordinate data in the 2D coordinate data of each of the reference points based on the probability map, Obtaining position coordinate data of an obstacle in the probability map, and determining that a probability value of the obstacle in the probability map is greater than a preset probability threshold value; 4. The automatic cleaning device according to claim 3, further comprising: comparing the position coordinate data of the obstacle with the 2D coordinate data of each of the reference points; and filtering the coordinate data if a distance between the position coordinate data of the obstacle and the 2D coordinate data of each of the reference points is greater than a first predetermined value.

5. determining whether an obstacle exists in an environment where the automatic cleaning device is located based on the coordinate data of each of the reference points and the probability map; clustering the reference points, and classifying all reference points whose distance from the coordinate data of adjacent reference points is less than a second predetermined value into one region; and analyzing and determining the size of each area, and if the size of the area is equal to or greater than a third predetermined value, determining that an obstacle is present in an environment in which the automatic cleaning device is located.

6. The size of the region is 6. The automatic cleaning device of claim 5, further comprising at least one of an area / volume of the region, a diameter of the region, and a minimum distance across a geometric center of the region.

7. The control system further comprises:

2. The automatic cleaning device according to claim 1, wherein when it is determined that an obstacle exists in an environment in which the automatic cleaning device is located, it is configured to determine whether a size of the obstacle is less than a preset threshold, and when the size of the obstacle is less than the preset threshold, control the automatic cleaning device to reduce a side brush rotation speed within a preset distance range from the obstacle.

8. The control system further comprises: The automatic cleaning device according to claim 1, configured to select an effective analysis range in the height direction from the point cloud map, and determine whether an obstacle exists within the effective analysis range based on the coordinate data of each of the reference points within the effective analysis range and the probability map.

Images