Cleaning robot and obstacle avoidance method thereof

Abstract

The present invention discloses a cleaning robot and an obstacle circumvention method thereof. The cleaning robot is provided with a camera, and the obstacle circumvention method comprises: obtaining an image in front of the cleaning robot through the camera; generating a virtual travel line and a virtual sight line on the image, wherein the virtual travel line on the image corresponds to a virtual line formed by extending forward from the left and right edges of the cleaning robot in the actual environment, and the virtual sight line in the image corresponds to a virtual line at a preset safety distance in front of the cleaning robot in the actual environment; and controlling the cleaning robot to circumvent the obstacle by taking the obstacle in the image being located on the side of the virtual travel line and the virtual sight line away from the cleaning robot as a constraint condition. The cleaning robot disclosed in the present invention can not only identify obstacles through the camera, but also circumvent obstacles during walking by generating a virtual travel line and a virtual sight line on the image captured by the camera.

Description

Technical Field

[0001] The present disclosure relates to the technical field of cleaning devices, and particularly to a cleaning robot and an obstacle avoidance method thereof. Background Art

[0002] A cleaning robot is an intelligent device capable of autonomously cleaning a home environment. When the cleaning robot is cleaning or constructing a map in a complex and diverse home environment, it usually relies on a laser ranging sensor to determine whether there is an obstacle in front of the cleaning robot to avoid collision between the cleaning robot and the obstacle in front. Then, sensors such as a wall-following sensor and a collision sensor are used to guide the cleaning robot around the obstacle.

[0003] Currently, there is no cleaning robot that can walk around an identified obstacle through a camera. Summary of the Invention

[0004] The present disclosure provides a cleaning robot and an obstacle avoidance method thereof, which can walk around an identified obstacle through a camera.

[0005] In a first aspect, the present disclosure provides an obstacle avoidance method for a cleaning robot, where a camera is provided on the cleaning robot, and the obstacle avoidance method includes:

[0006] Obtaining an image in front of the cleaning robot through the aforementioned camera;

[0007] Generating a virtual travel line and a virtual viewing distance line on the aforementioned image, where the virtual travel line on the aforementioned image corresponds to a virtual line formed by extending the left and right edges of the aforementioned cleaning robot forward in the actual environment, and the virtual viewing distance line in the aforementioned image corresponds to a virtual line at a preset safe distance in front of the aforementioned cleaning robot in the actual environment;

[0008] Controlling the cleaning robot to walk around the obstacle with the condition that the obstacle in the aforementioned image is on the side away from the cleaning robot of the aforementioned virtual travel line and the aforementioned virtual viewing distance line.

[0009] Optionally, the controlling the cleaning robot to walk around the obstacle with the condition that the obstacle in the aforementioned image is on the side away from the cleaning robot of the aforementioned virtual travel line and the aforementioned virtual viewing distance line includes:

[0010] When the obstacle in the aforementioned image contacts the aforementioned virtual viewing distance line, controlling the cleaning robot to walk around the obstacle with the condition that the aforementioned obstacle in the aforementioned image is tangent to the aforementioned virtual travel line.

[0011] Optionally, with the constraint that the obstacle in the aforementioned image is located on the side away from the cleaning robot of the aforementioned virtual travel line and the aforementioned virtual line of sight, controlling the aforementioned cleaning robot to walk around the aforementioned obstacle includes:

[0012] When the obstacle in the aforementioned image contacts the aforementioned virtual line of sight, determine the width of the aforementioned obstacle and the estimated value of the length;

[0013] Determine the obstacle avoidance end point of the aforementioned cleaning robot according to the estimated value of the length of the aforementioned obstacle, and the aforementioned obstacle avoidance end point and the current position of the aforementioned cleaning robot are respectively located on opposite sides of the aforementioned obstacle;

[0014] Plan an obstacle avoidance path according to the aforementioned current position and the aforementioned obstacle avoidance end point;

[0015] Make the aforementioned cleaning robot walk along the aforementioned obstacle avoidance path, and when the aforementioned virtual line of sight and / or the aforementioned virtual travel line contacts the aforementioned obstacle, update the aforementioned obstacle avoidance end point according to the image obtained during the travel, and adjust the subsequent obstacle avoidance path according to the updated obstacle avoidance end point.

[0016] Optionally, the aforementioned planning of the obstacle avoidance path according to the aforementioned current position and the aforementioned obstacle avoidance end point includes:

[0017] Determine the distance between the aforementioned current position and the aforementioned obstacle avoidance end point, and use the arc with a diameter equal to the aforementioned distance and connecting the aforementioned current position and the aforementioned obstacle avoidance end point as the aforementioned obstacle avoidance path.

[0018] Optionally, the aforementioned planning of the obstacle avoidance path according to the aforementioned current position and the aforementioned obstacle avoidance end point includes:

[0019] Use the line connecting the aforementioned current position and the aforementioned obstacle avoidance end point as the first axis;

[0020] Determine a second axis perpendicular to the aforementioned first axis according to the width of the aforementioned obstacle and the size of the aforementioned cleaning robot;

[0021] Use the elliptical arc with the major axis and minor axis being the aforementioned first axis and the aforementioned second axis respectively, and connecting the aforementioned current position and the aforementioned obstacle avoidance end point as the aforementioned obstacle avoidance path.

[0022] Optionally, the aforementioned planning of the obstacle avoidance path according to the aforementioned current position and the aforementioned obstacle avoidance end point includes:

[0023] Determine multiple intermediate points between the aforementioned current position and the aforementioned obstacle avoidance end point, and the aforementioned intermediate points are located outside the aforementioned obstacle;

[0024] Use the line connecting the aforementioned current position, the aforementioned intermediate points, and the aforementioned obstacle avoidance end point in sequence as the aforementioned obstacle avoidance path.

[0025] Optionally, generating a virtual travel line and a virtual line of sight on the foregoing image includes:

[0026] Adding the foregoing virtual travel line and the foregoing virtual line of sight to the foregoing image according to the positions of the pre-determined virtual travel line and the virtual line of sight mapped to the calibrated image captured by the foregoing camera.

[0027] Optionally, the foregoing obstacle avoidance method further includes:

[0028] Obtaining a calibrated image of a calibrated object through the foregoing camera; a first line representing the foregoing virtual travel line and a second line representing the foregoing virtual line of sight are provided on the foregoing calibrated object;

[0029] Determining the positions of the foregoing first line and the foregoing second line in the foregoing calibrated image.

[0030] Optionally, the foregoing obstacle avoidance method further includes:

[0031] In response to the foregoing cleaning robot turning, determining the turning trajectory of the foregoing cleaning robot;

[0032] Adjusting the foregoing virtual travel line to extend along the foregoing turning trajectory.

[0033] In a second aspect, the present disclosure provides a cleaning robot, which includes a processor, a memory, and execution instructions stored on the foregoing memory. The foregoing execution instructions are configured to enable the foregoing cleaning robot to execute the obstacle avoidance method described in any one of the technical solutions in the first aspect when executed by the foregoing processor.

[0034] Based on the foregoing description, those skilled in the art can understand that the present disclosure obtains an image in front of the cleaning robot through a camera, and generates a virtual travel line and a virtual line of sight in the image, so as to determine the path that the cleaning robot will walk through the virtual travel line, and determine the safe distance of the cleaning robot in the traveling direction through the virtual line of sight. Furthermore, taking the condition that the obstacle in the image is on the side away from the cleaning robot of the virtual travel line and the virtual line of sight as a constraint, the cleaning robot is controlled to walk around the obstacle. Therefore, the present disclosure can not only identify obstacles through a camera, but also generate a virtual travel line and a virtual line of sight on the image captured by the camera, and make the obstacle always on the side away from the cleaning robot of the virtual travel line and the virtual line of sight, avoiding the cleaning robot from colliding with the obstacle during walking.

[0035] Further optionally, when an obstacle in the image contacts the virtual line of sight, by taking the tangency of the obstacle in the image with the virtual travel line as a constraint condition, the cleaning robot is controlled to walk around the obstacle, so that during the process of the cleaning robot bypassing the obstacle, a relatively close distance is always maintained between the cleaning robot and the obstacle, in order to finely clean the environment around the obstacle.

[0036] Further optionally, when an obstacle in the image contacts the virtual line of sight, by determining the estimated values of the width and length of the obstacle in the image, the end point of the obstacle bypassing of the cleaning robot can be preliminarily determined, so that the cleaning robot can plan an obstacle bypass path according to the current position and the end point of the obstacle bypassing, thereby enabling the cleaning robot to quickly bypass the obstacle. At the same time, when the virtual line of sight and / or the virtual travel line contacts the obstacle, by updating the end point of the obstacle bypassing according to the image obtained during the travel process and adjusting the subsequent obstacle bypass path according to the updated end point of the obstacle bypassing, the situation that the cleaning robot collides with the obstacle during the obstacle bypassing process is effectively avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the present disclosure, the following will detail some embodiments of the present disclosure in conjunction with the drawings, where:

[0038] Figure 1 is a schematic flowchart of the main steps of the obstacle bypassing method in the first embodiment of the present disclosure;

[0039] Figure 2 is a schematic diagram of the effect of an image with a virtual travel line and a virtual line of sight in the first embodiment of the present disclosure;

[0040] Figure 3 is a schematic diagram of the fine obstacle bypass path of the cleaning robot in the first embodiment of the present disclosure;

[0041] Figure 4 is a schematic diagram of the effect of an image of the cleaning robot during the fine obstacle bypassing process in the first embodiment of the present disclosure;

[0042] Figure 5 is a flowchart of the steps for the cleaning robot to quickly bypass an obstacle in the first embodiment of the present disclosure;

[0043] Figure 6 is a schematic diagram of the quick obstacle bypass path of the cleaning robot in the first embodiment of the present disclosure (completed in one planning);

[0044] Figure 7 is a schematic diagram of the path of the cleaning robot during the quick obstacle bypassing process in the first embodiment of the present disclosure (the path is an arc);

[0045] Figure 8It is a schematic diagram of the path of the rapid obstacle avoidance process of the cleaning robot in the first embodiment of the present disclosure (the path is an elliptical arc);

[0046] Figure 9 It is a schematic diagram of the path of the rapid obstacle avoidance process of the cleaning robot in the first embodiment of the present disclosure (the path is a polygon);

[0047] Figure 10 It is a schematic diagram of the path of the rapid obstacle avoidance process of the cleaning robot in the first embodiment of the present disclosure (the path is a multi-I shape);

[0048] Figure 11 It is a schematic diagram of the rapid obstacle avoidance path of the cleaning robot in the first embodiment of the present disclosure (completed through multiple planning);

[0049] Figure 12 It is a partial step flow schematic diagram of the obstacle avoidance method in the second embodiment of the present disclosure;

[0050] Figure 13 It is a schematic diagram of calibrating the camera in the second embodiment of the present disclosure;

[0051] Figure 14 It is a partial step flow schematic diagram of the obstacle avoidance method in the third embodiment of the present disclosure;

[0052] Figure 15 It is a schematic diagram of the virtual travel line and the virtual line of sight when the cleaning robot turns in the third embodiment of the present disclosure;

[0053] Figure 16 It is a geometric schematic diagram of the steering angle of the virtual travel line in the third embodiment of the present disclosure;

[0054] Figure 17 It is a partial step flow schematic diagram of the obstacle avoidance method in the fourth embodiment of the present disclosure;

[0055] Figure 18 It is a schematic diagram of the structure of the cleaning robot in the fifth embodiment of the present disclosure.

[0056] List of reference numerals:

[0057] 1. Virtual travel line; 11. Left virtual travel line; 12. Right virtual travel line; 13. Middle virtual travel line;

[0058] 2. Virtual line of sight; 21. First virtual line of sight; 22. Second virtual line of sight; 23. Third virtual line of sight;

[0059] 3. Obstacle;

[0060] 4. Cleaning robot; 41. Camera

[0061] 5. Obstacle avoidance path;

[0062] 6. Calibration board. Specific implementation manners

[0063] To make the objectives, technical solutions and advantages of the present disclosure clearer, the technical solutions of the present disclosure will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Those skilled in the art should understand that the embodiments described in this section of specific implementation manners are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure. Based on the embodiments described in this section of specific implementation manners, all other embodiments obtained by those of ordinary skill in the art without creative efforts will not deviate from the technical principles of the present disclosure, and thus should fall within the protection scope of the present disclosure.

[0064] It should be noted that in the description of the present disclosure, each functional module can be either a physical module composed of multiple structures, components or electronic components, or a virtual module composed of multiple programs; each functional module can either exist independently of each other, or be a module divided from an overall module according to functions. Those skilled in the art should understand that on the premise of being able to implement the technical solutions described in the present disclosure, no matter how the composition manner, implementation manner and positional relationship of each functional module change, they will not deviate from the technical principles of the present disclosure, and thus should fall within the protection scope of the present disclosure.

[0065] The cleaning robot of the present disclosure includes robots with cleaning functions such as floor sweeping robots, mopping robots, and sweeping, sucking and mopping integrated robots. A camera is provided on the cleaning robot of the present disclosure, and the camera can collect images in front of the cleaning robot. The cleaning robot of the present disclosure further includes two driving wheels, and the two driving wheels are respectively located on the left and right sides of the cleaning robot, and the two driving wheels are used to drive the cleaning robot to walk and turn.

[0066] The following will refer to the drawings to elaborate on some embodiments of the present disclosure.

[0067] In the first embodiment of the present disclosure:

[0068] As Figure 1 shown, the obstacle avoidance method of the cleaning robot in this embodiment includes:

[0069] Step S110, obtaining an image in front of the cleaning robot through the camera.

[0070] In this embodiment, during the walking process of the cleaning robot, the cleaning robot can obtain an image in front of it through the camera at intervals of a certain period of time (such as 0.5 seconds, 1 second, 1.8 seconds, 5 seconds, etc.), or can also capture a video in front of the cleaning robot in real time through the camera, and then extract the frames of the video.

[0071] Step S120, generate a virtual travel line 1 and a virtual viewing distance line 2 on the image.

[0072] As Figure 2 shown, add the pre-stored virtual travel line data and virtual viewing distance line data to the image obtained in step S110, or generate a new image according to the image obtained in step S110 and the pre-stored virtual travel line data and virtual viewing distance line data, so as to combine the image obtained in step S110 with the pre-stored virtual travel line data and virtual viewing distance line data.

[0073] Continue to refer to Figure 2 , the virtual travel line 1 on the image corresponds to the virtual line formed by extending the left and right edges of the cleaning robot forward in the actual environment. Preferably, the left virtual travel line 11 and the right virtual travel line 12 on the image are the same width as the cleaning robot, so that when an obstacle that contacts or is located between the left virtual travel line 11 and the right virtual travel line 12 appears in the image, it can be determined that there is an obstacle in front of the cleaning robot that hinders its progress. In addition, on the premise that an obstacle that hinders the progress of the cleaning robot can be recognized from the image, those skilled in the art can also, according to needs, make the distance between the left virtual travel line 11 and the right virtual travel line 12 on the image greater than or less than the width of the cleaning robot.

[0074] Continue to refer to Figure 2 , the virtual viewing distance line 2 in the image corresponds to the virtual line at a preset safe distance in front of the cleaning robot in the actual environment. Wherein, the preset safe distance refers to the maximum distance traveled from the moment the cleaning robot obtains the image in front through the camera, to the moment the cleaning robot recognizes an obstacle in front through the obtained image, and then to the moment of stopping or turning. On the premise that the cleaning robot will not collide with the obstacle in front, the preset safe distance can be any feasible value, such as 5cm, 15cm, 20cm, 40cm, 50cm, 100cm, etc.

[0075] Furthermore, the number of virtual viewing distance lines 2 in the image can be one or multiple, such as Figure 2 the first virtual viewing distance line 21, the second virtual viewing distance line 22 and the third virtual viewing distance line 23 shown in. Among them, the value range of the distance between the first virtual viewing distance line 21 and the cleaning robot is [5cm, 15cm]; the value range of the distance between the second virtual viewing distance line 22 and the cleaning robot is [20cm, 40cm]; the value range of the distance between the third virtual viewing distance line 23 and the cleaning robot is [50cm, 100cm].

[0076] Step S130: With the constraint that the obstacle in the image is on the side away from the cleaning robot of the virtual travel line and the virtual line of sight, control the cleaning robot to walk around the obstacle.

[0077] As Example 1, when the obstacle in the image contacts the virtual line of sight 2, with the constraint that the obstacle in the image is tangent to the virtual travel line 1, control the cleaning robot to walk around the obstacle. The following will refer to Figure 3 and Figure 4 to elaborate on this example in detail.

[0078] As Figure 3 shown, when the cleaning robot 4 travels to position 1, the obstacle 3 contacts the first virtual line of sight 21, causing the cleaning robot 4 to turn in place until the obstacle 3 leaves the left virtual travel line 11 and is tangent to it (see Figure 4 ), and then make the cleaning robot 4 travel in a straight line from position 1 to position 2.

[0079] Furthermore, when the cleaning robot 4 moves to position 2, the left virtual travel line 11 is not tangent to the obstacle 3, causing the cleaning robot 4 to turn until the left virtual travel line 11 is tangent to the obstacle 3, and then make the cleaning robot 4 travel in a straight line from position 2 to position 3.

[0080] Repeat the above process to make the cleaning robot 4 travel to positions 3, 4, 5 until it travels to position 6, causing the cleaning robot to return to the original straight travel route without obstacles, and then make the cleaning robot turn in place to restore the forward travel posture at position 1 and continue to perform the original walking operation. Or, make the cleaning robot 4 repeat the above process to walk around the obstacle 3 for one week to achieve edge cleaning.

[0081] Those skilled in the art can understand that the path planning strategy in Example 1 of this embodiment can enable the cleaning robot to clean the edge area of the obstacle comprehensively and without dead corners, especially suitable for cleaning operations in narrow spaces such as corners and under beds and cabinets. However, this path planning strategy requires a large number of repeated obstacle detections and condition judgments, resulting in relatively large consumption of computing resources and occupation of computing power. For this reason, this embodiment also provides a path planning strategy for rapid cleaning. For details, please refer to Example 2 described below.

[0082] As Figure 5 shown, the path planning of Example 2 includes:

[0083] Step S131: When the obstacle in the image contacts the virtual line of sight, determine the width of the obstacle and the estimated length according to the image.

[0084] Step S132: Determine the obstacle avoidance end point of the cleaning robot according to the estimated length of the obstacle.

[0085] Step S133: Plan an obstacle avoidance path based on the current position and the obstacle avoidance end point.

[0086] Step S134: Make the cleaning robot walk along the obstacle avoidance path.

[0087] Step S135: Does the virtual line of sight and / or the virtual travel line touch an obstacle? If it touches an obstacle, execute Step S136; if it does not touch an obstacle, execute Step S134.

[0088] Step S136: Update the obstacle avoidance end point based on the image obtained during the travel process, adjust the subsequent obstacle avoidance path according to the updated obstacle avoidance end point, and then continue to execute Step S134.

[0089] The following combines Figures 6 to 11 to illustrate the path planning of Example 2 by way of example.

[0090] In Figures 6 to 11 , the circles 1, 2, 3, 4, 5, and 6 represent different nodes during the obstacle avoidance process of the cleaning robot. The dashed circles represent the trajectories of the centers of the first virtual line of sight 21 and the second virtual line of sight 22 in the horizontal plane formed when the cleaning robot rotates 360° in place. It is set that the initial coordinates of the current position of the cleaning robot are (0, 0), and the coordinates of the obstacle avoidance end point after bypassing the obstacle are (D0, 0). Among them, D0 is the straight-line distance between the positions of the cleaning robot on the front and back sides of the obstacle. Among them, D0 is greater than D. For example, it is set that D0 = D + 2 * d2, D is the estimated value of the length of obstacle 3, and d2 is a preset value to leave a certain safety gap between the cleaning robot and the obstacle.

[0091] In this example, the virtual line of sight in the image that touches the obstacle is the second virtual line of sight 22. Or those skilled in the art can also select any other feasible virtual line of sight as the virtual line of sight that touches the obstacle according to needs. For example, the first virtual line of sight 21, the third virtual line of sight 23, or any virtual line of sight inside the first, second, or third virtual line of sight.

[0092] Those skilled in the art can understand that since the cross-sections of most obstacles are not square or circular, the cleaning robot cannot accurately obtain the true data of the thickness (length) of the obstacle in the image. Only the estimated value of the length of obstacle 3 can be determined through the actual width of obstacle 3. This estimated value can be a preset multiple of the actual width of obstacle 3, such as 1 times or 2 times.

[0093] As Figure 6As shown, when the estimated length value is greater than or equal to the actual length value of the obstacle 3, the cleaning robot only needs to plan an obstacle avoidance path 5 once. The obstacle avoidance path 5 planned once can be Figure 7 the arc shown in Figure 8 the elliptical arc shown in Figure 9 the polygon shown in Figure 10 the bow shape shown in

[0094] In Figure 7 , first calculate the true width distance W of the obstacle 3 according to the pixel information of the obstacle 3 in the image, then estimate the length value D of the obstacle 3 with W, and set D0 = D + 2 * d2. Where d2 is the distance from the second virtual line of sight 22 to the center of the cleaning robot. Finally, generate an arc with a diameter of D0 and connect the starting point (0, 0) and the ending point (D0, 0) together as the obstacle avoidance path 5.

[0095] In Figure 8 , different from Figure 7 , an elliptical arc connecting the starting point (0, 0) and the ending point (D0, 0) is used as the obstacle avoidance path 5. Where the major axis (the first axis) of the elliptical arc is the line connecting the starting point (0, 0) and the ending point (D0, 0), and the minor axis (the second axis) of the elliptical arc is (W + 2 * R0). Where R0 represents the farthest distance that the center point of the cleaning robot can reach for cleaning.

[0096] In Figure 9 , the obstacle avoidance path 5 has 6 successively connected nodes: (0, 0), (0, D0 / 6), (D0 / 3, D0 / 2), (2 * D0 / 3, D0 / 2), (D0, D0 / 6), (D0, 0).

[0097] In Figure 10 , the obstacle avoidance path 5 has 4 successively connected nodes: (0, 0), (0, D0 / 2), (D0, D0 / 2), (D0, 0).

[0098] As Figure 11 shown, when the estimated length value is less than the actual length value of the obstacle 3, the cleaning robot needs to plan the obstacle avoidance path 5 multiple times. Specifically, when the obstacle in the image contacts the virtual line of sight, first determine the width and length estimated value of the obstacle according to the image, and then determine the preliminary obstacle avoidance path 5 in any way as Figures 7 to 10 shown, and make the cleaning robot start to move along the obstacle avoidance path 5.

[0099] When the virtual line of sight and / or the virtual movement line contacts the obstacle 3 again, first make the obstacle avoidance end point increase in the direction away from the obstacle avoidance start point, for example, increase by a distance d L , and become (D0 + d L) and adjust the subsequent obstacle avoidance path 5 according to the updated obstacle avoidance end point, and make the cleaning robot continue to travel along the adjusted obstacle avoidance path 5. If the virtual line of sight and / or the virtual travel line contacts the obstacle 3 again, move the obstacle avoidance end point away from the obstacle avoidance starting point by a distance d L . Among them, d L can be any feasible value, such as 2 cm, 5 cm, 9 cm, 11.5 cm, etc.

[0100] Or, when the virtual line of sight and / or the virtual travel line contacts the obstacle 3 again, use the images captured during the travel of the cleaning robot to recalculate the length estimation value of the obstacle 3, then update the obstacle avoidance end point, and adjust the subsequent obstacle avoidance path 5 according to the updated obstacle avoidance end point, and make the cleaning robot continue to travel along the adjusted obstacle avoidance path 5.

[0101] Based on the foregoing description, those skilled in the art can understand that the cleaning robot of the present disclosure can not only identify the obstacles ahead through the camera, but also guide the cleaning robot to perform fine obstacle avoidance or fast obstacle avoidance operations on the obstacles through the relationship between the virtual travel line 1 and the virtual line of sight 2 in the image and the obstacles.

[0102] It should be noted that the first embodiment of the present disclosure is only a basic embodiment of the obstacle avoidance method of the present disclosure, and other optional embodiments can be obtained on this basis, such as the following second embodiment.

[0103] In the second embodiment of the present disclosure:

[0104] As Figure 12 shown, different from the first embodiment, before step S110, the obstacle avoidance method of this embodiment further includes:

[0105] Step S210, obtaining a calibration image of the calibration object through the camera.

[0106] As Figure 13 shown, in this embodiment, first select a calibration board 6 in a checkerboard format as the calibration object. The calibration board 6 has the same width as the cleaning robot 4, both are W0, and the length L0 of the calibration board 6 exceeds the imaging range of the image of the camera 41.

[0107] Then, fix the cleaning robot 4 on the horizontal ground, and the specific position is as Figure 13 shown: make the linear travel direction of the cleaning robot 4 perpendicular to the width direction of the calibration board 6 on the horizontal ground, make the center line of the linear travel of the cleaning robot 4 coincide with the center line of the width of the calibration board 6, and fix the front end of the body of the cleaning robot 4 at the starting position of the calibration board 6.

[0108] Next, adjust the parameter variables such as the position, focal length, field of view angle, and imaging of the camera 41 so that the image captured by the camera 41 meets the following conditions: the wide side w is consistent with the Figure 13 in the Y direction, the high side h is consistent with the Figure 13 in the X direction, and Figure 13 the X axis in Figure 13 vertically bisects the wide side w of the image, so that the calibration plate 6 in the image is symmetric about the X axis, and the origin of the XY coordinate axes in the image appears at the midpoint of the wide side w at the bottom edge of the image.

[0109] Next, make the camera 41 acquire the calibration image of the calibration plate 6.

[0110] Finally, map the position and measurement information of the calibration plate 6 in the calibration image to the two-dimensional image space, and create a mask layer for the calibration image to store the generated virtual travel line 1 and virtual viewing distance line 2.

[0111] Step S220, determine the positions of the first line and the second line in the calibration image.

[0112] Among them, the first line and the second line are respectively on the calibration plate 6, and the first line represents the virtual travel line 1, and the second line represents the virtual viewing distance 2. Specifically, the first line is the Figure 13 upper edge line (left edge line in the calibration image) and lower edge line (right edge line in the calibration image) of the calibration plate 6 in Figure 13 . The second line is the Figure 13 line parallel to the Y axis of the calibration plate 6 in Figure 13 . Usually, there are multiple second lines in the calibration plate 6.

[0113] Step S230, determine the virtual travel line according to the first line, and determine the virtual viewing distance line according to the second line.

[0114] As Figure 2 and Figure 13 shown, extract the left edge line of the calibration plate 6 from the calibration image, and calculate the starting position PL0(xl0, yl0), ending position PL1(xl1, yl1) and slope of the left edge line. Use the data of PL0, PL1, and KL to generate the first virtual travel line 11 (left limit mark line Leftline) in the mask layer. The first virtual travel line 11 coincides with the left edge line of the calibration plate 6 in the calibration image. Similarly, extract the right edge line of the calibration plate 6 from the calibration image, and calculate the starting position PR0(xr0, yr0), ending position PR1(xr1, yr1) and slope Generate a second virtual travel line 12 (Rightline, the right limit marking line) on the mask layer using the PR0, PR1, and KR data. The second virtual travel line 12 coincides with the right edge line of the calibration board 6 in the calibrated image. Further, generate a third virtual travel line 13 (Centerline, the center marking line) on the mask layer. The third virtual travel line 13 is the vertical bisector in the left - right direction of the calibrated image.

[0115] From Figure 2 and Figure 13 it is not difficult to see that: The area between the first virtual travel line 11 and the second virtual travel line 12 is the safe lane for the straight - line travel of the cleaning robot 4.

[0116] Further, according to the traveling speed of the cleaning robot and actual needs, generate several virtual sight - distance lines 2 on the mask layer, such as Figure 2 the first virtual sight - distance line 21, the second virtual sight - distance line 22, and the third virtual sight - distance line 23 shown in

[0117] Among them, each virtual sight - distance line is perpendicular to the third virtual travel line 13 and is located between the first virtual travel line 11 and the second virtual travel line 12. Preferably, the actual distance corresponding to the distance d1 from the bottom edge of the mask layer to the first virtual sight - distance line 21 after affine transformation is 5 cm to 15 cm, the actual distance corresponding to the distance d2 from the bottom edge of the mask layer to the second virtual sight - distance line 22 after affine transformation is 20 cm to 40 cm, and the actual distance corresponding to the distance d3 from the bottom edge of the mask layer to the third virtual sight - distance line 23 after affine transformation is 50 cm to 100 cm.

[0117] Among them, the virtual sight - distance line is the basis for judging the distance between the cleaning robot and the front.

[0118] Furthermore, to prevent the cleaning robot from colliding with obstacles, the scaling factor α of the virtual sight - distance line can also be set according to the walking speed of the cleaning robot. α can be dynamically adjusted according to the real - time straight - line traveling speed of the cleaning robot. When the cleaning robot is at normal speed, α = 1. The larger the speed, the larger α, and the smaller the speed, the smaller α. Taking the distance d1' from the bottom edge of the mask layer to the first virtual travel line 11 as an example, d1' = α * d1.

[0119] After that, when performing step S120, the mask layer with the virtual travel line 1 and the virtual sight - distance line 2 can be superimposed on the image obtained in step S110, so as to generate the virtual travel line 1 and the virtual sight - distance line 2 on this image.

[0120] Based on the foregoing description, those skilled in the art can understand that in this embodiment, through the calibration board 6, the cleaning robot can generate the virtual travel line 1 and the virtual sight - distance line 2 for accurately detecting obstacles, improving the accuracy of the cleaning robot in detecting obstacles and the reliability of obstacle avoidance.

[0121] In the third embodiment of the present disclosure:

[0122] As Figure 14 shown, different from the first embodiment and / or the second embodiment, during the turning process of the cleaning robot in this embodiment, the obstacle avoidance method further includes:

[0123] Step S310, in response to the turning of the cleaning robot, determine the turning trajectory of the cleaning robot.

[0124] As Figure 15 and Figure 16 shown, point O is the center point of the turning trajectory of the cleaning robot; r is the turning radius of the center point of the cleaning robot; d0 is the distance between the left and right driving wheels of the cleaning robot; VL is the traveling speed of the left driving wheel; VR is the traveling speed of the right driving wheel; the linear speed at the center point between the two driving wheels is VM=(VL + VR) / 2; d is the distance that the right driving wheel travels more than the left driving wheel within Δt time; θ3 is the change amount of the heading angle of the cleaning robot. According to the geometric relationship, Figure 16 θ1 = θ2 = θ3 in

[0125] The heading angle of the cleaning robot:

[0126]

[0127] Since θ1 = θ2 = θ3, the angular velocity w of the cleaning robot moving around the center of the circle:

[0128]

[0129] The arc radius of the turning motion of the cleaning robot:

[0130]

[0131] Therefore, according to VR, VL, d0, and r, the current turning trajectory of the cleaning robot can be determined.

[0132] Step S320, adjust the virtual travel line to extend along the turning trajectory.

[0133] As Figure 15 shown, make the virtual travel line 1 extend along the extension direction of the turning trajectory, and make each virtual line of sight perpendicular to the tangent of the turning trajectory of the driving wheel.

[0134] Based on the foregoing description, those skilled in the art can understand that the virtual travel line 1 and the virtual line of sight 2 in this embodiment can be adjusted according to the turning angle of the cleaning robot, so that the cleaning robot can also perform obstacle avoidance operations through the virtual travel line 1 and the virtual line of sight 2 during the turning process.

[0135] In the fourth embodiment of the present disclosure:

[0136] As Figure 16 shown, different from the first embodiment, the second embodiment, and / or the third embodiment, the obstacle avoidance method of this embodiment further includes:

[0137] Step S401, obstacle detection.

[0138] Specifically, during the process of the cleaning robot moving forward, the camera continuously obtains the image in front of the cleaning robot, and then continuously identifies whether there are obstacles in the image.

[0139] Step S402, whether there is a moving obstacle in front.

[0140] Specifically, if an obstacle is detected in the image, the frame difference method is used to determine whether the obstacle is a moving obstacle. If not, step S403 is executed; if so, step S404 is executed.

[0141] Step S403, update the path table.

[0142] Specifically, every preset time period (such as 5 seconds, 7 seconds, 15 seconds, etc.) or preset distance (such as 3 cm, 5 cm, 9 cm, etc.), and / or every time the attitude of the cleaning robot changes, the path table of the cleaning robot is updated. The path table is used to guide the cleaning robot to move forward.

[0143] Step S404, determine whether the moving obstacle is within the first virtual line of sight.

[0144] Specifically, it is possible to determine whether the obstacle is located between the bottom edge of the image and the first virtual line of sight from the image obtained at the current moment. If it is located, step S405 is executed; if not, step S406 is executed.

[0145] Step S405, obstacle avoidance, turn in place, stop or retreat.

[0146] Specifically, the cleaning robot can be stopped at the current position to wait for the moving obstacle to leave; the cleaning robot can also be made to retreat to avoid the cleaning robot colliding with the moving obstacle; the cleaning robot can also be made to turn and then continue to move forward so that the cleaning robot can bypass the moving obstacle and continue the cleaning operation.

[0147] Step S406, determine whether the moving obstacle is within the second virtual line of sight.

[0148] Specifically, it is possible to determine whether an obstacle is located between the first virtual line of sight and the second virtual line of sight from the image obtained at the current moment. If it is located, step S407 is executed; if not, step S408 is executed.

[0149] Step S407: Obstacle avoidance, with actions of steering, stopping, or proceeding.

[0150] Specifically, the cleaning robot can be stopped at the current position to wait for the moving obstacle to leave; the cleaning robot can also be made to retreat to avoid collision with the moving obstacle; or the cleaning robot can be made to turn and then continue to move forward to bypass the moving obstacle and continue the cleaning operation.

[0151] Step S408: Determine whether the moving obstacle is within the third virtual line of sight.

[0152] Specifically, it is possible to determine whether an obstacle is located between the second virtual line of sight and the third virtual line of sight from the image obtained at the current moment. If it is located, step S409 is executed; if not, step S403 is executed.

[0153] Step S409: Motion assessment, with actions of stopping or proceeding.

[0154] Specifically, first, the moving state and moving speed of the obstacle are evaluated to determine whether the cleaning robot will collide with the moving obstacle according to the current walking direction and traveling speed. If no collision will occur, the cleaning robot is made to continue moving forward; if a collision will occur, the cleaning robot is made to stop moving forward.

[0155] Based on the foregoing description, those skilled in the art can understand that the obstacle avoidance method of this embodiment can also identify moving obstacles and evaluate moving obstacles, so as to adjust the traveling strategy of the cleaning robot when the moving obstacles affect the traveling of the cleaning robot, prevent the cleaning robot from colliding with the moving obstacles, and at the same time, it can also continue to clean the corresponding area after the moving obstacles leave. Therefore, the obstacle avoidance method of this embodiment can also enable the cleaning robot to clean the area where the moving obstacles stay.

[0156] In the fifth embodiment of the present disclosure:

[0157] As Figure 18 shown, the present disclosure also provides a cleaning robot. At the hardware level, this cleaning robot includes a processor, optionally also includes a memory and a bus, and in addition, this cleaning robot is also allowed to include other hardware required for other services.

[0158] Among them, the memory is used to store execution instructions, which are specifically computer programs that can be executed. Further, the memory may include a memory and a non-volatile memory, and provide the execution instructions and data to the processor. Exemplarily, the memory may be a high-speed random access memory (Random-Access Memory, RAM), and the non-volatile memory may be at least one disk memory.

[0159] Among them, the bus is used to interconnect the processor, the memory, and the network interface together. The bus may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, Figure 18 only a bidirectional arrow is used in the figure, but this does not mean that there is only one bus or one type of bus.

[0160] In a feasible implementation manner of the above cleaning robot, the processor may first read the corresponding execution instructions from the non-volatile memory into the memory and then run, or may first obtain the corresponding execution instructions from other devices and then run. When the processor executes the execution instructions stored in the memory, it can implement the obstacle avoidance method in any of the above obstacle avoidance method embodiments of the present disclosure.

[0161] Those skilled in the art can understand that the above obstacle avoidance method can be applied to a processor or implemented with the help of a processor. Exemplarily, a processor is an integrated circuit chip with the ability to process signals. During the process of the processor executing the above obstacle avoidance method, each step of the above obstacle avoidance method can be completed by an integrated logic circuit in hardware form or an instruction in software form in the processor. Further, the above processor can be a general-purpose processor, such as a Central Processing Unit (CPU), a Network Processor (NP), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, a microprocessor, and any other conventional processor.

[0162] Those skilled in the art can also understand that the steps of the above obstacle avoidance method embodiment of the present disclosure can be executed and completed by a hardware decoding processor, or can be executed and completed by a combination of hardware and software modules in the decoding processor. Among them, the software module can be located in other mature storage media in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. This storage media is located in the memory, and after the processor reads the information in the memory, it combines its hardware to complete the execution of the steps in the above obstacle avoidance method embodiment.

[0163] So far, the description of the technical solution of the present disclosure has been completed with reference to the accompanying drawings and in combination with the above embodiments.

[0164] It should be noted that in order to highlight the differences between the above multiple embodiments of the present disclosure, the above multiple embodiments of the present disclosure are arranged and described in a parallel manner and / or a progressive manner, and the subsequent embodiments only focus on explaining the differences between them and other embodiments. The same or similar parts between each embodiment can be referred to each other. For example, for the device / product embodiment, since the device / product embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the corresponding parts of the method embodiment.

[0165] The above is only the embodiment of the present disclosure and is not used to limit the present disclosure. For those skilled in the art, the present disclosure can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the technical principle of the present disclosure shall fall within the protection scope of the present disclosure.

Claims

1. A method for avoiding obstacles of a cleaning robot, wherein the cleaning robot is provided with a camera, characterized in that: The obstacle circumvention method comprises: Acquire an image in front of the cleaning robot through the camera; Generate a virtual travel line and a virtual sight line on the image, wherein the virtual travel line on the image corresponds to a virtual line formed by extending forward from the left and right edges of the cleaning robot in the actual environment, and the virtual sight line in the image corresponds to a virtual line at a preset safety distance in front of the cleaning robot in the actual environment; The obstacle in the image is located on a side of the virtual travel line and the virtual sight line far from the cleaning robot as a constraint condition, and the cleaning robot is controlled to walk around the obstacle.

2. The obstacle circumvention method according to claim 1, characterized in that: The step of controlling the cleaning robot to walk around the obstacle using the constraint condition that the obstacle in the image is located on a side of the virtual travel line and the virtual sight line far from the cleaning robot comprises: When an obstacle in the image contacts the virtual sight line, the cleaning robot is controlled to walk around the obstacle, with the tangency between the obstacle in the image and the virtual travel line as a constraint condition.

3. The obstacle circumvention method according to claim 1, characterized in that: The step of controlling the cleaning robot to walk around the obstacle using the constraint condition that the obstacle in the image is located on a side of the virtual travel line and the virtual sight line far from the cleaning robot comprises: When an obstacle in the image contacts the virtual sight line, determining a width and a length estimate of the obstacle according to the image; Determining an obstacle circumvention endpoint of the cleaning robot according to the estimated length of the obstacle, wherein the obstacle circumvention endpoint and the current position of the cleaning robot are located on opposite sides of the obstacle respectively; Planning an obstacle avoidance path according to the current position and the obstacle avoidance endpoint; The cleaning robot is caused to walk along the obstacle avoidance path, and when the virtual sight line and / or the virtual travel line contacts the obstacle, the obstacle avoidance endpoint is updated according to the image acquired during the travel process, and the subsequent obstacle avoidance path is adjusted according to the updated obstacle avoidance endpoint.

4. The obstacle circumvention method according to claim 3, characterized in that: The planning of the obstacle avoidance path according to the current position and the obstacle avoidance endpoint includes: The distance between the current position and the obstacle avoidance end point is determined, and an arc having a diameter equal to the distance and connecting the current position and the obstacle avoidance end point is used as the obstacle avoidance path.

5. The obstacle circumvention method according to claim 3, characterized in that: The planning of the obstacle avoidance path according to the current position and the obstacle avoidance endpoint includes: Taking the line between the current position and the obstacle avoidance end point as the first axis; Determining a second axis perpendicular to the first axis according to the width of the obstacle and the size of the cleaning robot; The major axis and the minor axis are respectively the first axis and the second axis, and an elliptical arc connecting the current position and the obstacle avoidance endpoint is used as the obstacle avoidance path.

6. The obstacle circumvention method according to claim 3, characterized in that: The planning of the obstacle avoidance path according to the current position and the obstacle avoidance endpoint includes: Determine a plurality of intermediate points between the current position and the obstacle avoidance endpoint, wherein the intermediate points are located outside the obstacle; A line connecting the current position, the middle point, and the obstacle avoidance end point in sequence is used as the obstacle avoidance path.

7. The obstacle circumvention method according to any one of claims 1 to 6, characterized in that: The step of generating a virtual travel line and a virtual sight line on the image comprises: According to the predetermined virtual travel line and virtual sight line mapped to the position in the calibration image taken by the camera, the virtual travel line and the virtual sight line are added to the image.

8. The obstacle circumvention method according to claim 7, characterized in that: The obstacle circumvention method also includes: Acquiring a calibration image of a calibration object through the camera; the calibration object having a first line for representing the virtual travel line and a second line for representing the virtual sight line; Positions of the first line and the second line in the calibration image are determined.

9. The obstacle circumvention method according to any one of claims 1 to 6, characterized in that: The obstacle circumvention method also includes: In response to the cleaning robot turning, determining a turning trajectory of the cleaning robot; The virtual travel line is adjusted to extend along the steering trajectory.

10. A cleaning robot, characterized in that: The cleaning robot comprises a processor, a memory and execution instructions stored in the memory, wherein the execution instructions are configured to enable the cleaning robot to perform the obstacle avoidance method according to any one of claims 1 to 9 when executed by the processor.

Images