Cleaning robot control method, cleaning robot and storage medium

Abstract

The invention discloses a cleaning robot control method, which comprises the following steps: determining a first motion path of a cleaning robot performing edge cleaning motion along an obstacle according to acquired first obstacle information; determining blind area information and non-blind area information of the obstacle relative to the visual range of the cleaning robot so as to determine the junction of the non-blind area and the blind area; under the condition that the cleaning robot moves to the position corresponding to the junction of the non-blind area and the blind area along the first movement path, the cleaning robot is controlled to obtain second obstacle information; and controlling the cleaning robot to perform edge cleaning motion along the obstacle according to the second obstacle information. According to the method, blind area judgment can be performed on the obstacle in the visual field of the robot in advance, then the cleaning robot is controlled to normally move on the edge path without the blind area, and the edge path is re-planned when the cleaning robot reaches the blind area, so that the frequency of repeatedly adjusting the posture of the cleaning robot around the obstacle is reduced, and the cleaning efficiency around the obstacle is improved; and the cleaning effect around the obstacle is improved.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent cleaning equipment, and in particular to a cleaning robot control method, a cleaning robot and a computer-readable storage medium. Background Art

[0002] In the related art, when a cleaning robot cleans along the edge of an obstacle during a cleaning task, it generally moves along the boundary of the obstacle for a fixed distance and then re-observes, and then moves along the boundary of the obstacle for a fixed distance and then re-observes based on the observation result, and repeats this cycle until the robot completely avoids the obstacle and cleans along the edge of the obstacle. However, in this way, the robot needs to adjust its posture and re-observe many times, and its cleaning route is not adaptive to different obstacles, so the cleaning efficiency of the area where the obstacle is located is low, and the cleaning effect is poor. Summary of the invention

[0003] The present application provides a cleaning robot control method, a cleaning robot and a computer-readable storage medium.

[0004] The cleaning robot control method involved in the embodiment of the present application includes:

[0005] Determining, based on the acquired first obstacle information, a first motion path for the cleaning robot to perform edge cleaning motion along the obstacle;

[0006] Determine the blind area information and non-blind area information of the obstacle relative to the visual range of the cleaning robot to determine the boundary between the non-blind area and the blind area;

[0007] When the cleaning robot moves along the first motion path to a position corresponding to the junction of the non-blind area and the blind area, controlling the cleaning robot to obtain second obstacle information;

[0008] The cleaning robot is controlled to perform edge cleaning motion along the obstacle according to the second obstacle information.

[0009] In this way, the present application can sense the obstacles within the robot's field of view in advance before performing edge cleaning movements on the obstacles, determine the blind spots on the image of the obstacle where the actual outline cannot be determined, and then control the cleaning robot to move normally on the edge path without the corresponding blind spots to avoid the obstacles. When the cleaning robot reaches the corresponding blind spot, it re-observes the part of the obstacle where the outline cannot be determined to re-plan the edge route, reducing the frequency of the cleaning robot repeatedly adjusting its posture to correct the route around the obstacle, effectively improving the cleaning efficiency around the obstacle, and effectively improving the cleaning effect on the area around the obstacle.

[0010] In certain embodiments, the method further comprises:

[0011] constructing an obstacle map according to the first obstacle information;

[0012] After acquiring the second obstacle information, the obstacle map is updated according to the first obstacle information and the second obstacle information.

[0013] In this way, the present application can construct and update an obstacle map for describing the spatial position of obstacles after acquiring two different sets of obstacle information.

[0014] In some embodiments, determining, based on the acquired first obstacle information, a first motion path for the cleaning robot to perform edge cleaning motion along the obstacle includes:

[0015] A first motion path for the cleaning robot to perform edge cleaning motion along the obstacle is determined according to at least one of the first semantic information, the first position information, and the first size information of the obstacle.

[0016] In this way, the present application can preliminarily determine the first motion path according to the basic conditions of the obstacle itself.

[0017] In some embodiments, determining the blind spot information of the obstacle relative to the visible range of the cleaning robot includes:

[0018] Acquire a contour point cloud image of the obstacle;

[0019] According to the visible range of the cleaning robot and the contour point cloud image, the blind area information of the obstacle relative to the visible range is determined, and the blind area is an area on the obstacle surface where the actual contour cannot be determined according to the contour point cloud image.

[0020] In this way, the present application can determine the blind spot of the obstacle relative to the visual range of the robot based on the contour point cloud image of the obstacle and the visual range of the cleaning robot, that is, the area where the actual contour cannot be determined based on the point cloud.

[0021] In some embodiments, the determining, according to the visible range of the cleaning robot and the contour point cloud image, the blind area information of the obstacle relative to the visible range also includes:

[0022] The visible range is determined according to the attribute information of the cleaning robot.

[0023] In this way, the present application can determine the visual range of the cleaning robot itself according to the attribute information of the cleaning robot itself and the devices arranged on the cleaning robot, thereby facilitating the determination of the blind spots.

[0024] In some embodiments, determining the blind spot information of the obstacle relative to the visible range according to the visible range of the cleaning robot and the contour point cloud image includes:

[0025] In the case that there is an area outside the visible range on the contour point cloud image, it is determined that the area where the obstacle is located outside the visible range is located in the blind spot of the robot.

[0026] In this way, the present application can determine the portion of the obstacle that is outside the visible range as the portion that is within the blind spot.

[0027] In some embodiments, determining the blind spot information of the obstacle relative to the visible range according to the visible range of the cleaning robot and the contour point cloud image includes:

[0028] When there is at least one point in the contour point cloud image whose height is greater than a preset threshold, it is determined that the portion of the obstacle corresponding to the point in the contour point cloud image whose height is greater than the preset threshold is located in the blind spot of the cleaning robot.

[0029] In this way, the present application can achieve that when there are points on the obstacle that are too high, the portion of the obstacle that cannot be directly observed from the contour point cloud image is determined as the portion located in the blind spot.

[0030] The determining, according to the visible range of the cleaning robot and the contour point cloud image, the blind area information of the obstacle relative to the visible range includes:

[0031] In the case where there is at least one point whose height is greater than a preset threshold value on the contour point cloud image except for the area determined not to be within the blind spot of the cleaning robot, it is determined that the part of the contour point cloud image on the obstacle corresponding to the point whose height is greater than the preset threshold value is located within the blind spot of the cleaning robot.

[0032] In this way, the present application can achieve that when there are points on the obstacle that are too high, the portion of the obstacle that cannot be directly observed from the contour point cloud image is determined as the portion located in the blind spot.

[0033] In some embodiments, determining the blind spot information of the obstacle relative to the visible range according to the visible range of the cleaning robot and the contour point cloud image includes:

[0034] In the case that there is a field of view obstruction within the visible range, it is determined that the area of ​​the obstacle blocked by the field of view obstruction is located in the blind spot of the robot.

[0035] In this way, the present application can achieve that when there is a visual obstruction within the visible range of the robot, the portion of the obstacle that is blocked by the obstruction and cannot be seen is determined as the portion located in the blind spot.

[0036] In some embodiments, when the cleaning robot moves along the first motion path to a position corresponding to the junction of the non-blind area and the blind area, controlling the cleaning robot to obtain second obstacle information includes:

[0037] When the cleaning robot moves along the first motion path to a position corresponding to the junction of the non-blind area and the blind area, controlling the cleaning robot to move in a direction away from the obstacle;

[0038] The second obstacle information is acquired, where the second obstacle information includes at least one of second semantic information, second position information, and second size information.

[0039] In some embodiments, controlling the cleaning robot to move in a direction away from the obstacle includes:

[0040] The cleaning robot is controlled to move to a preset position in a direction away from the obstacle. When the cleaning robot is at the preset position, the obstacle is within the visible range of the cleaning robot.

[0041] In this way, the present application can control the cleaning robot to adjust its own posture when the cleaning robot moves to the blind spot on the obstacle, so as to re-observe the specific characteristics of the blind spot on the obstacle, so that the cleaning robot can re-determine the path of performing edge cleaning movement on the obstacle based on the updated obstacle characteristics.

[0042] In some embodiments, the cleaning robot control method further includes:

[0043] When all the obstacles are outside the blind area of ​​the robot, the cleaning robot is controlled to move along the first motion path.

[0044] In this way, the present application can also perform edge cleaning motion directly along the boundary of the obstacle according to the first motion path when there is no blind spot on the obstacle.

[0045] The present application also provides a cleaning robot, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, a method as described in any one of the above embodiments is implemented.

[0046] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the method as described in any one of the above embodiments is implemented.

[0047] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0049] Figure 1 A schematic diagram of a flow chart of a cleaning robot control method provided in an embodiment of the present application;

[0050] Figure 2A Schematic diagram of the application scenario of the cleaning robot control method provided in the embodiment of the present application Figure 1 ;

[0051] Figure 2B Schematic diagram 2 of an application scenario of the cleaning robot control method provided in an embodiment of the present application;

[0052] Figure 2C Schematic diagram of the application scenario of the cleaning robot control method provided in the embodiment of the present application Figure 3 ;

[0053] Figure 3 A schematic diagram of a flow chart of a cleaning robot control method provided in an embodiment of the present application;

[0054] Figure 4 A schematic diagram of an application scenario of the cleaning robot control method provided in an embodiment of the present application;

[0055] Figure 5 A schematic diagram of an application scenario of the cleaning robot control method provided in an embodiment of the present application;

[0056] Figure 6 A schematic diagram of an application scenario of the cleaning robot control method provided in an embodiment of the present application;

[0057] Figure 7 A schematic diagram of an application scenario of the cleaning robot control method provided in an embodiment of the present application;

[0058] Figure 8 A schematic diagram of an application scenario of the cleaning robot control method provided in an embodiment of the present application;

[0059] Fig. 9A schematic diagram of an application scenario of the cleaning robot control method provided in an embodiment of the present application;

[0060] Fig.10 A schematic diagram of a flow chart of a cleaning robot control method provided in an embodiment of the present application;

[0061] Fig.11 Schematic diagram of application scenario of the cleaning robot control method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the embodiments of the present application.

[0063] During the normal operation of the cleaning robot, obstacles are often encountered. Since the shapes of obstacles vary and the range that the robot can detect is also limited, in the related art, when the cleaning robot cleans along the edge of an obstacle during the cleaning task, it generally moves along the edge of the obstacle for a fixed distance and then re-observes, and then moves along the edge of the obstacle for a fixed distance based on the observation results and re-observes, and repeats this cycle until the robot avoids the obstacle and cleans along the edge, so as to avoid problems with the robot's movement near unknown areas of obstacles. However, such a solution causes the robot to repeatedly cycle between observation and movement during operation, which is very inefficient.

[0064] In this context, Figure 1 As shown, the present application provides a cleaning robot control method, comprising the following steps:

[0065] 01: Determine a first motion path for the cleaning robot to perform edge cleaning motion along the obstacle according to the acquired first obstacle information;

[0066] 02: Determine the blind area information and non-blind area information of the obstacle relative to the visual range of the cleaning robot to determine the boundary between the non-blind area and the blind area;

[0067] 03: When the cleaning robot moves along the first motion path to a position corresponding to the junction of the non-blind area and the blind area, controlling the cleaning robot to obtain second obstacle information;

[0068] 04: Control the cleaning robot to perform edge cleaning motion along the obstacle according to the second obstacle information.

[0069] The present application also provides a cleaning robot control device, and the cleaning robot control method of the present application can be implemented by the cleaning robot control device. Specifically, the cleaning robot control device includes an initial path determination module, a blind spot identification module, an observation module, and a motion control module, wherein the initial path determination module is used to determine the first motion path of the cleaning robot for edge cleaning motion along the obstacle based on the acquired first obstacle information, the blind spot identification module is used to determine the blind spot information and non-blind spot information of the obstacle relative to the visual range of the cleaning robot to determine the junction of the non-blind spot and the blind spot, the observation module is used to control the cleaning robot to obtain the second obstacle information when the cleaning robot moves along the first motion path to the position corresponding to the junction of the non-blind spot and the blind spot, and the motion control module is used to control the cleaning robot to perform edge cleaning motion along the obstacle based on the second obstacle information.

[0070] The present application also provides a cleaning robot, including a memory and a processor, the memory storing a computer program, the processor being used to determine a first motion path for the cleaning robot to perform edge cleaning motion along an obstacle based on acquired first obstacle information, and to determine blind spot information and non-blind spot information of the obstacle relative to the visible range of the cleaning robot to determine the junction between the non-blind spot and the blind spot, and to control the cleaning robot to obtain second obstacle information when the cleaning robot moves along the first motion path to a position corresponding to the junction between the non-blind spot and the blind spot, and to use a motion control module to control the cleaning robot to perform edge cleaning motion along the obstacle based on the second obstacle information.

[0071] Specifically, in order to improve the efficiency of the cleaning robot in performing edge cleaning motion along obstacles, firstly, a binocular sensor or other type of sensor provided on the cleaning robot is used to obtain first obstacle information of the target obstacle, which generally includes information describing the shape and size of the obstacle itself and environmental information of the obstacle, such as the location information of the obstacle in a map. After obtaining the first obstacle information, the cleaning robot uses it as a basis to preliminarily determine a first motion path for the cleaning robot to perform edge cleaning motion along the obstacle. The first motion path may be a relatively rough motion path for the cleaning robot to perform edge cleaning motion along the periphery of the obstacle, or it may be a path for edge cleaning motion determined by the cleaning robot based on the portion of the obstacle itself that can be accurately observed.

[0072] In addition, due to the visual range of the cleaning robot itself and the accuracy of related sensors, there are often parts on the obstacle that the cleaning robot cannot accurately distinguish and observe. In order to enable the cleaning robot to obtain a relatively accurate edge motion path for these parts that cannot be accurately distinguished, in some examples, such as Figure 2A to Figure 2BAs shown, the control robot performs a visual screening and judgment of the obstacles according to its own visible range, and determines a part of the obstacle that cannot be accurately distinguished or determined within the visible range of the cleaning robot as the blind spot of the obstacle relative to the cleaning robot, and obtains blind spot information for describing the relevant characteristics of the blind spot, while a part of the obstacle that can be accurately determined by itself is determined as a non-blind spot.

[0073] Finally, when the cleaning robot performs edge cleaning motion around an obstacle along the first motion path, if the cleaning robot reaches the position P corresponding to the junction of the determined blind area and the non-blind area, the cleaning robot is controlled to re-observe and obtain the second obstacle information of the obstacle, so as to prepare data for the cleaning robot to re-define the path for performing edge cleaning motion along the obstacle. Figure 2A to Figure 2B The application scenario shown, Figure 2A and Figure 2B The dotted line portion corresponds to the first motion path, the circle represents the cleaning robot, the arrow on the first motion path represents the current moving direction of the cleaning robot, and the arrow in the cleaning robot represents the visual range of the cleaning robot ( Figure 2B It should be noted that the definition of the visible range of the cleaning robot will be described in detail later. When the cleaning robot moves along the first motion path to a position P corresponding to the junction of the non-blind area and the blind area of ​​the cleaning robot (the boundary of the blind area), the cleaning robot is controlled to reacquire obstacle information, that is, corresponding to the second obstacle information, and the acquired second obstacle information is used as the basic data for re-planning the motion path, so that the cleaning robot can perform relatively accurate edge cleaning movements on the obstacles according to the second obstacle information.

[0074] In this way, the present application can sense the obstacles within the robot's field of view in advance before performing edge cleaning movements on the obstacles, determine the blind spots on the image of the obstacle where the actual outline cannot be determined, and then control the cleaning robot to move normally on the edge path without the corresponding blind spots to avoid the obstacles. When the cleaning robot reaches the corresponding blind spot, it re-observes the part of the obstacle where the outline cannot be determined to re-plan the edge cleaning route, reducing the frequency of the cleaning robot repeatedly adjusting its posture to correct the route around the obstacles, effectively improving the cleaning efficiency around the obstacles, and effectively improving the cleaning effect on the area around the obstacles.

[0075] It can be understood that the robot in the embodiment of the present application can obtain obstacle information again each time it moves along the motion path to the blind spot during the process of cleaning the obstacles along the edge, that is, the obstacle information can be obtained multiple times during the process of cleaning the obstacles along the edge.

[0076] In some embodiments, the cleaning robot control method further includes:

[0077] constructing an obstacle map according to the first obstacle information;

[0078] After acquiring the second obstacle information, the obstacle map is updated according to the first obstacle information and the second obstacle information.

[0079] In some embodiments, the cleaning robot control device further includes a mapping module, which is used to construct an obstacle map according to the first obstacle information, and to update the obstacle map according to the first obstacle information and the second obstacle information after acquiring the second obstacle information.

[0080] In some embodiments, the processor is further configured to construct an obstacle map according to the first obstacle information, and to update the obstacle map according to the first obstacle information and the second obstacle information after acquiring the second obstacle information.

[0081] Specifically, the movement navigation of the cleaning robot is generally based on a map drawn according to the surrounding environment. Figure 1 Generally describes the relative position of each obstacle in the working environment. Therefore, on the basis of the above-mentioned implementation, in order to enable the robot to accurately bypass obstacles and reacquire obstacle information, after obtaining the first obstacle information, the cleaning robot directly constructs and draws an obstacle map based on the first obstacle information to describe the information observed in the current working environment. The above-mentioned obstacle map may include but is not limited to obstacle contour information, a first motion path, etc. After the robot completes the judgment of the blind spot and the non-blind spot on the obstacle, the cleaning robot can also mark the junction of the blind spot and the non-blind spot determined by the judgment on the obstacle map, so that the robot can reacquire the second obstacle information when it moves to the junction of the above-mentioned blind spot and the non-blind spot according to the obstacle map. After obtaining the second obstacle information, due to the update of the obstacle information, the map also needs to be updated with the update of the obstacle information.

[0082] In this way, after acquiring two different sets of obstacle information, the present application can construct and update an obstacle map for describing the spatial position of obstacles, so that the obstacle map is closer to the actual situation of the obstacles.

[0083] In some embodiments, step 01: determining a first motion path of the cleaning robot for edge cleaning motion along the obstacle according to the acquired first obstacle information, including:

[0084] A first motion path for the cleaning robot to perform edge cleaning motion along the obstacle is determined according to at least one of the first semantic information, the first position information, and the first size information of the obstacle.

[0085] In some embodiments, the initial path determination module is used to determine a first motion path for the cleaning robot to perform edge cleaning motion along the obstacle based on at least one of the first semantic information, the first position information, and the first size information of the obstacle.

[0086] In some embodiments, the processor is further used to determine a first motion path for the cleaning robot to perform edge cleaning motion along the obstacle based on at least one of the first semantic information, the first position information, and the first size information of the obstacle.

[0087] Specifically, the first obstacle information specifically includes but is not limited to at least one of the first semantic information, the first position information and the first size information of the obstacle. The first semantic information generally has a mapping relationship with the image information of the obstacle itself, and is generally obtained through identification and comparison based on a preset semantic information library and the image information of the obstacle itself, and is used to describe the type or attribute of the obstacle itself. For example, semantic information such as "shoes" and "vase" that represents the type, and semantic information such as "low obstacles" that represent the attributes. The first position information generally indicates the position or position range of the obstacle in the world coordinate system established by the cleaning robot itself based on the surrounding environment, and is generally expressed by coordinate values ​​or coordinate ranges. The first size information generally indicates the scale of the obstacle in three-dimensional space, and can generally also be expressed using coordinate values ​​in the world coordinate system or actual length values.

[0088] As for how to determine the first motion path, in some embodiments, such as Figure 2B As shown, since the motion calculation of the cleaning robot is generally based on its geometric center, in order to determine the first motion path, in addition to the distance along the edge of the obstacle, the size information of the cleaning robot needs to be added to determine the expansion size of the obstacle. The size information of the cleaning robot is generally expressed as the radius of the cleaning robot. The expansion size refers to the distance between the center of the cleaning robot and the outer contour of the obstacle when the cleaning robot avoids the obstacle and cleans along the obstacle. In some examples, the radius of the robot is 25 cm, the semantic information of the obstacle is "slippers", and the corresponding distance along the edge is 3 cm, then the expansion size is confirmed to be 28 cm. Then, after determining the expansion size, the outer contour of the obstacle is expanded outward by the expansion distance mentioned above, and the shape needs to be kept unchanged and the continuity of the curve needs to be maintained. The curve obtained in this way is the expansion contour. The expansion contour is the original state of the first motion path. After multiple processing steps, the expansion contour can be converted into the initially determined first motion path.

[0089] As for the specific determination of the first motion path, it can be achieved based on at least one of the above three types of information. For example, in some examples, the cleaning robot obtains the position range of the target obstacle in the world coordinate system based on the first position information in the first obstacle information, and then determines the above expansion size based on the outer contour of the position range, and finally determines a relatively rough first motion path that enables the cleaning robot to completely perform edge cleaning motion along the periphery of the obstacle. For example, in other examples, the cleaning robot determines the type of obstacle, the position range it occupies, and the size of the obstacle based on the first semantic information, the first position information, and the first size information in the first obstacle information. At the same time, a part of the occupied position range exceeds the visual range of the cleaning robot, so the above information can be combined to determine a first motion path for edge cleaning motion that is determined for the part of the obstacle that can be accurately observed.

[0090] In this way, the present application can preliminarily determine the first motion path according to the basic conditions of the obstacle itself.

[0091] It is worth noting that the "position P corresponding to the boundary between the blind area and the non-blind area" described in the embodiment of the present application refers to the intersection of the extension line of the boundary between the blind area and the non-blind area on the first motion path of the cleaning robot and the first motion path. Exemplarily, when the structural center of the cleaning robot moves to position P, it is considered that the cleaning robot has moved to the position corresponding to the boundary between the blind area and the non-blind area.

[0092] like Figure 3 As shown, in some embodiments, step 02: determining the blind area information and non-blind area information of the obstacle relative to the visual range of the cleaning robot to determine the junction of the non-blind area and the blind area, includes:

[0093] 021: Get the outline point cloud image of the obstacle;

[0094] 022: According to the visible range of the cleaning robot and the contour point cloud image, determine the blind area information of the obstacle relative to the visible range.

[0095] In some embodiments, the blind spot recognition module is used to obtain a contour point cloud image of the obstacle, and to determine the blind spot information of the obstacle relative to the visible range according to the visible range of the cleaning robot and the contour point cloud image.

[0096] In some embodiments, the processor is further used to obtain a contour point cloud image of the obstacle, and to determine blind spot information of the obstacle relative to the visible range based on the visible range of the cleaning robot and the contour point cloud image.

[0097] Specifically, regarding the determination of the blind spot of the obstacle relative to the cleaning robot, in some examples, the cleaning robot may be provided with a visual sensor (such as a binocular sensor), through which the outline point cloud image of the obstacle can be obtained, and the image analysis based on the outline point cloud image of the obstacle can determine the blind spot of the obstacle relative to the cleaning robot. In order to facilitate the analysis of the image, generally only the outline point cloud image used to describe the outer contour of the obstacle is obtained. However, the image range obtained by the visual sensor is limited, that is, the visible range of the cleaning robot is limited, so when analyzing and processing the obtained outline point cloud information, the boundary of the visible range of the cleaning robot should also be taken into account, and finally the part of the obstacle within the visible range that can be accurately distinguished by the cleaning robot can be obtained, that is, the so-called non-blind area, and the part of the obstacle within the visible range that cannot be accurately distinguished by the cleaning robot, that is, the so-called blind area. Specifically in the above example, the blind spot information of the obstacle relative to the visible range of the cleaning robot refers to the point cloud information of the area where the actual contour of the obstacle surface cannot be accurately determined according to the outline point cloud image.

[0098] It should be noted that the method of obtaining point cloud images in the present application includes but is not limited to obtaining point cloud images through visual sensors, laser ranging sensors (such as line structured light, radar, etc.).

[0099] In this way, the present application can determine the blind spot of the obstacle relative to the visual range of the robot based on the contour point cloud image of the obstacle and the visual range of the cleaning robot, that is, the area where the actual contour cannot be determined based on the point cloud.

[0100] In some embodiments, before step 022, the method further includes:

[0101] The visible range is determined according to the property information of the cleaning robot.

[0102] In some embodiments, the blind spot identification module is also used to determine the visible range according to the attribute information of the cleaning robot.

[0103] In some embodiments, the processor is further used to determine the visible range based on attribute information of the cleaning robot.

[0104] Specifically, the visual range of the cleaning robot should generally be included in the maximum range that the cleaning robot's visual sensor can distinguish. In addition, the visual range is generally affected by the installation method and properties of the sensors or other visual devices installed on the cleaning robot. For example, different installation methods such as installation height and installation angle, as well as the differences in the visual amplitude angle, visual range limit and other properties of the device itself will affect the visual range of the cleaning robot. In other words, the visual range of the cleaning robot can be changed by changing the above-mentioned influencing factors.

[0105] In this way, the present application can determine the visual range of the cleaning robot itself according to the attribute information of the cleaning robot itself and the devices arranged on the cleaning robot, thereby facilitating the determination of the blind spots.

[0106] In some embodiments, step 022 includes:

[0107] In the case that there is an area outside the visible range on the contour point cloud image, it is determined that the area where the obstacle is outside the visible range is within the blind spot of the robot.

[0108] In some embodiments, the blind spot recognition module is further used to determine that the area where the obstacle is located outside the visible range is within the blind spot of the robot when there is an area outside the visible range on the contour point cloud image.

[0109] In some embodiments, the processor is further configured to determine that, when there is an area outside the visible range on the contour point cloud image, the area where the obstacle is located outside the visible range is within the blind spot of the cleaning robot.

[0110] Specifically, several different types of blind spots and corresponding determination methods are described below.

[0111] First, the situation where the part beyond the visible range boundary is judged to be in the blind area is explained, such as Figure 4 The situation shown, Figure 4 It is a schematic diagram of the top-down perspective of the visual range of the cleaning robot. Points A and B in the figure represent one of a pair of visual sensors installed on the cleaning robot respectively. D is the midpoint of the line connecting points A and B. Rays AF and AC are the field of view boundaries of visual sensor A. Rays BE and BC are the field of view boundaries of visual sensor B. Therefore, rays AF and BE are the side boundaries of the field of view of the visual sensor pair. Line segment EF is the front boundary of the field of view of the visual sensor pair. G is the midpoint of line segment EF. Objects between the front boundary EF of the field of view and the visual sensor pair AB cannot be accurately focused by the visual sensor pair AB because they are too close to the visual sensor pair. Therefore, they cannot be accurately distinguished by the visual sensor pair AB. Line segment HI is the rear boundary of the field of view of the visual sensor pair. J is the midpoint of line segment HI. Objects outside the rear boundary HI of the field of view and far away from the visual sensor pair AB have exceeded the farthest viewing distance of the visual sensor pair AB and cannot be accurately distinguished by the visual sensor pair AB. Then Figure 4 The middle quadrilateral EFIH can be considered as the boundary of the visible range of the cleaning robot, and the interior of the quadrilateral represents the visible range of the cleaning robot.

[0112] And in Figure 5In the case shown, a part NCL (counterclockwise) of the object MNCLK exceeds the front boundary EF of the field of view in the visible range of the cleaning robot, that is, corresponding to the contour point cloud image, the NCL part is located between the front boundary EF of the field of view and the visual sensor pair AB. Therefore, the image corresponding to the NCL part cannot be accurately distinguished, while MN and LK can be accurately distinguished. Therefore, at this time, the NCL part on the object MNCLK is located outside the visible range of the cleaning robot. Therefore, it is determined that the NCL part of the object is located in the blind spot of the cleaning robot.

[0113] For example, Figure 6 In the case shown, a part ML (counterclockwise) of the object KML exceeds the side boundary BH of the field of view within the visible range of the cleaning robot, that is, on the contour point cloud image, the image corresponding to the right side of point M exceeds the side boundary of the contour point cloud image, so that the corresponding point cloud cannot be acquired. Therefore, at this time, the right side of point M on the object KML is judged to be located in the blind spot of the cleaning robot.

[0114] In addition to exceeding the visible range boundary in the horizontal direction, the same situation also exists in the vertical direction. Figure 7 The situation shown, Figure 7 The figure shows a schematic diagram of the side view of the visual range of the cleaning robot. Figure 7 Point O in the figure represents the visual sensor pair, rays OB and OG are the vertical boundaries of the visual field, ray AB is the ground, and line segment CD is the front boundary of the visual sensor pair's field of view (corresponding to Figure 4 The line segment EF in the figure is the line segment EF in the figure, and the line segment GF is the rear boundary of the field of view of the visual sensor pair (corresponding to Figure 4 The line segment HI in the figure shows that the quadrilateral CDGF is the boundary of the visual range of the cleaning robot, and the interior of the quadrilateral CDGF is the visual range of the cleaning robot. Figure 7 In the case shown, if a part of an obstacle is too high in the vertical direction and exceeds the ray OG, it is also considered to be beyond the visual range of the cleaning robot, and the part beyond the ray OB will be determined to be within the blind spot of the cleaning robot.

[0115] In this way, the present application can determine the portion of the obstacle that is outside the visible range as the portion that is within the blind spot.

[0116] In some embodiments, step 022 further includes:

[0117] When there is at least one point in the contour point cloud image whose height is greater than a preset threshold, it is determined that a portion of the obstacle corresponding to the point in the contour point cloud image whose height is greater than the preset threshold is located in the blind spot of the cleaning robot.

[0118] In some embodiments, the blind spot identification module is also used to determine that the part corresponding to the point in the contour point cloud image on the obstacle with a height greater than a preset threshold is located within the blind spot of the cleaning robot when there is at least one point in the contour point cloud image with a height greater than a preset threshold.

[0119] In some embodiments, the processor is also used to determine that the portion corresponding to the point in the contour point cloud image on the obstacle with a height greater than a preset threshold is located within the blind spot of the cleaning robot when there is at least one point in the contour point cloud image with a height greater than a preset threshold.

[0120] Specifically, the following describes a case where a point on an object has a height exceeding a threshold value and is judged to be partially within the blind spot. Figure 8 In the situation shown, the obstacle JK does not exceed the visible range of the obstacle in the horizontal direction, and it is known that it does not exceed the visible range of the obstacle in the vertical direction. However, on the side close to the visual sensor pair AB, in the contour point cloud image corresponding to the approximate arc JK, there are points in the vertical direction whose height exceeds the preset height threshold. Due to the nature of the contour point cloud image, if there is a point on the obstacle whose height is obviously higher than other parts of the object, the point cloud near the point will appear sparse, so that the cleaning robot cannot accurately distinguish the contour of the obstacle at that location based on the sparse point cloud. If there are more points exceeding the preset height threshold on the contour point cloud image corresponding to the approximate arc JK (clockwise), the visual sensor pair AB cannot accurately identify the contour of the obstacle far away from the visual sensor pair AB.

[0121] So Figure 8 In the situation shown, if on the side close to the visual sensor pair AB, in the contour point cloud image corresponding to the approximate arc JK, there is a point in the vertical direction whose height exceeds the preset height threshold, then the part corresponding to the point on the obstacle whose height exceeds the preset height threshold is judged to be within the blind spot of the cleaning robot. In this way, the two situations mentioned above can be solved at the same time, that is, the part whose height exceeds the threshold will be judged to be within the blind spot of the cleaning robot, and at the same time, the part away from the visual sensor pair AB that is blocked by the part whose height exceeds the threshold is also equivalent to being judged to be within the blind spot of the cleaning robot.

[0122] As for the preset height threshold, it can be determined according to the first semantic information in the first obstacle information. For example, if the first semantic information in the first obstacle information is "shoes", and the corresponding height threshold is 7cm, then the parts corresponding to the points on the obstacle with the semantic information "shoes" that are higher than 7cm will be judged to be in the blind spot of the cleaning robot. For another example, if the first semantic information in the first obstacle information is "pet feces", and the corresponding height threshold is 15cm, then the parts corresponding to the points on the obstacle with the semantic information "pet feces" that are higher than 15cm will be judged to be in the blind spot of the cleaning robot. Generally speaking, the more obstacles the cleaning robot needs to stay away from, the larger the preset height threshold.

[0123] In this way, the present application can achieve that when there are points on the obstacle that are too high, the portion of the obstacle that cannot be directly observed from the contour point cloud image is determined as the portion located in the blind spot.

[0124] In some embodiments, step 022 further includes:

[0125] When there is at least one point whose height is greater than a preset threshold value on the contour point cloud image except for the area determined not to be within the blind spot of the cleaning robot, it is determined that the part corresponding to the point whose height is greater than the preset threshold value in the contour point cloud image on the obstacle is located within the blind spot of the cleaning robot.

[0126] In certain embodiments, the blind spot identification module is also used to determine that the portion corresponding to the point in the contour point cloud image on the obstacle with a height greater than a preset threshold is located in the blind spot of the cleaning robot, when there is at least one point with a height greater than a preset threshold in the contour point cloud image except for the area determined not to be within the blind spot of the cleaning robot.

[0127] In certain embodiments, the processor is also used to determine that a portion of the contour point cloud image on the obstacle corresponding to the point whose height is greater than a preset threshold is located within the blind spot of the cleaning robot, except for an area on the contour point cloud image determined not to be within the blind spot of the cleaning robot.

[0128] Specifically, based on the above implementation, Figure 8In the situation shown, on the side far away from the visual sensor pair AB, there are points on the contour point cloud image corresponding to the approximate arc JK (clockwise) whose height exceeds the preset height threshold, while on the side close to the visual sensor pair AB, there is no such situation in the contour point cloud image corresponding to the approximate arc KJ. In this case, there are no points on the approximate arc KJ (clockwise) that are located in the blind spot of the cleaning robot. Due to the nature of the contour point cloud image, if there is a point on the obstacle that is significantly higher than other parts of the object, the point cloud near the point will appear sparse, so that the cleaning robot cannot accurately distinguish the contour of the obstacle at that location based on the sparse point cloud. Therefore, for the approximate arc JK, the portion corresponding to the point on the obstacle whose height exceeds the preset height threshold is determined to be within the blind spot of the cleaning robot.

[0129] In this way, the present application can achieve that when there are points on the obstacle that are too high, the portion of the obstacle that cannot be directly observed from the contour point cloud image is determined as the portion located in the blind spot.

[0130] In some embodiments, step 022 further includes:

[0131] In the case where there is a field of view obstruction within the visible range, it is determined that the area where the obstacle is blocked by the field of view obstruction is located within the blind area of ​​the robot.

[0132] In some embodiments, the blind spot identification module is further used to determine that the area blocked by the obstacle is within the blind spot of the robot when there is a visual field obstruction within the visible range.

[0133] In some embodiments, the processor is further configured to, when there is a field of view obstruction within the visible range, determine that an area where the obstacle is blocked by the field of view obstruction is within the blind spot of the robot.

[0134] Specifically, the following describes the situation where there are other visual field obstructions within the visible range.

[0135] like Fig. 9 In the situation shown, there is a field of view obstruction KI between the object NPQ and the visual sensor pair O, which causes the MQ part on the object NPQ to be unable to be seen by the visual sensor O. Therefore, similar to the situation beyond the visible range, the point cloud corresponding to the MQ part on the contour point cloud image cannot be obtained, and therefore it is determined that the MQ part of the object is located in the blind spot of the cleaning robot.

[0136] In this way, the present application can achieve that when there is a visual obstruction within the visible range of the robot, the portion of the obstacle that is blocked by the obstruction and cannot be seen is determined as the portion located in the blind spot.

[0137] like Fig.10 As shown, in some embodiments, step 03 includes:

[0138] 031: When the cleaning robot moves along the first motion path to a position corresponding to the junction of the non-blind area and the blind area, control the cleaning robot to move in a direction away from the obstacle;

[0139] 032: Get the second obstacle information;

[0140] The second obstacle information includes at least one of second semantic information, second position information and second size information.

[0141] In some embodiments, the observation module is also used to control the cleaning robot to move away from the obstacle when the cleaning robot moves along the first motion path to a position corresponding to the junction of the non-blind area and the blind area, and to obtain second obstacle information.

[0142] In some embodiments, the processor is also used to control the cleaning robot to move away from the obstacle when the cleaning robot moves along the first motion path to a position corresponding to the junction of the non-blind area and the blind area, and to obtain second obstacle information.

[0143] Specifically, the process of re-observing obstacles is explained by taking the cleaning robot performing edge cleaning motion along an obstacle on the right side of its forward direction as an example.

[0144] Please refer again to Figure 2A to Figure 2C The application scenario shown, Figure 2A to Figure 2C The dashed line portion corresponds to the first motion path, the circle represents the cleaning robot, the arrow on the first motion path indicates the current direction of travel of the cleaning robot, and the arrow inside the cleaning robot indicates the direction of the cleaning robot's visible range. Exemplarily, the cleaning robot is controlled to move to a preset position away from an obstacle, and when the cleaning robot is in the preset position, the obstacle is within the cleaning robot's visible range. When the cleaning robot moves along the first motion path to the junction of the non-blind area and the blind area (i.e., the boundary of the cleaning robot's blind area), as shown in FIG. Figure 2C and Fig.11 As shown in the scenario, the cleaning robot is controlled to leave the first motion path, move away from the obstacle and adjust the visible range to face the obstacle, so that the obstacle is within the visible range of the cleaning robot. This is done to re-observe the obstacle completely to update the obstacle information, thereby re-determining the path of the edge cleaning movement for the obstacle.

[0145] Please note that, please refer to Figure 2CAs shown, "the obstacle is within the visible range of the cleaning robot" only means that the obstacle is within the boundary of the visible range EFIH of the cleaning robot, but the obstacle observed by the cleaning robot when it is in this position may still have a blind spot, but the obstacle blind spot determined at this time is at least partially different from the obstacle blind spot determined by the cleaning robot last time.

[0146] After the above-mentioned posture adjustment is completed, the second obstacle information of the obstacle can be obtained through the visual device on the cleaning robot. The second obstacle information specifically includes but is not limited to at least one of the semantic information, position information and size information of the obstacle. The second semantic information generally has a mapping relationship with the image information of the obstacle itself, and is generally obtained by reconfirming through identification and comparison based on a preset semantic information library and the image information of the obstacle itself, and is used to describe the type or attribute of the obstacle itself. For example, semantic information such as "shoes" and "vase" that represents the type, and semantic information such as "low obstacles" that represent the attribute. The second position information generally indicates the position or position range of the obstacle reconfirmed in the world coordinate system established by the cleaning robot itself based on the surrounding environment, and is generally expressed in coordinate values ​​or coordinate ranges. The second size information generally indicates the reconfirmed scale of the obstacle in three-dimensional space, and can generally also be expressed using coordinate values ​​in the world coordinate system or actual length values.

[0147] In order to avoid information pollution, in an embodiment of the present application, after the second obstacle information is obtained, the first obstacle information can be updated to the second obstacle information to avoid the cleaning robot being misled by the first obstacle information and resulting in an unreasonable new path for edge cleaning movement.

[0148] In addition, similar to the process of determining the first motion path based on the first obstacle information, the cleaning robot can determine a new path for edge cleaning movement along the obstacle based on the second obstacle information just acquired, thereby controlling the cleaning robot to perform edge cleaning movement along the obstacle based on the second obstacle information.

[0149] In this way, the present application can control the cleaning robot to adjust its own posture when the cleaning robot moves to the blind spot on the obstacle, so as to re-observe the specific characteristics of the blind spot on the obstacle, so that the cleaning robot can re-determine the path of performing edge cleaning movement on the obstacle based on the updated obstacle characteristics.

[0150] In some embodiments, the cleaning robot control method further includes:

[0151] When all obstacles are located outside the blind area of ​​the robot, the cleaning robot is controlled to move along the first motion path.

[0152] In some embodiments, the motion control module is further used to control the cleaning robot to move along the first motion path when all obstacles are located outside the blind area of ​​the robot.

[0153] In some embodiments, the processor is further configured to control the cleaning robot to move along the first motion path when all obstacles are outside the blind spot of the robot.

[0154] Specifically, in some cases, there is no part of the obstacle within the blind spot of the cleaning robot's visible range. In this case, the first motion path previously determined is a reasonable, undisturbed edge cleaning motion path, and the cleaning robot can be controlled to move along this path to perform the edge cleaning task.

[0155] In this way, the present application can also perform edge cleaning motion directly along the boundary of the obstacle according to the first motion path when there is no blind spot on the obstacle.

[0156] The present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by one or more processors, the method in any one of the above embodiments is implemented.

[0157] In the description of this specification, the descriptions with reference to the terms "certain embodiments", "in an example", "exemplarily", etc., mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0158] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0159] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A cleaning robot control method, characterized in that: The method comprises: Determining, based on the acquired first obstacle information, a first motion path for the cleaning robot to perform edge cleaning motion along the obstacle; Determine the blind area information and non-blind area information of the obstacle relative to the visual range of the cleaning robot to determine the boundary between the non-blind area and the blind area; When the cleaning robot moves along the first motion path to a position corresponding to a junction between the non-blind area and the blind area, controlling the cleaning robot to obtain second obstacle information; The cleaning robot is controlled to perform edge cleaning motion along the obstacle according to the second obstacle information.

2. The method according to claim 1, characterized in that The method further comprises: constructing an obstacle map according to the first obstacle information; After acquiring the second obstacle information, the obstacle map is updated according to the first obstacle information and the second obstacle information.

3. The method according to claim 1, characterized in that The step of determining a first motion path for the cleaning robot to perform edge cleaning motion along the obstacle according to the acquired first obstacle information comprises: A first motion path for the cleaning robot to perform edge cleaning motion along the obstacle is determined according to at least one of the first semantic information, the first position information, and the first size information of the obstacle.

4. The method according to claim 1, characterized in that: The determining of the blind area information of the obstacle relative to the visual range of the cleaning robot includes: Acquire a contour point cloud image of the obstacle; According to the visible range of the cleaning robot and the contour point cloud image, the blind area information of the obstacle relative to the visible range is determined, and the blind area is an area on the obstacle surface where the actual contour cannot be determined according to the contour point cloud image.

5. The method according to claim 4, characterized in that The step of determining blind area information of the obstacle relative to the visible range according to the visible range of the cleaning robot and the contour point cloud image may also include: The visible range is determined according to the attribute information of the cleaning robot.

6. The method according to claim 4, characterized in that The determining, according to the visible range of the cleaning robot and the contour point cloud image, the blind area information of the obstacle relative to the visible range includes: In the case that there is an area outside the visible range on the contour point cloud image, it is determined that the area where the obstacle is located outside the visible range is located in the blind spot of the cleaning robot.

7. The method according to claim 4, characterized in that The determining, according to the visible range of the cleaning robot and the contour point cloud image, the blind area information of the obstacle relative to the visible range includes: When there is at least one point in the contour point cloud image whose height is greater than a preset threshold, it is determined that the portion of the obstacle corresponding to the point in the contour point cloud image whose height is greater than the preset threshold is located in the blind spot of the cleaning robot.

8. The method according to claim 4, characterized in that The determining, according to the visible range of the cleaning robot and the contour point cloud image, the blind area information of the obstacle relative to the visible range includes: When there is at least one point whose height is greater than a preset threshold value on the contour point cloud image except for the area determined not to be within the blind spot, it is determined that the part corresponding to the point whose height is greater than the preset threshold value in the contour point cloud image on the obstacle is located within the blind spot of the cleaning robot.

9. The method according to claim 4, characterized in that The determining, according to the visible range of the cleaning robot and the contour point cloud image, the blind area information of the obstacle relative to the visible range includes: In the case that there is a field of view obstruction within the visible range, it is determined that the area where the obstacle is blocked by the field of view obstruction is located in the blind spot of the cleaning robot.

10. The method according to claim 1, characterized in that When the cleaning robot moves along the first motion path to a position corresponding to a junction between the non-blind area and the blind area, controlling the cleaning robot to obtain second obstacle information includes: When the cleaning robot moves along the first motion path to a position corresponding to the junction of the non-blind area and the blind area, controlling the cleaning robot to move in a direction away from the obstacle; The second obstacle information is acquired, where the second obstacle information includes at least one of second semantic information, second position information, and second size information.

11. The method according to claim 10, characterized in that The controlling the cleaning robot to move in a direction away from the obstacle comprises: The cleaning robot is controlled to move to a preset position in a direction away from the obstacle. When the cleaning robot is at the preset position, the obstacle is within the visible range of the cleaning robot.

12. The method according to claim 1, characterized in that The method further comprises: When all the obstacles are located outside the blind area of ​​the cleaning robot, the cleaning robot is controlled to move along the first motion path.

13. A cleaning robot, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 12 is implemented.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the method according to any one of claims 1 to 12 is implemented.

Images