Self-moving robot and working method therefor

Abstract

The present application provides a self-moving robot and a working method therefor. The self-moving robot is provided with a first sensor and a second sensor. The first sensor is used to acquire an image, and the second sensor performs detection on the surrounding environment via signal transmission or via collision. Correspondingly, the working method for the self-moving robot comprises: on the basis of the image acquired by the first sensor in combination with detection information from the second sensor, determining whether a target object requiring edge-following is present; and, if such a target object is present, at least on the basis of the image acquired by the first sensor, determining an edge-following path for traveling along an outer contour of the target object, and traveling in accordance with the edge-following path. According to the solution provided in the present application, images and distance information acquired by different types of sensors are combined to determine whether a target object requiring edge-following is present, thereby greatly improving accuracy. Moreover, the algorithm is simple and the cost is low.

Description

Self-moving robot and working method thereof

[0001] The present application claims priority to the Chinese patent application No. 202411295173.0, filed on September 16, 2024, and entitled "Self-moving robot and working method thereof", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of cleaning equipment, in particular to a self-moving robot and a working method thereof. BACKGROUND

[0003] With the progress of science and technology and the improvement of people's living quality, intelligent electrical appliances have entered the lives of more and more people, such as self-moving robots. The self-moving robot can be a cleaning robot, a mowing robot, a robot providing housekeeping services, etc. Taking the cleaning robot as an example, the cleaning robot can automatically complete cleaning, dusting, mopping, etc. on the ground, providing considerable help to the sanitary conditions of the living environment. However, the cleaning coverage of the self-moving robot on the market is limited, and there is still room for improvement in the improvement of cleaning efficiency.

[0004] The existing self-moving robot has poor edge-following trajectory when walking along the edge when encountering obstacles, such as obstacles with irregular edges or low obstacles, and there is still a situation of missed cleaning. SUMMARY

[0005] In order to solve the technical problem of poor edge-following of the existing self-moving robot, the present application provides a self-moving robot and a working method thereof.

[0006] In the first embodiment of the present application, a self-moving robot working method is provided. The self-moving robot is provided with a first sensor and a second sensor, the first sensor is used to collect images, and the second sensor is used to collect distance information. The self-moving cleaning robot working method comprises:

[0007] Combining the images collected by the first sensor and the distance information collected by the second sensor, it is determined whether there is a target object that needs to follow the edge;

[0008] When there is, at least according to the images collected by the first sensor, an edge-following path following the outer contour of the target object is determined, and the edge-following path is followed;

[0009] When there is not, the planned path is followed.

[0010] In a second embodiment of the present application, a self-moving robot working method is provided. The self-moving robot is provided with a first sensor and a second sensor. The first sensor is used to collect images, and the second sensor is used to detect the surrounding environment by emitting signals or by collision. The self-moving robot working method comprises:

[0011] When both the image collected by the first sensor and the detection information of the second sensor determine that there is a target object, the type of the target object is determined in combination with the image and the detection information.

[0012] If the type of the target object is a type that needs to follow the edge, an edge-following path is determined based on the image collected by the first sensor and / or the detection information of the second sensor, and the self-moving robot walks along the edge-following path.

[0013] If the type of the target object is a non-edge-following type, the self-moving robot walks along a planned path.

[0014] In a third embodiment of the present application, a self-moving robot working method is provided. The self-moving robot is provided with a first sensor for collecting images. The self-moving robot working method comprises:

[0015] An obstacle is identified based on the image collected by the first sensor, and the obstacle is a first type of object.

[0016] The outer contour information of the obstacle is extracted according to the image.

[0017] The self-moving robot walks along the edge according to the outer contour information.

[0018] In a fifth embodiment of the present application, a self-moving robot working method is provided. The self-moving robot is provided with a first sensor and a second sensor. The first sensor is used to collect ground images, and the second sensor is used to obtain detection information of surrounding obstacles. The method comprises:

[0019] The image collected by the first sensor and the detection information of the second sensor are combined to determine whether the obstacle is a target object that needs to follow the edge.

[0020] If yes, an edge-following path along the outer contour of the target object is determined based on the image collected by the first sensor, and the self-moving robot walks along the edge-following path.

[0021] In a sixth embodiment of the present application, a self-moving robot working method is provided. The self-moving robot is provided with a first sensor and a second sensor. The first sensor is used to collect ground images, and the second sensor is used to obtain detection information of obstacles at a predetermined height. The method comprises:

[0022] determining, in combination with the ground image collected by the first sensor and the detection information of the second sensor, that an obstacle in the ground image is a target object that needs to be followed along an edge, the height of the obstacle not conforming to the predetermined height;

[0023] determining, based on the image collected by the first sensor, an edge-following path along the outer contour of the target object, and walking along the edge-following path.

[0024] In a seventh embodiment of the present application, a self-moving robot working method is provided, wherein the self-moving robot is provided with a first sensor and a second sensor, the first sensor is used to collect an image, and the second sensor is used to obtain detection information of a surrounding obstacle; and the method comprises:

[0025] when the target object is determined based on both the image collected by the first sensor and the detection information of the second sensor, determining the position of the target object in combination with the image and the detection information;

[0026] if the position of the target object is of a need-to-follow-along-an-edge type, determining an edge-following path based on the image collected by the first sensor, and walking along the edge-following path;

[0027] if the position of the target object is of a non-follow-along-an-edge type, walking along a planned path.

[0028] In an eighth embodiment of the present application, a self-moving robot working method is also provided, wherein the self-moving robot is provided with a first sensor and a second sensor, the first sensor is used to collect an image, and the second sensor is used to detect a surrounding environment by emitting a signal or by collision. And the method comprises:

[0029] determining the type of the target object based on the image collected by the first sensor and / or the detection information of the second sensor;

[0030] if the type of the target object is of a need-to-follow-along-an-edge type, determining an edge-following path along the outer contour of the target object based on at least the image collected by the first sensor, and walking along the edge-following path.

[0031] In a ninth embodiment of the present application, a self-moving robot working method is also provided, wherein the self-moving robot is provided with a first sensor and a second sensor, the first sensor is used to collect an image, and the second sensor is used to detect a surrounding environment by emitting a signal or by collision. And the method comprises:

[0032] When the target object is determined according to the image collected by the first sensor and the detection information of the second sensor, the type of the target object is determined according to the image and the detection information.

[0033] When the type of the target object is the type that needs to be followed along the edge, an edge-following path is determined according to the image collected by the first sensor and / or the detection information of the second sensor, and the robot moves along the edge-following path.

[0034] In a tenth embodiment of the present application, a self-moving robot is provided. The self-moving robot comprises:

[0035] a body, on which a walking assembly is arranged;

[0036] a first sensor arranged on the body and configured to collect an image;

[0037] a second sensor arranged on the body and configured to collect distance information;

[0038] a processor electrically connected to the first sensor, the second sensor and the walking assembly, and configured to execute the steps in the embodiments of the working method of the robot provided in the present application.

[0039] In the technical scheme provided in an embodiment of the present application, the image collected by the first sensor and the detection information of the second sensor are combined to determine whether there is a target object that needs to be followed along the edge. When it is determined that there is a target object that needs to be followed along the edge, an edge-following path along the outer contour of the target object is determined according to at least the image collected by the first sensor, and the robot moves along the edge-following path. The determination that there is a target object that needs to be followed along the edge may, for example, include determining that the type of the target object is the type that needs to be followed along the edge. It can be seen that the scheme provided in the present embodiment greatly improves the accuracy by combining the image and distance information collected by different types of sensors to determine whether there is a target object that needs to be followed along the edge, and the algorithm is simple, the cost is low, and almost all obstacles can be exhausted and classified.

[0040] In the technical scheme provided in another embodiment of the present application, when an obstacle is identified based on the image collected by the first sensor, the outer contour information of the obstacle can be directly extracted from the collected image and used to follow along the edge, or further, the confidence can be used to extract the outer contour information of the obstacle from the image only when the confidence meets the requirements, and the robot follows along the edge according to the outer contour information. The accuracy is improved, and the effect of following along the edge is very good.

[0041] Yet another embodiment of the present application provides a technical solution in which, when both the image collected by the first sensor and the detection information of the second sensor determine that there is a target object, the position of the target object is determined in combination with the image and the detection information, and when the target object is determined to be of the edge-following type according to the position of the target object, the edge-following path is determined based on the image collected by the first sensor, and the robot walks along the edge-following path; and when the target object is determined to be of the non-edge-following type according to the position of the target object, the robot walks along the planned path. This solution determines whether the target object is of the edge-following type according to the position of the target object, so that the robot can avoid false edge-following and reduce the occurrence of room missing. BRIEF DESCRIPTION OF DRAWINGS

[0042] The accompanying drawings, which are included to provide a further understanding of the present application and are incorporated in and constitute a part of this application, illustrate embodiments of the present application and serve to explain the present application. In the drawings:

[0043] FIG. 1 is a schematic view of a chair with a U-shaped chair leg;

[0044] FIG. 2 is a schematic view of a sawtooth edge-following trajectory of a conventional self-moving robot walking along a wavy edge;

[0045] FIG. 3 is a schematic view of various sensors provided on a conventional robot according to an embodiment of the present application;

[0046] FIG. 4-A is a schematic view of the front end of a self-moving robot according to an embodiment of the present application;

[0047] FIG. 4-B is a schematic view of the front end of another self-moving robot according to an embodiment of the present application;

[0048] FIG. 5 is a flowchart of a working method of a self-moving robot according to an embodiment of the present application;

[0049] FIG. 6 is a flowchart of a working method of a self-moving robot according to another embodiment of the present application;

[0050] FIG. 7 shows a schematic view of the principle of AI line extraction;

[0051] FIGS. 8-A, 8-B and 8-C are flowcharts of working methods of self-moving robots according to other embodiments of the present application;

[0052] FIG. 9 shows a schematic view corresponding to scenario one;

[0053] FIG. 10 shows a schematic view of the principle of a determination process in scenario two in which there is a step at a door;

[0054] FIG. 11 shows a schematic view of an edge-following path corresponding to a U-shaped chair leg in scenario three;

[0055] Fig. 12 shows a schematic diagram of a wave-shaped curtain rod corresponding to the edge path in scenario four;

[0056] Fig. 13 shows a schematic diagram of an electric wire;

[0057] Fig. 14 shows a schematic diagram of a self-moving robot;

[0058] Figs. 15-17 are flowcharts of the working method of the self-moving robot according to other embodiments of the present application. DETAILED DESCRIPTION

[0059] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below with reference to the embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative work fall within the scope of protection of the present application.

[0060] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, component or device that includes a list of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent in such a process, method, component or device. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, component or device including the element. In some of the processes, methods, components and devices described in the specification, claims and drawings of the present application, a plurality of operations appear in a specific order. These operations can not be performed in the order they appear in this document or in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and the operations can be performed in sequence or in parallel. It should be noted that the descriptions "first", "second", etc. in this document are used to distinguish different modes, devices, modules, etc., and do not represent the order of precedence. Also, "first" and "second" are not different types.

[0061] In the prior art, when a self-moving robot encounters an obstacle, some self-moving robots change the motion posture of the robot by colliding the bump plate on the robot body with the obstacle to avoid the obstacle; some self-moving robots use sensors on the robot to identify the obstacle, and then bypass the obstacle based on the identified obstacle.

[0062] The method for detecting obstacles by means of active collision of a bump plate. When a self-moving robot cleans along the edge of an obstacle, its cleaning path is irregular. From the perspective of the robot's behavior, the robot collides and then retreats and adjusts the direction; then it walks again and cannot completely travel along the edge of the obstacle.

[0063] Obstacles are identified by sensors on the robot, and then edge cleaning is performed based on the identified obstacles. At present, most robots use ranging sensors to identify obstacles.

[0064] For example, a face array TOF (Time of Flight) ranging sensor. Its principle is that the transmitter emits a face array light signal, which propagates in the air and is reflected back by the target object; the receiver receives the light signal reflected by the reflector, and calculates the distance of each light signal in the air by measuring the round-trip time of the light signal. The face array TOF ranging sensor can achieve high precision and large field of view distance measurement, and the face array TOF itself is expensive and requires a processor with high computing power to cooperate, and the overall use cost is very high. If the server's computing power is used to cooperate with the face array TOF, there is a large time delay phenomenon, which does not meet the needs of the real-time processing environment of the robot.

[0065] For example, some self-moving robots are provided with LDS laser ranging radar. After the LDS laser ranging radar rotates to measure the distance information on the horizontal plane, the robot can avoid obstacles and build maps in real time based on the distance information of the points on the horizontal plane at a predetermined height measured by the LDS laser ranging radar. However, the LDS can only detect obstacles at a predetermined height, and cannot measure or accurately measure low objects such as ground wires, table legs, chair legs, and the bottom wave-shaped curtain legs of floor curtains.

[0066] For example, a line laser sensor is provided on a self-moving robot. The line laser sensor emits a line of light signals in the vertical direction, and the receiver receives the reflected signals. The distance of multiple points on the target object at different heights in the vertical direction is calculated based on the round-trip time of the light signal or the triangulation method. At a certain moment, based on the distance information measured by the line laser sensor, the distances of multiple points on the target object in the vertical direction can be obtained. To obtain the information of the target object, the self-moving robot needs to collect the distance information of multiple points in the vertical direction at different positions of the target object multiple times, for example, 360-degree rotation, and then through relatively complex calculation, the contour of the target object can be identified. Its field of view is narrow and cannot obtain the contour of the object in real time.

[0067] There are also self-moving robots provided with edge sensors, such as infrared light couplings, which indirectly detect the distance between the robot body and the obstacle by detecting the returned light intensity signal in real time through the edge sensor. When walking along the edge, the walking path is adjusted by the light intensity signal measured in real time, so as to walk along the edge of the obstacle while maintaining a safe distance from the obstacle. However, the accuracy is not high, the walking path is zigzag, and there is a larger missed scanning area when walking along the edge.

[0068] The above-mentioned sensors, especially the sensors with ranging function, all obtain the distance information of some points on the object by emitting light signals and receiving the light signals reflected by the object, and by calculating the time length of light signal emission and reflection. These sensors can be collectively referred to as point laser sensors or structured light sensors (or line laser sensors).

[0069] There are some problems in realizing edge-following along the obstacle edge by means of the distance information collected by the point laser sensor or the structured light sensor. In the scenario of a straight edge, such as a wall or a wardrobe, the edge of which is straight, the edge-following effect is relatively good when the distance information collected by the structured light sensor is used to realize edge-following. However, for irregular edges, such as the wave-shaped edge formed by the curtain rod of a floor curtain, the U-shaped leg of a U-shaped chair, etc., the edge-following effect of the robot is poor. For example, when the robot performs edge-following based on the distance information collected by the point laser sensor or the structured light sensor, the edge-following trajectory of the robot is zigzag (as shown in FIG. 2), rather than a wave-shaped trajectory that fits the wave-shaped edge of the obstacle.

[0070] In the prior art, there are also some mobile robots provided with visual sensors, such as cameras, etc., but they are basically used for real-time photography or video shooting, and the pictures or videos are transmitted to the user. The visual sensor is not used for edge-following path planning, etc.

[0071] It needs to be supplemented here that the "edge-following" mentioned in the embodiment refers to walking along the edge of the obstacle while maintaining a certain safe distance from the obstacle.

[0072] In order to further improve the intelligent degree of the robot, the self-moving robot of the present application is provided with a visual sensor on the robot body. The visual sensor can be, but is not limited to, an intelligent visual sensor supporting local AI (Artificial Intelligence) or network-side AI calculation, such as an intelligent camera. The visual sensor can identify the obstacle and extract the outer contour information of the obstacle through the collected images. Through a large amount of training and a better AI algorithm (such as a neural network), the behavior of the self-moving robot is made more intelligent by using the visual sensor.

[0073] Fig. 3 shows a schematic diagram of a robot body provided with a vision sensor, a ranging sensor (e.g., an edge-following ranging sensor) and a laser radar. As shown in Fig. 3, the ranging sensor can only measure the portion of the obstacle with a first height h1, and the laser radar can only measure the portion of the obstacle with a second height h2. It can be seen that the low obstacle (e.g., an object with a height below the first height h1) cannot be detected by the ranging sensor and the laser radar. The low obstacle can be detected by the vision sensor.

[0074] The following embodiments of the present application provide a low-cost solution that can significantly improve the edge-following effect of a self-moving robot. Before introducing the working method of the self-moving robot, the hardware of the self-moving robot is briefly introduced.

[0075] The self-moving robot can be a cleaning robot (e.g., a sweeping robot, a mopping robot, a sweeping and mopping integrated robot, etc.), a mowing robot, a detection robot, a guiding service robot, etc.

[0076] The following embodiments of the present application provide a low-cost solution that can significantly improve the edge-following effect of a self-moving robot. Before introducing the working method of the self-moving robot, the hardware of the self-moving robot is briefly introduced.

[0077] Referring to FIGS. 4-A and 4-B, the self-moving robot can include, but is not limited to, a body 3, a dust cleaning assembly (not explicitly shown in the figures), a mop cleaning assembly (not explicitly shown in the figures), a walking assembly (not explicitly shown in the figures), a sensing assembly 4, a control device, and a side brush assembly (not explicitly shown in the figures). The dust cleaning assembly, the mop cleaning assembly, the walking assembly, the sensing assembly 4, and the control device are all arranged on the body 3. The dust cleaning assembly can include, but is not limited to, a dust box, a dust cleaning fan, a roller brush, a first motor, and the like. The first motor is used to drive the roller brush to rotate. The control device includes a hardware part and a software part. The hardware part is, for example, a mainboard. The mainboard can be provided with a processor, a storage medium (such as a memory), and the like. The software part is a computer program stored in the storage medium. The processor executes these computer programs to control the components of the self-moving robot, so that the self-moving robot has corresponding functions, such as mapping, path planning, obstacle recognition, obstacle avoidance cleaning, edge cleaning, base station return and docking, area recognition, cleaning mode switching (only dust cleaning, only mop cleaning, or dust cleaning followed by mop cleaning), and the like. The walking assembly can include a drive wheel and a drive wheel motor. The drive motor outputs corresponding power under the control of the mainboard to drive the drive wheel to rotate, realizing the forward movement, backward movement, stopping, turning, and the like of the self-moving robot. Further, the walking assembly can also include a universal wheel, which is a follow-up wheel and can be arranged at the front of the body. The side brush assembly can be one or two. The mop cleaning assembly can include, but is not limited to, a mop cleaning execution member (such as a mop, a roller, or a caterpillar roller), a second motor, a liquid supply mechanism, a dirt removal mechanism, and the like. The second motor drives the mop cleaning execution member to rotate, the liquid supply mechanism provides cleaning liquid to the mop cleaning execution member, the mop cleaning execution member wipes the ground, and the dirt removal mechanism scrapes and collects the dirt on the mop cleaning execution member. Correspondingly, the body 3 can also be provided with a sewage tank for storing the dirt collected by the dirt removal mechanism.

[0078] In the example shown in FIGS. 4-A and 4-B, in some embodiments of the present application, the sensing assembly 4 can be arranged on the front side of the body 3. The front side and the rear side of the body 3 are distinguished in terms of the walking direction of the self-moving robot. When the self-moving robot moves forward, the side facing forward is the front side. The sensing assembly 4 can detect the environment of the space in front of the walking direction of the self-moving robot to realize the identification of the working area (such as the cleaning area) and the identification of the obstacles in front, thereby assisting the self-moving robot to safely move forward.

[0079] The sensing assembly 4 can collect environmental information such as images, ranging information. The processor can identify boundary information (such as walls, table legs, cabinet corners, etc.) such as objects to be crossed (such as low thresholds), obstacles that cannot be crossed, and work environments (such as clean environments) based on the environmental information collected by the sensing assembly 4, and control the walking assembly based on the identification result to prevent the self-moving robot from colliding. In addition, the processor can also build a map based on the environmental information collected by the sensing assembly 4 to plan a walking path based on the built map and navigate the self-moving robot with the help of the sensing assembly 4.

[0080] In some embodiments of the present application, the sensing assembly 4 can include but is not limited to a second sensor module 42 for realizing image acquisition and structured light detection functions.

[0081] In an embodiment, when the second sensor module 42 realizes the structured light detection function, the first device 421 and the third device 423 on both sides can be used as transmitters and the second device 422 in the middle can be used as a receiver, and the second device 422 can also be used as a camera to realize the image acquisition function. In this case, the second sensor described in the present application can include the second sensor module 42 for detecting the surrounding environment by emitting signals (such as detecting the distance, range, appearance, and other information of objects), and the second device 422 in the middle of the second sensor module 42 can be used as the first sensor described in the present application to acquire images.

[0082] Alternatively, in another embodiment, when the second sensor module 42 realizes the structured light detection function, the second device 422 in the middle can be used as a transmitter and one of the first device 421 and the second device 423 on both sides can be used as a receiver, and the other of the first device 421 and the second device 423 can be used as a camera to realize the image acquisition function. In this case, the second sensor described in the present application can include the second device 422 in the middle of the second sensor module 42 and one of the first device 421 and the third device 423 on both sides to detect the surrounding environment by emitting signals, and the other of the first device 421 and the third device 423 on both sides of the second sensor module 42 can be used as the first sensor described in the present application to acquire images.

[0083] Further, the sensing assembly 4 can also include other sensor modules, such as the first sensor module 41. In specific implementation, the first sensor module can be an LDS (Laser Distance Sensor). The LDS can be used to collect environmental information for mapping, ranging, etc. The specific functions of the LDS are described in detail and can also be referred to existing content. Accordingly, the second sensor described in the present application can also include the second sensor module 41 such as an LDS.

[0084] Figure 4-A shows a schematic diagram of a first sensor module 41 (e.g. LDS) disposed in the body 3 (e.g. disposed in the middle of the front side of the body 3 (or referred to as the waist part)) and a second sensor module 42 disposed above the first sensor module 41. Figure 4-B shows a schematic diagram of a first sensor module 41 disposed at the top of the front end of the body 3 and a second sensor module 42 disposed below the first sensor module 41.

[0085] From the above, the first sensor described in the present application is used to collect images, and the first sensor can be a visual sensor. The second sensor is used to detect the surrounding environment by emitting signals, and the second sensor can be a point laser sensor (e.g. LDS) or a structured light sensor. In addition, in some other embodiments, the second sensor can also detect the surrounding environment by collision, and the second sensor is a collision sensor disposed at the front end of the body 3.

[0086] It should be noted here that the sensors on the self-moving robot are usually multiple, and in the case where the second sensor is used to detect the surrounding environment by emitting signals, the second sensor referred to in the present application can be understood as a second sensor module, which can include multiple sensors for realizing different functions respectively. For example, the second sensor module includes sensors for collecting environmental information to build a map, sensors for measuring the height of an object, sensors for measuring the distance of an object, etc. That is, the second sensor can include but is not limited to at least one of the following: a line laser sensor, an LDS laser ranging sensor, a face array TOF, etc. The above line laser sensor, LDS laser ranging sensor, etc. are all sensors that receive active light (e.g. receive the light reflected back by the emitted laser). The first sensor (visual sensor) in the present application is a camera without a transmitter, which is a passive natural light picture collector.

[0087] The following will introduce the working method of the self-moving robot provided by the embodiments of the present application.

[0088] Figure 5 shows a flowchart of a working method of a self-moving robot according to an embodiment of the present application. The working method of the self-moving robot includes:

[0089] 101. Determine whether there is a target object that needs to be followed along the edge based on the image collected by the first sensor and the detection information of the second sensor;

[0090] 102. When there is, determine an edge-following path along the outer contour of the target object based on at least the image collected by the first sensor, and walk along the edge-following path.

[0091] Further, the method provided by the embodiment further includes the following steps:

[0092] 103. If not, walk along the original planned path.

[0093] In the above 101, the detection information of the second sensor includes but is not limited to the distance information, orientation, width, appearance, etc. of the detected object. In the embodiment, the detection information can at least include the detected distance information. That is, in the above 101, the determination of whether there is a target object that needs to be followed along the edge can be made in combination with the image collected by the first sensor and the distance information of the second sensor.

[0094] In an implementable scheme, the above 101 "determining whether there is a target object that needs to be followed along the edge in combination with the image collected by the first sensor and the detection information of the second sensor" can include the following steps:

[0095] 1011. Determining that there is an object that needs to be followed along the edge according to the image collected by the first sensor;

[0096] 1012. Obtaining the type of the object that needs to be followed along the edge in the image;

[0097] 1013. When it is determined that there is no object that needs to be followed along the edge according to the detection information of the second sensor, then:

[0098] If the type of the object that needs to be followed along the edge in the image is the first type, it is determined that the object that needs to be followed along the edge in the image is the target object that needs to be followed along the edge.

[0099] If the type of the object that needs to be followed along the edge in the image is the second type, it is determined that there is no target object that needs to be followed along the edge.

[0100] In the above 1011-1012, the collected image can be recognized and analyzed to determine whether the image contains an object that is in contact with the ground or suspended at a certain height, so as to determine whether there is an object that needs to be followed along the edge. When it is determined that there is an object that needs to be followed along the edge in the image, the type of the object that needs to be followed along the edge can be recognized by but not limited to a semantic analysis method. Thus, the above 1012 "obtaining the type of the object that needs to be followed along the edge in the image" can include the following steps:

[0101] 10121. Recognizing semantic information of the object that needs to be followed along the edge in the image;

[0102] 10122. Determining the type of the object that needs to be followed along the edge in the image based on the semantic information.

[0103] In a specific implementation, the semantic information of the object that needs to be followed in the image can be recognized by using a machine learning algorithm (such as a deep learning algorithm or a neural network algorithm), and the type of the object that needs to be followed in the image can be determined according to the semantic information.

[0104] In 1013, for example, the second type can refer to, but is not limited to, an object with a certain step or slope, such as a step or a slope; and the first type can refer to, but is not limited to, an object such as a wire on the ground, a U-shaped leg of a sofa or a chair, a wall, a flowerpot, and the like.

[0105] Further, the step 101 can further include the following steps:

[0106] 1014, when it is determined according to the detection information of the second sensor that there is an object that needs to be followed, the object that needs to be followed determined based on the image is the object that needs to be followed determined based on the detection information.

[0107] Here, the object that needs to be followed determined based on the detection information of the second sensor and the image collected by the first sensor can be determined as the object that needs to be followed.

[0108] Further, the step 101 can further include the following steps:

[0109] S011, when it is determined based on the detection information of the second sensor that there is an object that needs to be followed, and it is also determined based on the image collected by the first sensor that there is an object that needs to be followed, the following path is planned according to the detection information of the second sensor; or

[0110] S012, when it is determined based on the detection information of the second sensor that there is an object that needs to be followed, and it is also determined based on the image collected by the second sensor that there is an object that needs to be followed, the contour information of the target object is obtained according to the image collected by the first sensor, and the following path is planned according to the contour information; or

[0111] S013, when it is determined based on the detection information of the second sensor that there is no object that needs to be followed, and it is determined based on the image collected by the first sensor that there is a first type of object that needs to be followed, the contour information of the target object is obtained according to the image collected by the first sensor, and the following path is planned according to the contour information.

[0112] In the above S011-S012, if a certain target object is determined to be an edge-following object based on both the detection information of the second sensor and the image collected by the first sensor, it can be explained that the target object is a higher obstacle, such as a regular wall, etc., which can be collected by the first sensor and detected by the second sensor (such as an LDS laser ranging sensor, etc.). In this case, the edge-following path can be directly planned according to the detection information of the second sensor to avoid obstacles, and it is not necessary to extract the outer contour information of the target object from the image collected by the first sensor to plan the edge-following path. This edge-following path planning method is relatively simple and fast.

[0113] Of course, in this case, the outer contour information of the target object can be extracted from the image collected by the first sensor, and the edge-following path can be planned according to the outer contour information or according to the outer contour information and the detection information of the second sensor to walk along the planned path.

[0114] In the above S013, if a certain target object is determined to be not an edge-following object based on the detection information of the second sensor and is determined to be an edge-following object based on the image collected by the first sensor, it can be explained that the target object is a low obstacle (such as a low step, slope, etc.), which cannot be detected by the second sensor due to the height being lower than the detection height range of the second sensor. In this case, visual edge-following is needed to avoid obstacles, that is, the outer contour information of the target object is obtained according to the image collected by the first sensor, and the edge-following path is planned according to the outer contour information to walk along the planned edge-following path to avoid obstacles.

[0115] For specific implementation description of identifying whether there is an edge-following target object according to the image collected by the first sensor and obtaining the outer contour information of the target object, please refer to the related content described in other embodiments. For example, please refer to the related content of Example 1 and Example 2 given below.

[0116] In the above 102, the edge-following path can be determined according to the image collected by the first sensor or according to the image collected by the first sensor and the detection information of the second sensor to walk along the edge-following path to avoid obstacles. For specific implementation description of the above 102, please refer to the related content described in the above S011-S012 and / or the related content given in other embodiments, which will not be described in detail here.

[0117] In the above 103, when it is determined that there is no target object that needs to follow the edge, the original planned path can be walked. The original planned path can refer to but is not limited to the path planned in advance based on the map established for the working environment, such as a bow-shaped path, a "Z" shaped path, etc.

[0118] As mentioned above, the self-moving robot can be a cleaning robot (such as a sweeping robot, a mopping robot, a sweeping and mopping integrated robot, etc.), a mowing robot, a surveying robot, a guiding service robot, etc. The first sensor provided on the self-moving robot can be a vision sensor, which can be, but is not limited to, an intelligent vision sensor (such as an intelligent camera) supporting local AI (Artificial Intelligence) or network-side AI calculation. In this application, the vision sensor provided on the self-moving robot at least has the functions of collecting an environment image, and the processor can identify whether there is an object in the image based on the collected image, and can also extract the outer contour information of the object in the image.

[0119] During the walking operation of the self-moving robot in a space site (such as a shopping mall, a house, etc.), the first sensor provided thereon can collect images of the surrounding environment in real time. When the processor of the self-moving robot identifies and confirms that the environment image contains obstacles such as sofas, U-shaped legs of tables and chairs, and electric wires, it performs a wire extraction operation to extract the outer contour information of the obstacles from the environment image. In this application, the extracted outer contour information can include horizontal and vertical outer contour information.

[0120] In specific implementation, the environment image collected by the vision sensor is generally a plane color image in RGB format, and the environment images collected at different times have the same size and do not contain object height information in vision. The vision sensor can be internally provided with a processing program (such as an image processing algorithm, a neural network model, etc.), which can be used to perform obstacle identification analysis and extraction of the outer contour information of the obstacles on the collected environment image. Alternatively, the vision sensor only has an image collection function, and the processor of the self-moving robot is configured with a processing program (such as an image processing algorithm, a calculation model, etc.), which is used to process the image to identify the obstacles and extract the outer contour information of the obstacles. The image processing algorithm can include an image recognition algorithm, an outer contour information extraction algorithm of the obstacles, etc. Among them, the image recognition algorithm is not specifically limited in this embodiment. The outer contour information extraction algorithm of the obstacles can be, but is not limited to, using color image (such as RGB image) or grayscale image technology to identify the edges or contours of the obstacles, so as to realize the extraction of the contour information of the obstacles.

[0121] The following examples are given to illustrate the principle of extracting the outer contour information of the obstacles:

[0122] Example 1: Using color image to extract the outer contour information of the obstacles

[0123] When the vision sensor collects an image (such as an RGB image) and identifies that there is an obstacle, the following steps can be performed to extract the outer contour information of the obstacle from the environment image:

[0124] S11, color space conversion is performed on the image to obtain a converted image.

[0125] Specifically, the image is converted from the RGB color space to other color spaces such as HSV, HSI or Lab, because these other color spaces are often more suitable for processing color information, because the sensitivity of human vision to brightness is much stronger than the sensitivity to color depth, so in order to facilitate color information processing and recognition, other color spaces such as HSV, HSI or Lab are often used, which are more in line with human visual characteristics than the RGB color space.

[0126] After color space conversion, the obtained converted image is an HSV image or an HSI image or a Lab image, etc., so that subsequent obstacles and backgrounds can be distinguished by more obvious color features.

[0127] S12, the converted image is segmented to separate the obstacles and the background in the converted image, so as to extract the obstacle outer contour information. The converted image can be segmented by using, but not limited to, the following methods: color threshold segmentation, edge detection algorithm, region generation algorithm, machine learning and deep learning method, etc. Specifically,

[0128] For example, assuming that the converted image is an HSV image, different color objects can be separated by adjusting the color threshold in the HSV color space of the converted image, so as to separate the obstacles and the background in the converted image. The color threshold can be set according to the color features of the obstacles, and / or the color threshold can be determined using the color histogram of the converted image. In the scheme of determining the color threshold according to the color features of the obstacles and the color histogram of the converted image, the color histogram can be used to assist in determining the best suitable color threshold.

[0129] For another example, Canny algorithm, Sobel algorithm, Prewitt algorithm, Roderts algorithm, Kirsch algorithm, etc. or wavelet transform can be used to perform edge detection on the converted image to extract the desired obstacle outer contour information from the converted image. The various edge detection algorithms given above usually identify the outline of the obstacle based on the gradient size and direction of the converted image, so as to obtain the obstacle outer contour information.

[0130] For example, a region growing algorithm can be used to extract the obstacle contour information from the converted image. The region growing algorithm starts from a set of seed points, and gradually expands to neighboring pixels according to a preset color or texture standard. In the expansion process, adjacent pixels with similar properties (specific range of gray level or color) to the seed are added to the seed of the growing region, thereby forming the complete object boundary.

[0131] For another example, a trained classifier or a deep learning model (such as a convolutional neural network (CNN) model) can be used to identify and segment the obstacle in the converted image, thereby extracting the obstacle contour information.

[0132] Example 2: Using a gray-scale image (such as a black-and-white image (also referred to as a binary image)) to extract the obstacle contour information.

[0133] When the visual sensor identifies that the collected environmental image contains an obstacle, the following steps can be performed to extract the obstacle contour information using a gray-scale image:

[0134] S21, performing a gray-scale conversion on the image to obtain a corresponding gray-scale image.

[0135] In this regard, a linear transformation or a logarithmic transformation can be used, but the application is not limited thereto. The transformation can be used to enhance the contrast of the image, so that the obstacle and the background in the image are more easily distinguished, thereby converting the image into a corresponding gray-scale image.

[0136] S22, processing the gray-scale image to extract the obstacle contour information therefrom.

[0137] In a specific implementation, the gray-scale image can be subjected to threshold segmentation, edge detection, morphological operation, region analysis, texture analysis, or the like, to extract the obstacle contour information from the gray-scale image.

[0138] In the threshold segmentation of the gray-scale image, a global threshold or an adaptive threshold can be used to convert the gray-scale image into a binary image, thereby separating the obstacle from the background. The binary image has only two gray-scale values for each pixel, specifically, 0 or 255. The binary image is also referred to as a black-and-white image. Further, to improve the accuracy of the contour extraction, the binary image can be subjected to some processing such as a smoothing filter processing and / or an inflation and corrosion processing, to remove the noise in the binary image. The smoothing filter processing can be a median filter, a mean filter, a Gaussian filter, or the like. The inflation and corrosion processing is a morphological operation.

[0139] And, when performing edge detection on the grayscale image, the edge detection algorithm used can refer to the related content described in Example 1 above. That is, the edge detection algorithm can also be used for the grayscale image to identify the edges of the obstacles in the image.

[0140] When performing morphological operations on the grayscale image, the morphological operations performed on the grayscale image can include operations such as dilation, erosion, opening operation, and closing operation, which help to remove noise and fill in gaps in the object boundary.

[0141] When performing region analysis on the grayscale image, regions can be formed by connecting adjacent pixels, and then the object boundary is determined according to the properties of the region, such as area, shape, texture, etc.

[0142] When performing texture feature analysis on the grayscale image, because the texture of the obstacle can be different from the background, analyzing the texture features of the grayscale image also helps to analyze the obstacles and the background in the image.

[0143] Of course, in addition to using the methods given in the above examples to extract the obstacle outer contour information from the environment image, other methods can also be used. For example: since the outer contour information is generally manifested by a sudden change in grayscale value, the sudden change in grayscale in the grayscale image can also be captured by methods such as differentiation, so as to extract the obstacle outer contour information therefrom. Among them, but not limited to, the Sobel operator, Laplance operator, etc. can be used to differentiate the grayscale image. The Sobel operator is also called a discrete differentiation operator or a first-order differentiation operator, which is to do Gaussian smoothing first and then do differentiation. The Laplance operator is to find the second derivative, and when finding the second derivative, the value at the maximum change is 0, i.e. the second derivative of the edge is 0. Therefore, the specific implementation way of extracting the obstacle outer contour information from the environment image in this embodiment is not specifically limited.

[0144] According to the above-obtained obstacle boundary extraction method, it is possible that all boundary lines such as the gap between the floor or the tiles, or a line artificially drawn on the ground, etc. are extracted as the obstacle boundary. In order to improve the accuracy of the obstacle boundary line extraction and reduce the false extraction, the application also provides a more optimized obstacle contour information extraction method, which is as follows:

[0145] The processing model built-in in the visual sensor is a machine learning model trained in advance, and the training process of the machine learning model can be as follows:

[0146] Obtain training samples.

[0147] The training samples include: a sample image and obstacle outer contour information in the sample image. The sample image can be artificially photographed or drawn; the obstacle outer contour information in the sample image can be artificially drawn or automatically recognized by a machine. Then, a large number of training samples are input into a machine learning model (i.e., a neural network model) for training, so that the trained machine learning model can recognize the boundary line of an object in an image on a horizontal plane and a vertical plane, and the boundary line is the outer contour information of the object (or obstacle).

[0148] Further, the machine learning model outputs the extracted obstacle outer contour information and a corresponding confidence level. The confidence level can be simply understood as a probability. That is, the higher the confidence level of the obstacle outer contour information output by the machine learning model, the more accurate the output result. The confidence level refers to the degree of belief that a certain object is an obstacle. For example, when the machine learning model performs object recognition on an input environment image to extract the boundary line of an object, if the confidence level of a certain object (such as a gap between the floor or tiles) is less than a first threshold value (such as 0.3), it is considered that the object is not an obstacle, the boundary is not extracted, and the object is considered to be non-existent; if the confidence level of a certain object (such as a relatively thick black line drawn on the ground, a relatively thin solid line (such as a piece of needle thread) on the ground, etc.) is between the first threshold value and a second threshold value (i.e., greater than or equal to the first threshold value and less than or equal to the second threshold value (such as 0.8)), it is considered that it is uncertain whether the object is an obstacle, and the object needs to be re-identified and checked to determine whether the object is an obstacle, wherein a detection sensor (such as a structured light sensor) provided on the robot can be used to realize the re-identification and checking of the object; and if the confidence level of a certain object (such as a power line on the ground (as shown in FIG. 13), a U-shaped leg of a table and chair, etc.) is greater than the second threshold value (such as 0.8), the object is directly considered to be an obstacle without re-identification, and the boundary of the object is extracted.

[0149] That is, when the confidence level is high, the self-moving robot can not need the assistance of the second sensor to collect information, and can realize edge walking based on the image collected by the first sensor. Specifically, another embodiment of the present application provides a self-moving robot working method. As shown in FIG. 6, the self-moving robot is provided with a first sensor for collecting images; and the self-moving robot working method comprises:

[0150] 201, identifying an obstacle based on the image collected by the first sensor, and the obstacle being a first type of object;

[0151] 202, extracting an outer contour of the obstacle according to the image;

[0152] 203, walking along the edge according to the outer contour information.

[0153] In the step 201, the first type of object can be, but is not limited to, a low object that cannot be detected by the second sensor, such as a low step, a slope, and the like, a wire on the ground, and the like. Since the height of the low object is lower than the detection height range of the second sensor, the low object cannot be detected by the second sensor, and thus, the vision edge following needs to be performed to avoid the obstacle. Details of the vision edge following can be found in the related content of other embodiments (e.g., the step S012 in other embodiments).

[0154] Further, between the steps 201 and 202, the method provided in the embodiment further includes the following steps:

[0155] A1, determining a confidence of the recognized obstacle;

[0156] A2, when the confidence meets a requirement, triggering the step 202 of extracting the contour information of the obstacle according to the image;

[0157] Further, when the confidence does not meet the requirement, the detection information (e.g., distance information) collected by the second sensor needs to be used for assistance. That is, the method provided in the embodiment can further include the following steps:

[0158] 204, when the confidence does not meet the requirement but belongs to a value to be verified, determining whether the obstacle recognized based on the image collected by the first sensor needs to be edge followed in combination with the detection information of the second sensor;

[0159] 205, when the edge following is needed, extracting the contour information of the obstacle according to the image collected by the first sensor and / or the detection information of the second sensor, and walking along the contour information;

[0160] 206, when the edge following is not needed, walking according to the original planned path.

[0161] The “requirement” in the step can be that the confidence is greater than a second threshold. The value of the confidence to be verified refers to that the confidence is between a first threshold and a second threshold (i.e., greater than or equal to the first threshold and less than or equal to the second threshold. When the confidence is lower than the first threshold, neither the “requirement” is met nor the value to be verified.

[0162] In summary, the edge following walking scheme based on the first sensor (e.g., a vision sensor) provided in the application can be briefly described as follows:

[0163] As shown in FIG. 7, during the walking process of the self-moving robot, after the surrounding environment image of the surrounding environment is collected by the first sensor, the obstacles in the surrounding environment image can be further identified and detected, and the line extraction processing of the identified and detected obstacles can be performed to obtain the outer contour information of the obstacles. The line extraction processing can be AI line extraction, that is, the AI model (such as a machine learning model) is used to identify and detect the obstacles in the surrounding environment image and extract the boundary line of the obstacles (i.e., the outer contour information of the obstacles). Then, the self-moving robot can plan an edge-following path according to the extracted outer contour information of the obstacles and walk along the edge-following path. At present, various sensors cannot well identify various linear objects and U-shaped chair legs (such as FIG. 1). However, the AI model used in the present application can extract the grounding lines (boundary lines or outer contour information) of various linear objects and U-shaped chairs, so as to realize edge-following walking to effectively avoid or bypass obstacles, which can effectively reduce the occurrence of robot trapping and winding and the like.

[0164] That is, the robot in the present application executes different walking strategies (such as edge-following walking or crossing) through the first sensor to collect images, which specifically includes the following cases:

[0165] Case 1:

[0166] Based on the image collected by the first sensor (such as a vision sensor), it is determined that there is an obstacle on the ground, such as a table leg on the ground, a storage box of a certain height, a wall, a power line on the ground (such as the charging line shown in FIG. 13), and the like. At this time, if the confidence of the obstacle recognition is high (i.e., greater than a second threshold), the outer contour information of the object can be directly extracted based on the image collected by the first sensor, an edge-following path is planned, and then the edge-following path is walked along to avoid the object.

[0167] Alternatively, in other embodiments, when it is confirmed that there is an object on the ground based on the image collected by the first sensor, such as a table leg on the ground, a storage box of a certain height, a wall, a power line on the ground, and the like, regardless of the confidence, in order to further improve the accuracy, the distance information collected by the second sensor can also be combined to assist in confirmation. When it is determined that the object is an obstacle that needs to be followed based on the distance information collected by the second sensor, the outer contour information of the object is extracted based on the image collected by the first sensor, an edge-following path is planned, and then the edge-following path is walked along to avoid the object.

[0168] It needs to be added here that: like the low obstacles on the ground such as the electric wire, the U-shaped table and chair legs (referred to as the first type of obstacles), the structured light sensor (such as the line laser sensor) is often difficult to detect. The visual sensor can be recognized and confirmed, so these first type of obstacles are the obstacles that need to be edge-following. Therefore, the first type of obstacles does not need to be determined by the structured light sensor. That is: for the first type of obstacles, only the boundary (outer contour information) is extracted according to the visual sensor for detection.

[0169] Case 2:

[0170] When it is not determined based on the image collected by the first sensor whether a certain object on the ground (such as the gap between two floors or tiles, a thick black line drawn on the ground, etc.) is an obstacle and whether it needs to be edge-followed, for example, the confidence output by the machine learning model for the object is low (such as below the first threshold mentioned above, or between the first threshold and the second threshold). That is, when the confidence does not meet the requirements and the value to be verified, the detection information (such as distance information) collected by the second sensor needs to be combined to determine. If it is determined based on the detection information collected by the second sensor that the object is not an obstacle and does not need to be edge-followed, the self-moving robot directly considers that the object does not exist and proceeds according to the original planned path, such as directly crossing the object. This object that appears to be an object in the visual sensor may be a recessed groove on the ground, a low threshold (step), etc. If it is determined based on the distance information collected by the second sensor that the object is an obstacle and needs to be edge-followed, the outer contour information of the object as an obstacle is obtained based on the image collected by the first sensor, and an edge-following path is planned according to the outer contour information, and the self-moving robot walks along the edge-following path.

[0171] Further, the method provided by the embodiment further includes the following steps:

[0172] 208、When the confidence does not meet the requirements and does not belong to the value to be verified, walk according to the original planned path.

[0173] That is, when the confidence is lower than the first threshold, the obstacle is considered to be non-existent, and the self-moving robot walks according to the original plan.

[0174] In addition, the present application also provides a self-moving robot working method. In fact, the self-moving robot can identify the object and determine the type of the object based on the image collected by the first sensor and the distance information collected by the second sensor. For example, identify the object such as a low threshold, which can be crossed and does not need to be edge-followed; for example, the U-shaped chair leg of the chair shown in FIG. 1, which should be edge-followed instead of crossed. For this purpose, the present application provides the following embodiments shown in FIGS. 8-A to 8-C, specifically:

[0175] As shown in FIG. 8-A, the self-moving robot working method provided by the present application comprises the following steps:

[0176] A301, when the image collected by the first sensor and the detection information of the second sensor both indicate that there is a target object, the type of the target object is determined by combining the image and the detection information;

[0177] A302, when the type of the target object is a type that needs to follow the edge (such as a wall, a wire on the ground, etc.), an edge-following path along the outer contour of the target object is determined based on at least the image collected by the first sensor, and the robot walks along the edge-following path.

[0178] Further, the method provided by the present embodiment can further comprise the following steps:

[0179] A303, when the type of the target object is a non-edge-following type (such as a line drawn on the ground), the robot walks according to the original planned path.

[0180] In the present application, it is considered that there may be errors in simply relying on the first sensor to identify whether it is an obstacle that needs to follow the edge. For example, a line drawn on the ground (such as a black line), a solid line on the ground (such as a thin line (such as a needle line)), a gap between two floors or tiles, etc. These lines may be misjudged as obstacles that need to follow the edge, while in fact these lines on the ground are non-obstacles that need to be directly executed by the robot. Based on this, in order to improve the accuracy of identifying whether it is an obstacle that needs to follow the edge, the self-moving robot also combines the detection information of the second sensor, such as the distance information collected by the laser radar, infrared sensor, etc. When the information collected by the two sensors both indicate that there is an object, the object is comprehensively identified as an object that needs to follow the edge or an object that does not need to follow the edge. That is, the second sensor is arranged on the self-moving robot and can detect the distance, position, height, etc. of the obstacle information of the obstacle. By combining the image collected by the first sensor and the detection information of the second sensor, the type of the object, such as the edge-following type or the non-edge-following type, can be accurately determined. Then, according to the determined type, the corresponding walking strategy is executed.

[0181] Therefore, in the present application, the robot identifies the type of the object on the ground by collecting the image by the first sensor and combining the detection information of the second sensor, so as to execute different walking strategies (such as executing edge-following or executing crossing) according to the type of the object. Specifically:

[0182] For example, based on the image collected by the first sensor, it is determined that a certain object on the ground (such as a gap between two floors or tiles, a thin solid line (such as a piece of needle thread) on the ground, a black line drawn on the ground, etc.) is a need-to-follow edge type obstacle; but based on the detection information (such as distance information) of the second sensor, it is determined that the object is a non-obstacle, a non-need-to-follow edge type. At this time, the self-moving robot determines that the object is a non-obstacle, a non-need-to-follow edge type, and the corresponding walking strategy is to proceed according to the original planned path or to walk across, etc.

[0183] For example, for a slope or a balcony track threshold, based on the image collected by the first sensor, it is determined that an object on the ground is a need-to-follow edge type obstacle, but based on the detection information of the second sensor, it is determined that the object is a need-to-cross obstacle. At this time, the self-moving robot determines that the object is a non-obstacle, a non-need-to-follow edge type, and the corresponding walking strategy is to proceed according to the original planned path or to walk across. For a slope or a balcony track threshold (which can be referred to as a second type of obstacle), it is necessary to further fuse the detection information of the second sensor based on the image collected by the first sensor to comprehensively confirm.

[0184] For example, for a fallen wave line curtain, a fallen bed sheet or a wavy wall surface (which can be referred to as a third type of obstacle), based on the image collected by the first sensor and the position information collected by the second sensor, it is determined that they are need-to-follow edge types, and finally it is determined that these boundary uneven obstacles are need-to-follow edge types, so that the self-moving robot obtains the obstacle contour based on the image collected by the first sensor, and then plans a follow edge path according to the contour information, and walks along the edge according to the follow edge path.

[0185] That is, in some implementable embodiments, the step A301 of “determining the type of the target object in combination with the image and the detection information (such as distance information)” can include:

[0186] When it is determined that the target object is an obstacle based on the image and that the target object is an obstacle based on the detection information, it is determined that the type of the target object is a need-to-follow edge type;

[0187] When it is determined that the target object is an obstacle based on the image, but that the target object is a non-obstacle based on the detection information, it is determined that the type of the target object is a non-follow edge type;

[0188] When it is determined that the target object is a non-obstacle based on the image, and that the target object is a non-obstacle based on the detection information, it is determined that the type of the target object is a non-follow edge type.

[0189] Further, the method provided by the embodiment can further include the following steps:

[0190] 304、according to the image collected by the first sensor, it is determined that there is an object, but according to the detection information, it is determined that there is no object, and the robot walks according to the original planned path.

[0191] This situation is applicable to the following cases: there are wool lines on the ground, lines drawn on the ground, or gaps between tiles, etc., which can be collected by the first sensor and then identified as objects based on the image. However, the second sensor cannot collect detection information of the object. Based on the detection information collected by the second sensor, there is no object. In this case, the information collected by the second sensor should be used as the reference, and the walking strategy is to walk according to the original planned path.

[0192] For specific implementation of each step (such as step 302) in this embodiment, please refer to the related content in other embodiments.

[0193] In addition, as shown in FIG. 8-B, the present application also provides another self-moving robot working method, which comprises:

[0194] B301, determining the type of the target object according to the image collected by the first sensor and / or the detection information of the second sensor;

[0195] B302, if the type of the target object is the edge-following type, determining an edge-following path along the outer contour of the target object based on at least the image collected by the first sensor, and walking according to the edge-following path.

[0196] For example, a target object is identified as the edge-following type by the first sensor, such as a wire on the ground, a low door sill, a sofa, a wall, etc. At this time, visual edge-following walking can be performed based on only the first sensor, that is, an edge-following path along the outer contour of the target object can be determined based on only the image collected by the first sensor, and then walking according to the edge-following path.

[0197] Alternatively, for example, a target object such as a wall or a sofa is detected by the second sensor. In this case, it can be determined that the target object is of the edge-following type, and the object that can be detected by the second sensor can also be collected by the first sensor. Therefore, when the target object is identified as the edge-following type by the first sensor and / or the target object is detected as the edge-following type by the second sensor, visual edge-following walking can be performed based on the first sensor and / or planning path edge-following walking can be performed based on the second sensor, that is, an edge-following path is determined based on the detection information of the second sensor (such as the outer contour information and position information of the detected target object contained therein), and then walking according to the edge-following path.

[0198] The specific implementation of each step in this embodiment can also be seen from the related content in other embodiments.

[0199] Further, as shown in FIG. 8-C, the present application further provides another self-moving robot working method, which comprises the following steps:

[0200] C301, when the target object is determined according to the image collected by the first sensor and the detection information of the second sensor, the type of the target object is determined by combining the image and the detection information;

[0201] C302, when the type of the target object is the type of needing to follow the edge, the edge-following path is determined based on the image collected by the first sensor and / or the detection information of the second sensor, and the self-moving robot walks along the edge-following path.

[0202] The specific implementation of each step in this embodiment can also be seen from the related content in other embodiments.

[0203] In the present application, the visual range of the first sensor (such as a visual sensor) on the self-moving robot often contains the ground and a certain height space above the ground, and accordingly, the collected image can contain objects laid on the ground. Based on this, the image collected by the first sensor can be called a ground image, which refers to the ground environment information and the environment information of a certain height space above the ground. In addition, the second sensor arranged on the self-moving robot is used to obtain detection information of detecting surrounding obstacles or detection information of detecting obstacles at a predetermined height. Therefore, the present application further provides two embodiments of self-moving robot working methods shown in FIGS. 15 and 16.

[0204] As shown in FIG. 15, the present application further provides a self-moving robot working method, which comprises the following steps:

[0205] 401, whether the obstacle is a target object needing to follow the edge is determined by combining the ground image collected by the first sensor and the detection information of the second sensor.

[0206] 402, when yes, the edge-following path following the outer contour of the target object is determined based on the ground image collected by the first sensor, and the self-moving robot walks along the edge-following path.

[0207] In this embodiment, the target object is mainly an object in contact with the ground, such as a sofa, a carton, etc. Based on the image collected by the first sensor, the contour line of the target object in contact with the ground (which can be called the interface line between the target object and the ground) can be extracted, so as to plan the edge-following path according to the extracted contour line and walk along the edge-following path.

[0208] The detailed implementation of each step of the embodiment can be found in the related content in other embodiments. In addition, it should be noted that: in the embodiment, the second sensor is used to obtain detection information of detecting the surrounding obstacles.

[0209] In addition, as shown in FIG. 16, the self-moving robot working method provided by the application also includes the following steps:

[0210] 501, in combination with the ground image collected by the first sensor and the detection information of the second sensor, determine that the obstacle in the ground image is a target object that needs to be followed along the edge, and the height of the obstacle does not meet the predetermined height;

[0211] 502, based on the ground image collected by the first sensor, determine an edge-following path along the outer contour of the target object, and walk along the edge-following path.

[0212] In the embodiment, the second sensor is used to obtain detection information of detecting obstacles of a predetermined height. Therefore, whether the obstacle in the ground image meets the predetermined height can be determined according to whether the second sensor detects the obstacle. Specifically, if the second sensor does not detect the obstacle, it can be determined that the obstacle in the ground image does not meet the predetermined height, for example, a wire on the ground; on the contrary, if the second sensor detects the obstacle, it can be determined that the obstacle in the ground image meets the predetermined height, for example, a wall.

[0213] The detailed implementation of each step of the embodiment can be found in the related content in other embodiments.

[0214] In addition, during the operation of the self-moving device, for the detected object, it can also be determined whether the object is an object type that needs to be followed along the edge according to the position of the object. For example, when the self-moving robot is working from one room to another room, it often needs to perform a crossing door action to enter another room. At this time, if an object such as a wire is located at the door position, because the current main task is to cross, the object such as the wire on the ground located at the door position will be determined as an obstacle that does not need to be followed along the edge, and the crossing action can be directly performed. For this scenario, the application also provides another mobile robot working method, wherein the self-moving robot is provided with a first sensor and a second sensor, the first sensor is used to collect images, and the second sensor is used to obtain detection information of detecting surrounding obstacles. As shown in FIG. 17, the self-moving robot working method includes the following steps:

[0215] 601, when the image collected by the first sensor determines that there is a target object, in combination with the image and the detection information of the second sensor, determine the position of the target object;

[0216] 602、the target object belongs to a position requiring edge-following type, an edge-following path is determined based on the image collected by the first sensor, and the self-moving robot walks along the edge-following path;

[0217] 603、the target object belongs to a non-edge-following type position, and the self-moving robot walks along the original planned path.

[0218] For example, the first sensor identifies a target object, such as a wire, and the self-moving robot determines the position of the target object on the door through real-time positioning and mapping by the second sensor, such as a laser radar, and determines that the target object belongs to a non-edge-following type position and performs crossing walking; if it is determined that the target object is located at a vacant position in the room, such as a wire, the target object belongs to an edge-following type position, and the self-moving robot performs edge-following path walking.

[0219] Alternatively, the self-moving robot working method comprises the following steps:

[0220] 701、when the target object is determined based on the image collected by the first sensor and the detection information of the second sensor, the position of the target object is determined based on the image and the detection information;

[0221] 702、the target object belongs to a position requiring edge-following type, an edge-following path is determined based on the image collected by the first sensor, and the self-moving robot walks along the edge-following path;

[0222] 703、the target object belongs to a non-edge-following type position, and the self-moving robot walks along the original planned path.

[0223] For example, the target object is a crossable threshold at the door position, the target object belongs to a non-edge-following type position, and the self-moving robot can cross the threshold.

[0224] In this embodiment, when defining whether the object is of the along-side type or the non-along-side type, the information relied on includes but is not limited to the following information: position information of the object, attribute information of the object (including the type to which the object belongs, size information (including length, width, and height), slope information, etc.), the main task currently performed by the robot (such as a crossing task to enter another room for work), and the like. Thus, when it is determined according to at least the position information of the target object (which can also include size information of the target object, etc.) that the target object meets the along-side type, such as when the target object is located at a position in a room rather than at a door position and has a height greater than a set height, the robot can execute the along-side path determined based on the image collected by the first sensor and walk along the along-side path. When it is determined according to at least the position information of the target object (which can also include size information of the target object, etc.) that the target object meets the non-along-side type, such as when the target object is located at a door in a room and has a height less than or equal to a set height, the robot can execute the original planned path and walk along the original planned path.

[0225] The specific implementation details of each step of this embodiment can also be seen in the related content of other embodiments.

[0226] Finally, the technical solutions provided by the embodiments of the present application are described in combination with specific application scenarios.

[0227] Scenario one:

[0228] Referring to FIG. 9, a user has a self-moving robot 10 at home, which is a cleaning robot, specifically a sweeping robot, and the sweeping robot is installed with a visual sensor, an LDS, and a structured light sensor. The user starts the self-moving robot 10 to clean the residence 2 including a living room, a study, and the like. When the self-moving robot 10 walks in the living room, it identifies through the visual sensor on it that a tea table 21 in the living room is an obstacle that needs to be walked along the side, and at this time, it will execute the along-side walking action based on the boundary of the tea table 21 extracted through the visual sensor. Alternatively, in the case where the tea table 21 is identified through the visual sensor as an obstacle that needs to be walked along the side, to further ensure the accuracy of the identification result, the LDS on it can also confirm that the tea table 21 is an obstacle that needs to be walked along the side, and then the boundary of the tea table 21 is extracted through the visual sensor, and the along-side walking is executed for the tea table 21 according to the extracted boundary.

[0229] After the robot 10 finishes cleaning the living room, it enters the study to clean. During the robot's travel between points a and b along the wall (referred to as wall ab) in the study, the vision sensor on the robot identifies that the confidence of the gap 22 between the two floors is between 0.3 and 0.8. It is therefore not certain whether the gap 22 is an obstacle that needs to be traveled along the edge. The structure light sensor on the robot is further used to assist in confirming that the gap 22 is not an obstacle. As a result, the vision sensor does not perform edge extraction on the gap 22 and directly considers it as non-existent. Accordingly, the robot performs crossing travel on the gap 22 and continues to travel along the wall ab in the original travel direction.

[0230] Further, when the robot is about to reach the end of the wall ab (i.e., when the robot is about to reach a wall that is perpendicular to the wall ab, such as when the distance between the robot 10 and the wall that is perpendicular to the wall ab is less than a set threshold), a rotation action is performed to adjust the travel direction, so that the wall that is perpendicular to the wall ab is identified and the edge-following travel around the obstacle is continued. When the rotation action is performed, the preset rotation angle is not less than 90 degrees, so as to better perform the edge-following travel around the obstacle. If the preset rotation angle is too small, such as for a triangular wall, the self-moving robot cannot detect the other edge when it turns, and the travel around the obstacle is not good. It should be noted that, for walls and the like, the robot usually determines that the walls are obstacles that need to be traveled along the edge by using the vision sensor and the LDS or structure light sensor.

[0231] In summary of the above examples, the robot comprehensively identifies and confirms whether an object on the ground is an obstacle that needs to be traveled along the edge by using the vision sensor and the LDS or structure light sensor. If it is confirmed, the edge-following travel is performed based on the obstacle contour information extracted by the vision sensor. The obstacle identification cost of the present application is low, and the identification accuracy is relatively high. The present application can effectively avoid the obstacle misclassification caused by the vision sensor alone, such as misclassifying the gap between two floors or tiles, the electric wire on the ground, and the black line drawn on the ground as an obstacle that needs to be traveled along the edge.

[0232] Scenario two:

[0233] The second scenario is a specific scenario for the second type of obstacle mentioned above, such as a ramp or a balcony rail threshold. Specifically, in conjunction with FIG. 9, FIG. 10 assumes that there is a step (not shown in the figure) at the door of the study. When the self-moving robot is walking from the living room to the study, it will determine the position of the study door according to the corresponding partition algorithm. Further, when the boundary of the obstacle is extracted by the vision sensor, it will be determined whether the extracted boundary line falls on the position of the study door. If the extracted boundary line falls on the position of the study door, it will be determined that the boundary line is invalid, so that the robot will directly walk through the door without performing the edge-following action.

[0234] Scenario three:

[0235] There is a chair with a U-shaped chair leg in front of the self-moving robot as shown in FIG. 1. The first sensor on the self-moving robot is a vision sensor, and the second sensor is a structured light sensor. Based on the image collected by the vision sensor as shown in FIG. 1, the neural network model pre-installed on the self-moving robot is used to identify that there is an obstacle (i.e. the U-shaped chair leg of the chair) in the image and output the corresponding confidence. The confidence is greater than the second threshold. At this time, based on the image collected by the vision sensor, the outer contour information of the obstacle, i.e. the outer contour information of the U-shaped chair leg, is extracted; an edge-following path is planned according to the outer contour information, and the edge-following path is followed as shown in FIG. 11.

[0236] It should be noted here that FIG. 11 is a schematic image of the chair. In fact, the height of the self-moving robot is higher than the low, and the shooting angle of the vision sensor on the self-moving robot should be top-down. The perspective shown in FIG. 11 is not very accurate, mainly for illustrative purposes to show the edge-following path.

[0237] Scenario four:

[0238] The self-moving robot walks along the wave-shaped curtain rod of the floor-to-ceiling curtain. The first sensor on the self-moving robot is a vision sensor, and the second sensor is an LDS and / or a structured light sensor. Based on the image collected by the vision sensor, the neural network model pre-installed on the self-moving robot is used to identify that there is an object in the image. Based on the distance information collected by the structured light sensor, it is determined that there is an object. According to the image and the distance information, it is determined that the object belongs to the edge-following type, then the contour information of the wave-shaped curtain rod is extracted based on the image collected by the vision sensor, and then an edge-following trajectory as shown in FIG. 12 is planned based on the contour information, and the edge-following trajectory is followed very smoothly. It should be noted that although the LDS can also detect the wave-shaped curtain rod, if the path is planned based on the LDS, the path of the walk will be jagged and not smooth, and the scanning will be missed.

[0239] Scenario five:

[0240] In combination with FIG. 9, it is assumed that there is a wire such as a charging wire (or a low-height step) at the door between the living room and the study, which is not shown in the figure. When the self-moving robot completes the task in the living room, it walks from the living room to the study according to the pre-planned path according to the built-in map of the residence 2 to enter the study for work. When it is determined according to the map that the door to the study needs to be crossed, it is detected that there is a wire at the door position. At this time, it can be determined that the wire is a non-edge type according to the position, size information, etc. of the wire, and in addition, it can be determined that the wire at the door is a non-edge type obstacle according to the current main task (crossing the door), so as to continue walking according to the original planned path, so as to cross the wire to enter the study.

[0241] It should be noted here that when a non-edge type obstacle is detected at the door, such as a wire, a low-height inclined step, etc., the wire can no longer be displayed in the built-in map. If the low-height object such as the wire is at a certain position in the room, as shown in FIG. 9 in the living room, it can be determined that the wire is a need-to-follow edge type according to the position information of the wire (or the current main task of the self-moving robot, such as cleaning the living room). The self-moving robot will perform edge walking for the wire during the work in the living room, and in this case, the wire can also be displayed in the built-in map at the corresponding position.

[0242] In related technologies, robots mainly perceive the surrounding environment through sensors carried by themselves to determine the same behavior strategy. However, the actual work environment of the robot (such as a home environment) often has various objects, which can easily cause the robot to be trapped and unable to continue working, requiring user intervention. At present, there are many edge-following, obstacle-avoiding, and anti-collision sensors on the market, including monocular cameras, binocular cameras, single-line structured light, double-line structured light, bump plates, infrared light couplings, and the like. Based on the above, the robot collision, scratching, and trapping situation has been alleviated, but because of the limitations of various sensors themselves, there are still many scenarios that cannot be solved, especially low-height obstacles such as various linear objects and various U-shaped chairs. In view of the existing obstacle avoidance problem, the present application uses AI (Artificial Intelligence) technology and combines various strategies to achieve the ability to move along low-height obstacles; moreover, it ensures that the robot does not get stuck and does not wrap, and also ensures that no work area is missed, which can significantly reduce the trapping rate and improve work efficiency.

[0243] The application further provides a self-moving robot which can implement the above-mentioned method embodiments. In addition to the above-mentioned self-moving robot related content, FIG. 14 shows a schematic diagram of the principle of the self-moving robot. As shown in FIG. 14, the robot 10 includes a traveling assembly, a first sensor, a second sensor, a processor, etc. Among them, the traveling assembly is used to provide the self-moving robot with traveling power, the first sensor is used to collect images of the surrounding environment, and the second sensor is used to collect distance information. The processor is used to perform the steps in the methods provided by other embodiments of the application based on the information collected by the first sensor and the second sensor.

[0244] Further, in addition to the above-mentioned components, the robot can also include other components, such as a bumper, etc. The memory is used to store one or more computer instructions or programs. Specifically, the memory can be configured to store various other data to support operations on the self-moving device. Examples of these data include instructions for any application or method operating on the self-moving device. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. And the above-mentioned processor is coupled with the memory, and is specifically used to implement the steps in the methods provided by other embodiments of the application by executing one or more computer instructions or programs stored in the memory using the above-mentioned detection sensor and visual sensor.

[0245] Correspondingly, the embodiments of the application also provide a computer readable storage medium storing a computer program, which can implement the steps or functions of the methods provided by other embodiments of the application when executed by a computer.

[0246] In addition, the embodiments of the application also provide a computer program product, which includes a computer program that can implement the steps or functions of the methods provided by other embodiments of the application when executed by a processor.

[0247] Those skilled in the art can clearly understand the implementation of the various embodiments by means of software and the necessary general hardware platform from the above description of the embodiments, and of course, the embodiments can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that contributes to the technical solutions can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the methods.

[0248] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A self-moving robot working method, characterized by, The self-moving robot is provided with a first sensor and a second sensor, the first sensor is used for collecting images, and the second sensor detects the surrounding environment by emitting signals or detects the surrounding environment by collision; and The method comprises: In combination with the image collected by the first sensor and the detection information of the second sensor, it is determined whether there is a target object that needs to be followed along an edge; If there is, at least according to the image collected by the first sensor, an edge-following path along the outer contour of the target object is determined, and the edge-following path is walked along.

2. The method of claim 1, wherein, In combination with the image collected by the first sensor and the detection information of the second sensor, it is determined whether there is a target object that needs to be followed along an edge, comprising: According to the image collected by the first sensor, it is determined that there is an object that needs to be followed along an edge; Obtain the type of the object that needs to be followed along an edge in the image; When it is determined according to the detection information of the second sensor that there is no object that needs to be followed along an edge: If the type of the object that needs to be followed along an edge in the image is a first type, it is determined that the object that needs to be followed along an edge in the image is the target object; If the type of the object that needs to be followed along an edge in the image is a second type, it is determined that there is no target object that needs to be followed along an edge.

3. The method of claim 2, wherein, Obtaining the type of the object that needs to be followed along an edge in the image comprises: Identify the semantic information of the object that needs to be followed along an edge in the image; Based on the semantic information, determine the type of the object that needs to be followed along an edge in the image.

4. The method of claim 2, wherein, Further comprising: When it is determined according to the detection information of the second sensor that there is an object that needs to be followed along an edge, determine the target object according to the object that needs to be followed along an edge based on the image and the object that needs to be followed along an edge based on the detection information.

5. The method of claim 4, wherein, Further comprising: When it is determined based on the detection information of the second sensor that there is an object that needs to be followed along an edge, and based on the image collected by the first sensor that there is an object that needs to be followed along an edge, the edge-following path is planned according to the detection information of the second sensor; Or When it is determined based on the detection information of the second sensor that there is an object that needs to be followed along an edge, and based on the image collected by the first sensor that there is an object that needs to be followed along an edge, the outer contour information of the target object is obtained according to the image collected by the first sensor, and the edge-following path is planned according to the outer contour information; Or When it is determined based on the detection information of the second sensor that there is no object that needs to be followed along an edge, and based on the image collected by the first sensor that there is an object that needs to be followed along an edge of a first type, the outer contour information of the target object is obtained according to the image collected by the first sensor, and the edge-following path is planned according to the outer contour information.

6. A self-moving robot working method characterized by, The self-moving robot is provided with a first sensor and a second sensor, the first sensor is used for collecting images, and the second sensor detects the surrounding environment by emitting signals or detects the surrounding environment by collision; and The method comprises: When it is determined according to the image collected by the first sensor and the detection information of the second sensor that there is a target object, the type of the target object is determined in combination with the image and the detection information; If the type of the target object is a type that needs to be followed along an edge, at least based on the image collected by the first sensor, an edge-following path along the outer contour of the target object is determined, and the edge-following path is walked along.

7. A self-moving robot working method characterized by, The self-moving robot is provided with a first sensor and a second sensor, the first sensor is used for collecting images, and the second sensor detects the surrounding environment by emitting signals or detects the surrounding environment by collision; and The method comprises: According to the image collected by the first sensor and / or the detection information of the second sensor, the type of the target object is determined; If the type of the target object is a type that needs to be followed along the edge, an edge-following path along the outer contour of the target object is determined based on at least the image collected by the first sensor, and the robot walks along the edge-following path.

8. A self-moving robot working method, characterized by, The self-moving robot is provided with a first sensor and a second sensor, the first sensor is used for collecting images, and the second sensor detects the surrounding environment by emitting signals or detects the surrounding environment by collision; and The method comprises: When both the image collected by the first sensor and the detection information of the second sensor determine that there is a target object, the type of the target object is determined in combination with the image and the detection information; If the type of the target object is a type that needs to be followed along the edge, an edge-following path is determined based on the image collected by the first sensor and / or the detection information of the second sensor, and the robot walks along the edge-following path.

9. A self-moving robot working method characterized by, The self-moving robot is provided with a first sensor for collecting images; And The method comprises: An obstacle is identified based on the image collected by the first sensor, and the obstacle is a first type of object; According to the image, the outer contour information of the obstacle is extracted; According to the outer contour information, the robot walks along the edge.

10. The method of claim 9, wherein, Further comprising: The confidence of the identified obstacle is determined; If the confidence meets the requirements, the step of extracting the outer contour information of the obstacle according to the image is triggered.

11. The method of claim 10, wherein, The self-moving robot is also provided with a second sensor, which detects the surrounding environment by emitting signals or detects the surrounding environment by collision; and The method further comprises: When the confidence does not meet the requirements but belongs to a value to be verified, it is determined whether the obstacle identified based on the image collected by the first sensor needs to be followed along the edge in combination with the detection information of the second sensor; When it needs to be followed along the edge, the outer contour information of the obstacle is extracted according to the image collected by the first sensor and / or the detection information of the second sensor, and the robot walks along the edge according to the outer contour information; When it does not need to be followed along the edge, the robot walks according to the original planned path.

12. A self-moving robot working method, characterized by, The self-moving robot is provided with a first sensor and a second sensor, the first sensor is used for collecting ground images, and the second sensor obtains detection information of surrounding obstacles; And The method comprises: In combination with the ground image collected by the first sensor and the detection information of the second sensor, it is determined whether the obstacle is a target object that needs to be followed along the edge; If so, an edge-following path along the outer contour of the target object is determined based on the ground image collected by the first sensor, and the robot walks along the edge-following path.

13. A self-moving robot working method, characterized by, The self-moving robot is provided with a first sensor and a second sensor, the first sensor is used for collecting ground images, and the second sensor obtains detection information of obstacles at a predetermined height; And The method comprises: Determine, in combination with the ground image collected by the first sensor and the detection information of the second sensor, that the obstacle in the ground image is a target object that needs to be followed along an edge, and that the height of the obstacle does not conform to the predetermined height; Determine, based on the ground image collected by the first sensor, an edge-following path along the outer contour of the target object, and walk along the edge-following path.

14. A self-moving robot working method characterized by, The self-moving robot is provided with a first sensor and a second sensor, the first sensor is used to collect images, and the second sensor is used to obtain detection information of surrounding obstacles; And The method comprises: When it is determined that there is a target object according to both the image collected by the first sensor and the detection information of the second sensor, determine the position of the target object in combination with the image and the detection information; If the position of the target object is of the edge-following type, determine an edge-following path based on the image collected by the first sensor, and walk along the edge-following path; If the position of the target object is of the non-edge-following type, walk along the original planned path.

15. A self-moving robot, characterized in that, Comprise: A machine body provided with a walking assembly; A first sensor arranged on the machine body and used to collect images; A second sensor arranged on the machine body and used to collect distance information; A processor electrically connected with the first sensor, the second sensor and the walking assembly, used to execute the steps in the working method of the self-moving robot in any one of claims 1-5, or execute the steps in the working method of the self-moving robot in claim 6 or claim 7 or claim 8, or execute the steps in the working method of the self-moving robot in any one of claims 9-11, or execute the steps in the working method of the self-moving robot in claim 12, or execute the steps in the working method of the self-moving robot in claim 13, or execute the steps in the working method of the self-moving robot in claim 14.

Images