Method and device for guiding mobile robot to return to station, base station and household appliance

By installing image acquisition equipment at the base station location, acquiring environmental images and planning the return path, the problem that mobile robots cannot avoid obstacles and return to the base station in complex environments is solved, and efficient and accurate return path guidance is achieved.

CN118936465APending Publication Date: 2024-11-12MIDEA ROBOZONE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310523927.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-10
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Mobile robots are difficult to efficiently avoid obstacles and return to the base station in complex base station working environments. Existing infrared or laser sensors cannot detect all obstacles, affecting the accuracy of returning to the base station.

Method used

By installing image acquisition equipment at locations higher than the base station, we can obtain higher and wider viewing angle environmental images, identify obstacles and plan back to the station path, and use internal and external parameter data of the image acquisition equipment to determine the location of obstacles and mobile robots, and realize path planning and guide.

Benefits of technology

Effectively avoid obstacle collisions, improve the accuracy and efficiency of mobile robots in complex environments, and replace traditional infrared and other methods of return reference line search steps.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118936465A_ABST
    Figure CN118936465A_ABST
Patent Text Reader

Abstract

The invention discloses a method and device for guiding a mobile robot to return to a station, a base station and a household appliance. Firstly, an image acquisition device at a position higher than the height of a base station shoots a working environment of the base station to obtain an environment image, and then the mobile robot is identified and / or guided to return to the base station at least partially based on the environment image. The image acquisition device is installed at the position higher than the base station, the environment image in front of the base station is acquired at a higher and wider visual angle, the mobile robot can be recognized and / or guided to return to the station at least partially based on the environment image, the step of searching of the mobile robot is omitted, and the mobile robot is efficiently guided to return to the base station.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of robot technology, and in particular to a method, device, base station and household appliance for guiding a mobile robot to return to a station. Background Art

[0002] There may be some shoes, toys and other items in front of the base station that are obstacles for the mobile robot. It can be seen that the environment in front of the mobile robot is relatively complex, so how to efficiently guide the mobile robot back to the base station is a problem that needs to be solved. Summary of the invention

[0003] The embodiments of this specification are intended to solve at least one of the technical problems in the related art to a certain extent. To this end, the embodiments of this specification propose a method, device, base station and home appliance for guiding a mobile robot to return to a station.

[0004] The present specification provides a method for guiding a mobile robot to return to a station, the method comprising:

[0005] Acquire an environmental image; wherein the environmental image is obtained by photographing the base station working environment through an image acquisition device; the height of the image acquisition device is greater than the height of the top of the base station;

[0006] Based at least in part on the image of the environment, a mobile robot is identified and / or directed to return to the base station.

[0007] In one embodiment, guiding the mobile robot back to the base station includes:

[0008] Determining first position data of an obstacle in the working environment of the base station based on the environment image;

[0009] Performing path planning based on the first position data and the second position data of the mobile robot to obtain a first return path for the mobile robot to return to the base station;

[0010] The mobile robot is guided to return to the base station according to the first return path.

[0011] In one embodiment, guiding the mobile robot back to the base station includes:

[0012] Based on the environment image, determining second position data of the mobile robot;

[0013] Performing path planning based at least in part on the second position data to obtain a first return path for the mobile robot to return to the base station;

[0014] The mobile robot is guided to return to the base station according to the first return path.

[0015] In one embodiment, the method for determining the first position data includes:

[0016] Receiving the environment image sent by the image acquisition device;

[0017] Identify the environment image to obtain the image coordinate system coordinates of the obstacle in the environment image;

[0018] Coordinate conversion is performed according to the image coordinate system coordinates of the obstacle and the internal and external parameter data of the image acquisition device to obtain the first position data of the obstacle in the operation map of the mobile robot.

[0019] In one embodiment, directing the mobile robot back to the base station based at least in part on the image of the environment includes:

[0020] Determine the second position data of the mobile robot in the operation map based on the environmental image and the internal and external parameter data of the image acquisition device; determine the return reference point of the base station according to the external parameter data of the image acquisition device and the relative position data between the image acquisition device and the return reference point; guide the mobile robot back to the base station through the second position data and the return reference point; or

[0021] Based on the environmental image and the internal and external parameter data of the image acquisition device, the second position data of the mobile robot in the operation map is determined; according to the second position data and the position data of the base station, path planning is performed to guide the mobile robot back to the base station.

[0022] In one embodiment, the method comprises:

[0023] If the position of the obstacle changes, obtaining third position data of the obstacle after the change;

[0024] The path is re-planned based on the third position data and the real-time position data of the mobile robot to obtain a second return path for the mobile robot to return to the base station.

[0025] In one embodiment, if the position of the obstacle changes, obtaining third position data of the obstacle after the change includes:

[0026] Receiving a real-time image of the environment after the position of the obstacle changes, which is sent in real time by the image acquisition device;

[0027] The third position data is obtained by performing calculation based on the real-time image of the environment and the internal and external parameter data of the image acquisition device.

[0028] In one embodiment, the return-to-station reference line of the base station is fixed relative to the image acquisition device; the position of the return-to-station reference line is determined by external parameters of the image acquisition device and relative position data between the return-to-station reference line and the image acquisition device.

[0029] In one embodiment, performing path planning based at least in part on the second position data to obtain a first return path for the mobile robot to return to the base station includes:

[0030] Path planning is performed at least partially based on the second position data of the mobile robot and the position of the return-to-station reference line to obtain the first return-to-station path.

[0031] In one embodiment, before performing path planning based at least in part on the second position data of the mobile robot and the position of the return-to-station reference line to obtain the first return-to-station path, the method further includes:

[0032] Determining first position data of an obstacle in the working environment of the base station based on the environment image;

[0033] The performing path planning at least partially based on the second position data of the mobile robot and the position of the return-to-station reference line to obtain the first return-to-station path includes:

[0034] Path planning is performed based on the first position data of the obstacle, the second position data of the mobile robot, and the position of the return-to-station reference line to obtain the first return-to-station path.

[0035] In one embodiment, the base station is arranged at the bottom of the household appliance or as a part of the household appliance, and the image acquisition device is arranged on the household appliance; or

[0036] The base station is arranged at the bottom of the furniture or as a part of the furniture, and the image acquisition device is arranged on the furniture.

[0037] In one embodiment, the household appliance includes a clothes handling appliance.

[0038] An embodiment of the present specification provides a base station, comprising: a processor and a memory configured to store a computer program that can be run on the processor, wherein the processor is configured to execute the steps of the method described in any one of the above embodiments when running the computer program.

[0039] An embodiment of the present specification provides a household appliance, which includes: a processor and a memory configured to store a computer program that can be run on the processor, wherein the processor is configured to execute the steps of the method described in any one of the above embodiments when running the computer program.

[0040] In one embodiment, the household appliance includes a clothes handling appliance.

[0041] The embodiment of this specification provides a device for guiding a mobile robot to return to a station, the device comprising:

[0042] An environmental image acquisition module, used to acquire an environmental image; wherein the environmental image is obtained by photographing the base station working environment through an image acquisition device; the height of the image acquisition device is greater than the height of the top of the base station;

[0043] An identification and guidance module is used to identify the mobile robot and / or guide the mobile robot back to the base station based at least in part on the environment image.

[0044] In the above-mentioned implementation mode of the specification, the working environment of the base station is first photographed by an image acquisition device at a height greater than the base station to obtain an environmental image, and then the mobile robot is identified and / or guided to return to the base station at least partially based on the environmental image. By installing an image acquisition device at a position higher than the base station, the environmental image in front of the base station is captured at a higher and wider viewing angle, and the mobile robot can be identified and / or guided to return to the station at least partially based on the environmental image, thereby eliminating the step of searching for the mobile robot and achieving efficient guidance of the mobile robot back to the base station. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flowchart of a method for guiding a mobile robot to return to a station provided in an embodiment of this specification;

[0046] Figure 2 A schematic diagram of a process for guiding a mobile robot to return to a base station provided in an embodiment of this specification;

[0047] Figure 3 A schematic diagram of a process for guiding a mobile robot to return to a base station provided in an embodiment of this specification;

[0048] Figure 4 A schematic diagram of a process for determining first position data provided in an embodiment of this specification;

[0049] Figure 5 A flowchart of a method for guiding a mobile robot to return to a station provided in an embodiment of this specification;

[0050] Figure 6A schematic diagram of a process for determining third position data provided in an embodiment of this specification;

[0051] Figure 7 A schematic diagram of a base station return reference line provided in an embodiment of this specification;

[0052] Figure 8 A schematic diagram of a process for determining a first return station path provided in an implementation manner of this specification;

[0053] Fig. 9 A schematic diagram of a base station and an image acquisition device provided in an embodiment of this specification;

[0054] Fig.10 A flowchart of a method for guiding a mobile robot to return to a station provided in an embodiment of this specification;

[0055] Fig.11 A schematic diagram of a device for guiding a mobile robot to return to a station provided in an embodiment of this specification. DETAILED DESCRIPTION

[0056] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0057] This specification provides a method for guiding a mobile robot to return to a station. Figure 1 The method for guiding the mobile robot back to the station may include the following steps:

[0058] S110: Acquire an environment image.

[0059] S120. Identify the mobile robot and / or guide the mobile robot back to the base station based at least in part on the environment image.

[0060] The environment image is obtained by photographing the base station working environment through an image acquisition device; the height of the image acquisition device is greater than the height of the top of the base station. The base station working environment can be the layout of objects or obstacles around the base station.

[0061] In some cases, an image acquisition device is installed at a position higher than the base station, and the image acquisition device interacts with the mobile robot to guide the mobile robot back to the base station with a higher and wider perspective.

[0062] Specifically, an image acquisition device is installed at a position higher than the base station, and then the image acquisition device can be used to acquire images of the working environment of the base station to obtain an environmental image. The mobile robot can identify its own position based on at least part of the environmental image acquired, or can be guided back to the base station based on at least part of the environmental image.

[0063] In the above method for guiding the mobile robot back to the station, the working environment of the base station is photographed by an image acquisition device at a height greater than the base station to obtain an environmental image, and then the mobile robot is identified and / or guided to return to the base station at least partially based on the environmental image. By installing an image acquisition device at a position higher than the base station, the environmental image in front of the base station is captured at a higher and wider viewing angle, and the mobile robot can be identified and / or guided to return to the station at least partially based on the environmental image, thereby eliminating the step of searching for the mobile robot and achieving efficient guidance of the mobile robot back to the base station.

[0064] With the continuous development of artificial intelligence technology, a series of artificial intelligence products based on artificial intelligence technology have gradually entered people's lives, providing many conveniences for people's lives. For example, a cleaning robot can automatically identify obstacles, avoid obstacles and perform cleaning tasks on the objects to be cleaned, reducing the cleaning tasks for users. Cleaning robots are usually equipped with base stations to assist the cleaning robots in their work, such as providing charging services for cleaning robots, providing mop cleaning services for cleaning robots, and providing garbage collection services for cleaning robots. There may be some shoes, toys and other items in front of the base station that are obstacles for mobile robots. It can be seen that the working environment of the base station of the mobile robot is relatively complex, so guiding the mobile robot to effectively avoid obstacles and efficiently return to the base station is a problem that needs to be solved.

[0065] In related technologies, mobile robots usually use infrared, laser and other sensors to locate the positions of base stations and obstacles.

[0066] However, due to the limitations of infrared or laser methods, there is a possibility that all obstacles in the base station working environment cannot be detected, so that undetected obstacles will affect the mobile robot's return to the base station.

[0067] Based on this, the embodiments of this specification provide a method for guiding a mobile robot to return to the station. Specifically, an image acquisition device at a height greater than the base station is used to capture the working environment of the base station to obtain an environmental image, and then the mobile robot is identified and / or guided to return to the base station based at least in part on the environmental image. By installing an image acquisition device at a position higher than the base station, the mobile robot is guided back to the base station and the environmental image in front of the base station is captured with a higher and wider viewing angle. Based at least in part on the environmental image, the mobile robot can be identified and / or prevented from colliding with obstacles during the return process. The mobile robot can be effectively guided to the position of the return reference line in a complex base station working environment and the mobile robot can be guided back to the base station.

[0068] The method for guiding a mobile robot to return to the station provided in the embodiment of this specification can be applied to a mobile robot corresponding to a base station integrated under a washing machine, and an image acquisition device can be installed on the washing machine and the installation height of the image acquisition device must be higher than the base station (such as the top of the washing machine). Obstacles can be items that need to be avoided, such as toys, shoes, and stools. The mobile robot can be a sweeping robot, a mopping robot, or other equipment equipped with a base station and can move by itself. Exemplarily, the image acquisition device sends the environmental image obtained by shooting the working environment of the base station to a cloud server, and the cloud server can identify the received environmental image and obtain the image coordinate system coordinates of the obstacle in the environmental image. The cloud server sends the obtained image coordinate system coordinates of the obstacle to the mobile robot, and the mobile robot can plan the path of the mobile robot back to the base station according to the received image coordinate system coordinates of the obstacle and the positioning data of the mobile robot. Exemplarily, the image acquisition device sends the environmental image obtained by shooting the working environment of the base station to the mobile robot, and the mobile robot can identify the received environmental image and obtain the image coordinate system coordinates of the obstacle in the environmental image. The mobile robot can plan the path of the mobile robot back to the base station according to the received image coordinate system coordinates of the obstacle and the positioning data of the mobile robot.

[0069] The cloud server can be an electronic device with certain computing and processing capabilities. It can have a network communication module, a processor, and a memory. The cloud server can be a distributed server, which can be a system with multiple processors, memories, network communication modules, etc. operating in coordination. Alternatively, the cloud server can also be a server cluster formed by several servers. Alternatively, with the development of science and technology, the cloud server can also be a new technical means that can realize the corresponding functions of the implementation method of the specification. For example, it can be a new form of "server" based on quantum computing.

[0070] In some embodiments, see Figure 2, guiding the mobile robot back to the base station may include the following steps:

[0071] S210: Determine first position data of an obstacle in a working environment of the base station based on the environment image.

[0072] Among them, the first position data is calculated based on the environmental image and the internal and external parameter data of the image acquisition device. The environmental image is obtained by shooting the working environment of the base station by the image acquisition device. The height of the image acquisition device is greater than the height of the top of the base station. The internal parameter data of the image acquisition device can be parameters related to the characteristics of the image acquisition device itself, such as the focal length and pixel size of the camera. The external parameter data of the image acquisition device can be the position of the image acquisition device in the world coordinate system, which can be understood as converting the position information of the image acquisition device from the image acquisition device coordinate system to the world coordinate system.

[0073] In some cases, an image acquisition device is installed at a position higher than the base station. The image acquisition device can be used to guide the mobile robot with a higher and wider perspective, interact with the mobile robot, achieve efficient return to the base station, and more accurately and comprehensively determine the location of obstacles in the working environment of the base station.

[0074] Specifically, the position data of the obstacle in the working environment of the base station, namely, the first position data, can be obtained by calculating the environment image and the internal reference data and the external reference data of the image acquisition device.

[0075] In some embodiments, the environmental image can be uploaded to a cloud server, where the location data of obstacles in the base station working environment, i.e., first location data, can be obtained by calculation based on the environmental image and the internal and external parameter data of the image acquisition device.

[0076] In some other embodiments, the environmental image can be sent to a mobile robot, in which the position data of obstacles in the working environment of the base station, i.e., first position data, can be obtained by calculation based on the environmental image and the internal and external parameter data of the image acquisition device.

[0077] S220: Perform path planning based on the first position data and the second position data of the mobile robot to obtain a first return path for the mobile robot to return to the base station.

[0078] S230: guiding the mobile robot to return to the base station according to the first return path.

[0079] The first return path may be a path for the mobile robot to return to the base station and the path may avoid collisions with obstacles. The second position data of the mobile robot may be calculated based on the environment image and the internal and external reference data of the image acquisition device, or the mobile robot may obtain its own position through positioning and obtain the position data of the position, i.e., the second position data of the mobile robot.

[0080] In some cases, there are requirements for the placement of the base station. Generally, a distance of 30 to 50 centimeters is required on the left and right sides of the base station, and a space of 1 to 1.5 meters is required in front of the base station. However, in reality, not all base station placements can perfectly meet the above requirements, so it is necessary to plan a path back to the base station, which can enable the mobile robot to avoid obstacles and guide the mobile robot back to the base station during the return process. The method of installing image acquisition equipment and guiding the mobile robot back to the base station through the first return path can replace or supplement the step of searching for the return reference line in front of the base station by infrared or other means.

[0081] Specifically, the mobile robot can combine the location of the obstacle and the location of the mobile robot on the same map according to the first location data of the obstacle. The mobile robot can plan a path to return to the base station, i.e., the first return path, according to its own location and the location of the obstacle on the map, and can guide the mobile robot back to the base station according to the planned first return path.

[0082] In some embodiments, the cloud server sends the first position data of the obstacle to the mobile robot, and the mobile robot can combine the position of the obstacle and the position of the mobile robot on the same operation map according to the first position data of the obstacle. The mobile robot can plan a path to return to the base station, i.e., the first return path, according to its own position and the position of the obstacle on the operation map.

[0083] In the above method for guiding the mobile robot to return to the station, the first position data of the obstacle in the working environment of the base station can be calculated based on the environmental image and the internal and external parameter data of the image acquisition device at a position greater than the height of the base station. Then, the path planning is performed based on the first position data of the obstacle and the second position data of the mobile robot, and the first return path of the mobile robot to the base station can be obtained, and the mobile robot is guided to return to the base station according to the first return path. By installing the image acquisition device at a position higher than the base station, the mobile robot is guided and the environmental image in front of the base station is collected at a higher and wider viewing angle. The planned first return path can prevent the mobile robot from colliding with obstacles during the return process, and can effectively guide the mobile robot to the position of the return reference line in the complex working environment of the base station and guide the mobile robot back to the base station.

[0084] In some embodiments, see Figure 3 , guiding the mobile robot back to the base station may include the following steps:

[0085] S310: Determine second position data of the mobile robot based on the environment image.

[0086] The second position data may be the position of the mobile robot in the operation map, and the second position data may be calculated based on the environment image and the internal and external parameter data of the image acquisition device.

[0087] In some cases, an image acquisition device is installed at a position higher than the base station, and the position of the mobile robot can be determined from a higher and wider perspective through the image acquisition device, thereby achieving interaction with the mobile robot and efficiently guiding the mobile robot back to the station. Specifically, the position data of the mobile robot in the working environment of the base station, i.e., the second position data, can be obtained by calculating the environment image and the internal and external reference data of the image acquisition device.

[0088] In some embodiments, the environmental image can be uploaded to a cloud server, where the position data of the mobile robot in the base station working environment, i.e., the second position data, can be obtained by calculation based on the environmental image and the internal and external parameter data of the image acquisition device.

[0089] In some other embodiments, the environmental image can be sent to a mobile robot, in which the position data of the mobile robot in the base station working environment, i.e., the second position data, can be obtained by calculation based on the environmental image and the internal and external parameter data of the image acquisition device.

[0090] S320: Perform path planning at least partially based on the second position data to obtain a first return path for the mobile robot to return to the base station.

[0091] S330: guiding the mobile robot to return to the base station according to the first return path.

[0092] Specifically, when the position of the mobile robot is known, a path planning for returning to the base station can be performed based on the second position data to obtain a first return path for the mobile robot to return to the base station. In some embodiments, the mobile robot can combine the location of the obstacle and the location of the mobile robot on the same map according to the second position data. The mobile robot can plan a path to return to the base station, i.e., the first return path, based on its own location and the location of the obstacle on the map, and the mobile robot can be guided back to the base station according to the planned first return path.

[0093] In some embodiments, the cloud server sends the obtained second position data of the mobile robot to the mobile robot, and the mobile robot can combine the position of the obstacle and the position of the mobile robot on the same operation map according to the second position data of the mobile robot. The mobile robot can plan a path to return to the base station, i.e., the first return path, according to its own position and the position of the obstacle on the operation map.

[0094] In the above method for guiding the mobile robot to return to the station, the second position data of the mobile robot in the working environment of the base station can be calculated based on the environmental image and the internal and external parameter data of the image acquisition device at a position greater than the height of the base station. Then, the path planning is performed at least partially based on the second position data to obtain the first return path for the mobile robot to return to the base station, and the mobile robot is guided to return to the base station according to the first return path. By installing the image acquisition device at a position higher than the base station, the mobile robot is guided with a higher and wider viewing angle, and the planned first return path can enable the mobile robot to return to the station efficiently and conveniently. Furthermore, the probability of collision with obstacles is reduced, and the mobile robot can be effectively guided to reach the position of the return reference line in the complex working environment of the base station and to guide the mobile robot back to the base station.

[0095] In some embodiments, see Figure 4 , the method for determining the first position data may include the following steps:

[0096] S410: Receive an environment image sent by an image acquisition device.

[0097] S420: Perform obstacle detection based on the environment image to obtain the image coordinate system coordinates of the obstacle in the environment image.

[0098] The image is composed of pixels, and the image coordinate system coordinates are the positions of the pixels in the image. The image acquisition device can be any device with image acquisition function, such as a monocular camera, a binocular camera, or a camera.

[0099] Specifically, the image acquisition device uploads the acquired environment image to the designated receiving device. The designated receiving device receives the environment image sent by the image acquisition device, and then performs obstacle detection on the environment image in the designated device. By performing obstacle detection on the environment image, obstacles, mobile robots, and backgrounds (floor, wall, etc.) in the environment image can be determined. If there are obstacles, the image coordinate system coordinates of the obstacles in the environment image can be obtained by performing obstacle detection on the environment image.

[0100] In some embodiments, an obstacle detection model can be trained, and the obstacle detection model can be deployed on a cloud server. The environment image collected by the image acquisition device is uploaded to the cloud server, and the cloud server inputs the environment image into the trained obstacle detection model, and it can be determined whether the feature information of the obstacle extracted by the obstacle detection model matches the trained obstacle detection model. If the extracted feature information does not match the trained obstacle detection model, there is no obstacle in the collected environment image. If the extracted feature information matches the trained obstacle detection model, there is an obstacle in the collected environment image, and the obstacle detection model can identify the obstacle category and detect the position frame of the obstacle in the environment image and the coordinates of the position frame, and the coordinates of the position frame are the image coordinate system coordinates of the obstacle in the environment image. Furthermore, a number of pixels are distributed in the environment image, and the coordinates of the position frame detected can be regarded as the pixel coordinates of a vertex of the position frame. The image coordinate system coordinates of the obstacle can be expressed as a position frame, and a certain number of pixels are distributed in the position frame. It should be noted that the location border can be used to mark the area where the obstacle is located in the environment image, such as using a rectangular frame to frame the obstacle in the environment image. The location border can be a bounding box determined when the obstacle is detected by the obstacle detection model. Based on the location border, the image coordinate system coordinates of the obstacle corresponding to the location border can be determined. The obstacle detection model can use a deep learning network. The obstacle detection model can also use an SSD (Single Shot MultiBox Detector) model. Deploying the obstacle detection model on a cloud server can reduce the consumption of mobile robot resources.

[0101] In other embodiments, an obstacle detection model can be deployed on a mobile robot, and an image acquisition device uploads the collected environment image to the mobile robot. The mobile robot inputs the environment image into the obstacle detection model, and the obstacle detection model detects obstacles in the environment image, and can obtain the position frame of the obstacle in the environment image and the image coordinate system coordinates of the obstacle.

[0102] S430, performing coordinate conversion according to the image coordinate system coordinates of the obstacle and the internal and external parameter data of the image acquisition device to obtain first position data of the obstacle in the operation map of the mobile robot.

[0103] The operation map of the mobile robot may be marked with the position of the mobile robot, the position of obstacles, and the relative position relationship between the position of the mobile robot and the position of obstacles.

[0104] In some cases, displaying the mobile robot position and obstacle positions on the same map can facilitate the mobile robot's path planning to return to the base station.

[0105] Specifically, according to the internal and external parameter data of the image acquisition device, the image coordinate system coordinates of the obstacle can be converted into the operation map of the mobile robot, and the position data of the obstacle in the operation map of the mobile robot is the first position data.

[0106] In some embodiments, if the image acquisition device is a binocular camera, the depth information of the obstacle can be directly calculated. Therefore, the image coordinate system coordinates of the obstacle can be converted into the working map of the mobile robot more conveniently through coordinate conversion.

[0107] In other embodiments, if the image acquisition device is a monocular camera, the coordinate conversion requires a certain algorithm to complete, and the coordinate conversion can be performed through PnP (Perspective-n-Point), which is a method for 3D-2D mapping relationship. The basic formula is This formula can be used to transform the image coordinate system coordinates of the obstacle into the world map. Based on the world map, the operation map of the mobile robot can be obtained. The position of the obstacle in the operation map of the mobile robot is the first position data. Where P is the coordinate of the obstacle in the image coordinate system, P C is the coordinate of the obstacle in the camera coordinate system, p W can be the external parameter data of the image acquisition device, ω is the depth of the obstacle, K is the internal parameter data of the image acquisition device, R CW and is the pose transformation from the world coordinate system to the camera coordinate system.

[0108] In the above method for guiding the mobile robot to return to the station, an environment image sent by an image acquisition device is received. Obstacle detection is performed on the environment image to obtain the image coordinate system coordinates of the obstacle in the environment image. Coordinate conversion is performed based on the image coordinate system coordinates of the obstacle and the internal and external parameter data of the image acquisition device to obtain the first position data of the obstacle in the operation map of the mobile robot. By obtaining the first position data of the obstacle in the operation map of the mobile robot, the path back to the base station can be planned more accurately.

[0109] In some embodiments, directing the mobile robot back to the base station based at least in part on the image of the environment may include:

[0110] Determine the second position data of the mobile robot in the operation map based on the environment image and the internal and external parameter data of the image acquisition device; determine the return reference point of the base station according to the external parameter data of the image acquisition device and the relative position data between the image acquisition device and the return reference point; guide the mobile robot back to the base station through the second position data and the return reference point; or

[0111] Based on the environmental image and the internal and external parameter data of the image acquisition device, the second position data of the mobile robot in the operation map is determined; according to the second position data and the position data of the base station, the path is planned to guide the mobile robot back to the base station.

[0112] The return reference point may be a designated location that the mobile robot needs to reach when returning to the base station. After the mobile robot reaches the return reference point, it can return to the base station more accurately, thereby reducing collisions between the mobile robot and the base station.

[0113] In some embodiments, the second position data of the mobile robot in the operation map can be obtained by calculating the environmental image and the internal and external parameter data of the image acquisition device. The return reference point of the base station can be obtained by calculating the relative position data of the image acquisition device and the return reference point and the external parameter data of the image acquisition device. Path planning is performed based on the second position data and the return reference point to determine the path for the mobile robot to reach the return reference point and return to the base station, and the mobile robot can be guided back to the base station based on the determined return path.

[0114] In other embodiments, the second position data of the mobile robot in the operation map can be obtained by calculating the environment image and the internal and external reference data of the image acquisition device. Path planning can be performed based on the second position data and the position data of the base station to determine the path for the mobile robot to return to the base station, and the mobile robot can be guided back to the base station based on the determined return path.

[0115] In the above method for guiding the mobile robot to return to the base station, the path for the mobile robot to return to the base station can be flexibly determined in the above manner, and the return path that best meets the user's requirements can be selected.

[0116] In some embodiments, see Figure 5 , the method for guiding the mobile robot back to the station may include the following steps:

[0117] S510: If the position of the obstacle changes, obtain third position data of the obstacle after the change.

[0118] The third position data may be position data after the obstacle position changes, and the third position data is calculated based on the real-time image of the environment and the internal and external parameter data of the image acquisition device.

[0119] In some cases, not all obstacles in the working environment of the mobile robot are static. There are movable obstacles in the working environment of the mobile robot, such as pets at home, other mobile robots other than the mobile robot, movable toys, etc.

[0120] Specifically, the image acquisition device can be used to acquire images of the working environment of the base station in real time, and obtain a series of several frames of real-time environment images, and the time interval between two adjacent frames of real-time environment images is very short. Because the movement of the obstacle changes the position of the obstacle, it is necessary to calculate based on each frame of the real-time environment image and the internal and external reference data of the image acquisition device to obtain the position data of the obstacle in the frame of the environment image after the position changes, that is, the third position data.

[0121] In some embodiments, each frame of the real-time environment image can be uploaded to a cloud server. In the cloud server, calculations can be performed based on each frame of the real-time environment image and the internal and external parameter data of the image acquisition device to obtain the position data of the obstacle in the real-time environment image after the position changes, that is, the third position data.

[0122] In some other embodiments, each frame of the real-time environment image can be uploaded to a mobile robot. In the mobile robot, calculations can be performed based on each frame of the real-time environment image and the internal and external parameter data of the image acquisition device to obtain the position data of the obstacle in the real-time environment image after the position has changed, i.e., the third position data.

[0123] S520: re-plan the path based on the third position data and the real-time position data of the mobile robot to obtain a second return path for the mobile robot to return to the base station.

[0124] The second return path is a path that the mobile robot replans to return to the base station after the position of the obstacle changes, and the path can avoid collision with the obstacle.

[0125] Specifically, the mobile robot can combine the obstacle location and the real-time location of the mobile robot on the same operation map according to the third location data of the obstacle. The mobile robot can re-plan the path back to the base station, i.e., the second return path, according to its real-time location and the obstacle location on the operation map.

[0126] In some embodiments, the cloud server sends the obtained third position data of the obstacle to the mobile robot, and the mobile robot can combine the location of the obstacle and the real-time location of the mobile robot on the same operation map according to the third position data of the obstacle. The mobile robot can re-plan the path back to the base station, i.e., the second return path, according to its own real-time location and the location of the obstacle on the operation map.

[0127] In the above method for guiding the mobile robot to return to the station, if the position of the obstacle changes, the third position data after the obstacle changes is obtained, and the path is replanned based on the third position data and the real-time position data of the mobile robot to obtain the second return path of the mobile robot to the base station. For dynamic scenes, the working environment in front of the base station is photographed in real time by an image acquisition device. If the position of the obstacle changes, the environmental image will also be updated in real time. The real-time position of the obstacle can be determined through the real-time updated environmental image, and the path of the mobile robot returning to the base station can be replanned based on the real-time position of the obstacle.

[0128] In some embodiments, see Figure 6 If the position of the obstacle changes, obtaining the third position data after the obstacle changes may include the following steps:

[0129] S610: Receive a real-time image of the environment after the obstacle position changes, which is sent by an image acquisition device in real time.

[0130] S620: Calculate based on the real-time image of the environment and the internal and external parameter data of the image acquisition device to obtain third position data.

[0131] Specifically, because the position of the obstacle changes, the image acquisition device needs to collect the working environment of the base station in real time, obtain the real-time image of the environment and upload the collected real-time image of the environment to the designated receiving device. The designated receiving device receives the real-time image of the environment, and then identifies the real-time image of the environment in the designated device. By performing obstacle detection on the real-time image of the environment, the obstacles, mobile robots, and backgrounds (floors, walls, etc.) existing in the real-time image of the environment can be determined. If there is an obstacle, the image coordinate system coordinates of the obstacle in the real-time image of the environment can be obtained by performing obstacle detection on the real-time image of the environment. Then, according to the image coordinate system coordinates of the obstacle in the real-time image of the environment and the internal and external parameter data of the image acquisition device, coordinate conversion is performed, and the image coordinate system coordinates of the obstacle can be converted into the operation map of the mobile robot. The position data after the position of the obstacle changes in the operation map of the mobile robot is the third position data.

[0132] In the above method for guiding the mobile robot back to the base station, a real-time image of the environment after the position of the obstacle changes is received from the image acquisition device in real time; the third position data is obtained by calculation based on the real-time image of the environment and the internal and external parameter data of the image acquisition device. By obtaining the third position data, the position of the obstacle after the movement can be obtained in real time, so that the mobile robot can plan the path to return to the base station in real time.

[0133] In some implementations, the return-to-station reference line of the base station is fixed relative to the image acquisition device; the position of the return-to-station reference line is determined by external parameters of the image acquisition device and relative position data between the return-to-station reference line and the image acquisition device.

[0134] The back-to-station reference line may be the center line of the base station. For example, see Figure 7 , Figure 7 The line segment 710 in FIG. 7 is the center line of the base station and can be used as a reference line for returning to the base station.

[0135] In some cases, the return reference line of the base station can make the mobile robot return to the base station more accurately, reducing the deviation between the position where the mobile robot stays in the base station and the position where the mobile robot should stay, so that the mobile robot can be cleaned more comprehensively.

[0136] In some embodiments, performing path planning based at least in part on the second position data to obtain a first return path for the mobile robot to return to the base station includes:

[0137] Path planning is performed at least partially based on the second position data of the mobile robot and the position of the return-to-station reference line to obtain a first return-to-station path.

[0138] In some embodiments, the mobile robot can plan a path to return to the base station based on the second position data of the mobile robot and the position of the return reference line, and select an obstacle avoidance path from the multiple planned paths to return to the base station that can ensure the safety factor of obstacle avoidance and can still return to the base station after obstacle avoidance, and the turning time during obstacle avoidance is the shortest. Of course, the mobile robot can also send the multiple planned paths to return to the base station to the terminal device, and the terminal device specifies the corresponding path to return to the base station. The path to return to the base station can be regarded as the first return path. After specifying the first return path, the mobile robot is controlled to perform the operation of returning to the base station according to the path to return to the base station specified by the user on the terminal device side.

[0139] In the above method for guiding the mobile robot to return to the station, path planning is performed at least partially based on the second position data of the mobile robot and the position of the return reference line to obtain a first return path. The first return path can effectively guide the mobile robot to return to the base station in a complex pre-station environment.

[0140] In some embodiments, see Figure 8 Before performing path planning based at least in part on the second position data of the mobile robot and the position of the return reference line to obtain a first return path, the method further includes:

[0141] S810: Determine first position data of an obstacle in a working environment of the base station based on the environment image.

[0142] Performing path planning based at least in part on the second position data of the mobile robot and the position of the return-to-station reference line to obtain a first return-to-station path includes:

[0143] S820: Perform path planning based on the first position data of the obstacle, the second position data of the mobile robot, and the position of the return-to-station reference line to obtain a first return-to-station path.

[0144] In some embodiments, because the return reference line of the base station is relatively fixed to the image acquisition device, the position of the return reference line of the base station can be obtained through the external parameters of the image acquisition device and the relative position data between the return reference line of the base station and the image acquisition device. Based on the principle of the shortest line segment between two points, the mobile robot can plan the path to return to the base station through its own positioning data and the position of the return reference line. In this path, if there is an obstacle, the path can be adjusted at the position where the obstacle exists in the path to avoid the obstacle, and the adjusted path to return to the base station can be obtained. If there is an obstacle in the adjusted path, the path is also adjusted at the position where the obstacle is located until there is no obstacle on the path back to the base station. The finally determined path to return to the base station can be regarded as the first return path, and the mobile robot can return to the base station according to the first return path.

[0145] In other embodiments, the planning of the return path can be based on the safety of obstacle avoidance, obstacle avoidance time, obstacle avoidance path, time to return to the base station, etc. The mobile robot can plan the path to return to the base station based on the first position data of the obstacle, the positioning data of the mobile robot, and the position of the return reference line, and select an obstacle avoidance path from the multiple planned paths to return to the base station that can ensure the safety factor of obstacle avoidance and can still return to the base station after obstacle avoidance and has the shortest turning time during obstacle avoidance. Of course, the mobile robot can also send the planned multiple paths back to the base station to the terminal device, and the terminal device specifies the corresponding path back to the base station, which can be regarded as the first return path. After specifying the first return path, the mobile robot is controlled to perform the operation of returning to the base station according to the path back to the base station specified by the user on the terminal device side.

[0146] It should be noted that obstacles can be classified into multiple categories, such as obstacles that do not need to be avoided (such as low steps that are lower than the height that the mobile robot can cross), and obstacles that need to be avoided (such as weight scales, shoes, etc.). It can be first determined whether the obstacle is an obstacle that needs to be avoided based on the obstacle category, position and depth information. If it is an obstacle that does not need to be avoided, the task of returning to the base station continues along the current obstacle direction. If it is an obstacle that needs to be avoided, it is determined whether the obstacle is an obstacle of the avoidable type. If it is an obstacle of the avoidable type, the mobile robot is controlled to perform an automatic obstacle avoidance operation. When the obstacle is an obstacle of the avoidable type, it can be further determined whether the obstacle is an obstacle that needs to be cleaned. If it is an obstacle that needs to be cleaned, the cleaning operation is directly performed without performing the automatic obstacle avoidance operation. If it is not an obstacle that needs to be cleaned, the mobile robot is controlled to perform the automatic obstacle avoidance operation. If the obstacle is an obstacle that cannot be avoided, the path back to the base station is replanned without performing the automatic obstacle avoidance operation. When it is determined that the sweeping robot needs to avoid obstacles and can avoid them, a corresponding path back to the base station can be planned for the mobile robot according to the position and depth information corresponding to the obstacle category. A TOF (Time of Flight) camera can be used to further obtain the depth information of the obstacle.

[0147] In the above method for guiding the mobile robot to return to the station, path planning is performed based on the first position data of the obstacle, the positioning data of the mobile robot and the position of the return reference line to obtain a first return path. The first return path can effectively guide the mobile robot to return to the base station in a complex pre-station environment.

[0148] In some embodiments, the base station is set at the bottom of the household appliance or as a part of the household appliance, and the image acquisition device is set on the household appliance; or the base station is set at the bottom of the furniture or as a part of the furniture, and the image acquisition device is set on the furniture.

[0149] Among them, the household appliances may include clothing processing equipment, which may be washing machines, dryers, and integrated washing and drying machines. The household appliances may also be any one of refrigerators, dishwashers, televisions, air purifiers, water purifiers, air conditioners, steaming and baking equipment, sinks, integrated stoves, etc. The furniture may be any one of shoe cabinets, TV cabinets, bookcases, bathtubs, fish tanks, cabinets, toilets, immovable tables and chairs, bars, washbasins, etc.

[0150] For example, see Fig. 9 , Fig. 9910 is the location of the image acquisition device, and 920 is the location of the base station. Since the clothing processing device is equipped with an independent socket for power supply and automatic water supply and discharge devices, the base station can automatically drain water through the drain port shared with the clothing processing device. Therefore, it is very convenient to install the base station next to the clothing processing device. Installing an image acquisition device on the clothing processing device can collect the working environment of the base station so that the mobile robot can plan the path back to the base station.

[0151] The embodiment of the present specification also provides a method for guiding a mobile robot to return to a station, wherein a base station is arranged at the bottom of a household appliance, and an image acquisition device is arranged on the household appliance. The household appliance includes a clothing processing device. For example, refer to Fig.10 The method for guiding the mobile robot back to the station may include the following steps:

[0152] S1002: Receive an environment image sent by an image acquisition device.

[0153] The environmental image is obtained by photographing the base station working environment through an image acquisition device.

[0154] S1004: Perform obstacle detection on the environment image to obtain the image coordinate system coordinates of the obstacle in the environment image.

[0155] S1006: Perform coordinate conversion according to the image coordinate system coordinates of the obstacle and the internal and external parameter data of the image acquisition device to obtain first position data of the obstacle in the operation map of the mobile robot.

[0156] The first position data is calculated based on the environment image and the internal and external parameter data of the image acquisition device at the preset position; the height of the image acquisition device is greater than the height of the base station.

[0157] S1008. Determine the second position data of the mobile robot in the operation map based on the environmental image and the internal and external parameter data of the image acquisition device.

[0158] S1010: If the position of the obstacle does not change, path planning is performed based on the first position data of the obstacle, the second position data of the mobile robot, and the position of the return-to-station reference line to obtain a first return-to-station path.

[0159] The return reference line of the base station is relatively fixed to the image acquisition device; the position of the return reference line is determined by the external parameters of the image acquisition device and the relative position data between the return reference line and the image acquisition device.

[0160] S1012: If the position of the obstacle changes, receive a real-time image of the environment after the position of the obstacle changes, which is sent by an image acquisition device in real time.

[0161] S1014: Calculate based on the real-time image of the environment and the internal and external parameter data of the image acquisition device to obtain third position data.

[0162] S1016. Re-plan the path based on the third position data and the real-time position data of the mobile robot to obtain a second return path for the mobile robot to return to the base station.

[0163] This specification embodiment provides a device 1100 for guiding a mobile robot to return to a station. Fig.11 The device 1100 for guiding the mobile robot to return to the station includes: an environment image acquisition module 1110 and an identification and guidance module 1120.

[0164] The environment image acquisition module 1110 is used to acquire an environment image; wherein the environment image is obtained by photographing the base station working environment through an image acquisition device; the height of the image acquisition device is greater than the height of the top of the base station;

[0165] The identification and guidance module 1120 is used to identify the mobile robot and / or guide the mobile robot back to the base station based at least in part on the environment image.

[0166] For a detailed description of the device for guiding the mobile robot to return to the station, please refer to the description of the method for guiding the mobile robot to return to the station above, which will not be repeated here.

[0167] The embodiment of the present specification provides a base station, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described in any one of the above embodiments when executing the computer program. The household appliance includes a clothing processing device.

[0168] The embodiment of the present specification provides a household appliance, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described in any of the above embodiments when executing the computer program. The household appliance includes a clothes processing device.

[0169] An embodiment of the present specification provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method described in any one of the above embodiments are implemented.

[0170] One embodiment of the present specification provides a computer program product, wherein the computer program product includes instructions, and when the instructions are executed by a processor of a computer device, the computer device is enabled to perform the steps of the method described in any one of the above embodiments.

[0171] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

Claims

1. A method for guiding a mobile robot to return to a station, characterized in that: The method comprises: Acquire an environmental image; wherein the environmental image is obtained by photographing the base station working environment through an image acquisition device; the height of the image acquisition device is greater than the height of the top of the base station; Based at least in part on the image of the environment, a mobile robot is identified and / or directed to return to the base station.

2. The method according to claim 1, characterized in that: The step of guiding the mobile robot to return to the base station comprises: Determining first position data of an obstacle in the working environment of the base station based on the environment image; Performing path planning based on the first position data and the second position data of the mobile robot to obtain a first return path for the mobile robot to return to the base station; The mobile robot is guided to return to the base station according to the first return path.

3. The method according to claim 1, characterized in that The step of guiding the mobile robot to return to the base station comprises: Based on the environment image, determining second position data of the mobile robot; Performing path planning based at least in part on the second position data to obtain a first return path for the mobile robot to return to the base station; The mobile robot is guided to return to the base station according to the first return path.

4. The method according to claim 2, characterized in that: The method for determining the first position data includes: Receiving the environment image sent by the image acquisition device; Perform obstacle detection based on the environment image to obtain the image coordinate system coordinates of the obstacle in the environment image; Coordinate conversion is performed according to the image coordinate system coordinates of the obstacle and the internal and external parameter data of the image acquisition device to obtain the first position data of the obstacle in the operation map.

5. The method according to claim 1, characterized in that Based at least in part on the image of the environment, directing the mobile robot back to the base station includes: Determine the second position data of the mobile robot in the operation map based on the environmental image and the internal and external parameter data of the image acquisition device; determine the return reference point of the base station according to the external parameter data of the image acquisition device and the relative position data between the image acquisition device and the return reference point; guide the mobile robot back to the base station through the second position data and the return reference point; or Based on the environmental image and the internal and external parameter data of the image acquisition device, the second position data of the mobile robot in the operation map is determined; according to the second position data and the position data of the base station, path planning is performed to guide the mobile robot back to the base station.

6. The method according to claim 2, characterized in that The method comprises: If the position of the obstacle changes, obtaining third position data of the obstacle after the change; The path is re-planned based on the third position data and the real-time position data of the mobile robot to obtain a second return path for the mobile robot to return to the base station.

7. The method according to claim 6, characterized in that If the position of the obstacle changes, obtaining third position data of the obstacle after the change includes: Receiving a real-time image of the environment after the position of the obstacle changes, which is sent in real time by the image acquisition device; The third position data is obtained by performing calculation based on the real-time image of the environment and the internal and external parameter data of the image acquisition device.

8. The method according to claim 3, characterized in that The return-to-station reference line of the base station is relatively fixed to the image acquisition device; the position of the return-to-station reference line is determined by external parameters of the image acquisition device and relative position data between the return-to-station reference line and the image acquisition device.

9. The method according to claim 8, characterized in that Performing path planning based at least in part on the second position data to obtain a first return path for the mobile robot to return to the base station includes: Path planning is performed at least partially based on the second position data of the mobile robot and the position of the return-to-station reference line to obtain the first return-to-station path.

10. The method according to claim 9, characterized in that Before performing path planning based at least in part on the second position data of the mobile robot and the position of the return-to-station reference line to obtain the first return-to-station path, the method further includes: Determining first position data of an obstacle in the working environment of the base station based on the environment image; The performing path planning at least partially based on the second position data of the mobile robot and the position of the return-to-station reference line to obtain the first return-to-station path includes: Path planning is performed based on the first position data of the obstacle, the second position data of the mobile robot, and the position of the return-to-station reference line to obtain the first return-to-station path.

11. The method according to any one of claims 1 to 10, characterized in that: The base station is arranged at the bottom of the household appliance or is a part of the household appliance, and the image acquisition device is arranged on the household appliance; or The base station is arranged at the bottom of the furniture or as a part of the furniture, and the image acquisition device is arranged on the furniture.

12. The method according to claim 11, characterized in that The household appliance includes a clothes treating device.

13. A base station, characterized in that: The base station comprises: a processor and a memory configured to store a computer program that can be run on the processor, wherein the processor is configured to execute the steps of the method according to any one of claims 1 to 12 when running the computer program.

14. A household appliance, comprising: A processor and a memory configured to store a computer program that can be run on the processor, wherein the processor is configured to perform the steps of the method according to any one of claims 1 to 12 when running the computer program.

15. The device according to claim 14, characterized in that The household appliance includes a clothes treating device.

16. A device for guiding a mobile robot to return to a station, characterized in that: The device comprises: An environmental image acquisition module, used to acquire an environmental image; wherein the environmental image is obtained by photographing the base station working environment through an image acquisition device; the height of the image acquisition device is greater than the height of the top of the base station; An identification and guidance module is used to identify the mobile robot and / or guide the mobile robot back to the base station based at least in part on the environment image.