Mobile robot control method and device, storage medium and mobile robot

Through the image sensor and electronic devices in the home working together, a map of escape is generated and a remote control robot is solved, and the problem of wheeled robots is improved and the success rate and efficiency of escape is improved.

CN118938887APending Publication Date: 2024-11-12MIDEA ROBOZONE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310519062.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-10
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Wheeled robots are easily trapped when encountering obstacles, and the existing technology is difficult to effectively get out of trouble, resulting in a low success rate for robots to get out of trouble.

Method used

By obtaining the trapped signals sent by the robot, using image sensors to collect trapped images, the server controls the robot to perform escape movement, uses electronic devices in the home such as surveillance cameras or home appliances to capture the trapped situation of the robot, generate a escape map and send a escape route , the user escapes through the terminal remote control robot.

Benefits of technology

It improves the efficiency and success rate of robots to get out of trouble, reduces manual intervention, and enhances the robot's autonomous operation ability in complex environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118938887A_ABST
    Figure CN118938887A_ABST
Patent Text Reader

Abstract

The invention provides a control method and device of a mobile robot, a storage medium and the mobile robot, and belongs to the technical field of robot control. The control method comprises the following steps: acquiring a trapped signal of a trapped robot; determining an image sensor, wherein the visual field of the image sensor covers the trapped robot or the image sensor is arranged on the trapped robot; acquiring an image of an image sensor; and controlling the trapped robot to execute the escape motion according to the image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of robot control technology, and in particular to a control method, device, storage medium and mobile robot of a mobile robot. Background Art

[0002] In the related art, when a wheeled robot, such as a sweeping robot, is working, it may be disturbed by obstacles, causing it to be unable to continue moving along a set route, causing the robot to be trapped.

[0003] When a general robot is trapped, it cannot move freely and the success rate of escaping from the trap is low. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art or related art.

[0005] To this end, a first aspect of the present invention provides a control method for a mobile robot.

[0006] A second aspect of the present invention provides a control method for a mobile robot.

[0007] A third aspect of the present invention provides a control method for a mobile robot.

[0008] A fourth aspect of the present invention provides a control method for a mobile robot.

[0009] A fifth aspect of the present invention provides a control device for a mobile robot.

[0010] A sixth aspect of the present invention provides a control device for a mobile robot.

[0011] A seventh aspect of the present invention provides a control device for a mobile robot.

[0012] An eighth aspect of the present invention provides a control device for a mobile robot.

[0013] A ninth aspect of the present invention provides a control device for a mobile robot.

[0014] A tenth aspect of the present invention provides a readable storage medium.

[0015] An eleventh aspect of the present invention provides a mobile robot.

[0016] In view of this, a first aspect of the present invention provides a control method for a mobile robot, which is applied to a server. The control method includes:

[0017] Acquire a trapped signal of the trapped robot; determine an image sensor, the field of view of the image sensor covers the trapped robot or the image sensor is set on the trapped robot; acquire an image from the image sensor; and control the trapped robot to execute an escape movement according to the image.

[0018] In this technical solution, the robot may be a robot with mobility used in the home, such as a sweeping robot, a mopping robot or a patrol robot. When the robot is performing a task, the robot may be blocked by an obstacle. In this case, the robot will first try to cross the obstacle or adjust the direction of travel to bypass the obstacle.

[0019] If the robot fails to cross or bypass an obstacle after multiple attempts, the robot determines that it has entered a trapped state, and then the robot sends a trapped signal to the server.

[0020] Exemplarily, the trapped signal carries information such as the trapped state, location information, and identification information of the first robot.

[0021] The server can be a cloud server, a robot base station, or a device such as an intelligent gateway that establishes a wireless network connection with the robot.

[0022] After receiving the trapped signal of the mobile robot, the server determines the image sensor, which may be an image sensor in the area where the trapped robot is located, or an image sensor arranged on the robot.

[0023] The server obtains the image captured by the image sensor, which can reflect the trapped situation of the trapped robot. Therefore, the trapped robot is controlled to escape according to the image captured by the image sensor, so that the trapped robot can perform targeted escape movements according to the trapped situation, specifically the location of the obstacle, the stuck part, etc.

[0024] Exemplarily, the image sensor is an image sensor in the area where the trapped robot is located. The image sensor in the area where the trapped robot is located is used to capture an image of the trapped robot from a third perspective. The trapped condition of the robot is determined using the trapped image from the third perspective, and the robot is controlled to perform an escape movement.

[0025] Exemplarily, the image sensor is an image sensor provided on the trapped robot, which collects the trapped condition of the robot from a first perspective through the image sensor on the robot, determines the trapped state of the robot based on the trapped image from the first perspective, and controls the robot to execute an escape movement.

[0026] The embodiment of the present invention can effectively improve the robot's escape efficiency by acquiring a trapped image of the robot through an image sensor when the robot is trapped and controlling the robot to escape based on the trapped image.

[0027] In addition, the robot control method in the above technical solution provided by the present invention may also have the following additional technical features:

[0028] In the above technical solution, the step of determining the image sensor includes: determining the trapped area of ​​the trapped robot according to the trapped signal; determining at least one electronic device in the trapped area, wherein the at least one electronic device is provided with an image sensor;

[0029] The step of acquiring an image from an image sensor includes: controlling the electronic device to collect a trapped image corresponding to the trapped robot through the image sensor.

[0030] In this technical solution, when the server receives a trapped signal sent by the trapped robot, the server determines the identity and location information of the trapped robot based on the information carried in the trapped signal, such as determining that the robot is a sweeping robot and the trapped area is the living room.

[0031] After determining the trapped area of ​​the trapped robot according to the trapped signal, the server searches for at least one electronic device in the trapped area, wherein the electronic device is an electronic device having an image sensor.

[0032] For example, the electronic device can be a home surveillance camera, or a home appliance with a camera, such as a smart TV, a somatosensory game console, a smart refrigerator, a smart air conditioner, etc. The electronic device can also be other robots in the home. For example, if the trapped robot is a sweeping robot, the electronic device can also be a mopping robot or a patrol robot that is not trapped.

[0033] The above electronic devices form an Internet of Things through an intelligent gateway. After determining the trapped area of ​​the robot, the server sends a query request to the home Internet of Things through a network connection with the intelligent gateway. After successfully authenticating the request to the server, the home Internet of Things sends a device list of electronic devices with cameras in the trapped area, such as the living room, to the server.

[0034] The server determines at least one electronic device in the trapped area that can capture the trapped situation of the robot based on the received device list, and sends an escape signal to these electronic devices via the Internet.

[0035] After receiving the escape signal, the electronic device in the trapped area first authenticates the server to determine whether the server has the authority to obtain image data. After successful authentication, the electronic device captures image information in the trapped area according to the escape signal, specifically capturing the trapped image of the trapped robot.

[0036] Exemplarily, the escape signal carries the identification information and trapped position of the trapped robot, and the electronic device photographs the trapped position of the trapped robot according to the escape signal, thereby acquiring an image of the trapped robot.

[0037] After obtaining the trapped image of the robot, the obstacle information near the robot can be determined, such as the obstacle position, shape, height, etc. Through this obstacle information, the trapped robot can be helped to plan an escape route, thereby helping the trapped robot to perform escape actions.

[0038] It is understandable that in some specific scenarios, the robot may be blocked by obstacles, such as the robot being stuck under a sofa. At this time, the server can determine whether the image information captured by the electronic devices in the current trapped area includes the trapped robot. If at least one electronic device can capture the trapped robot, the trapped robot is helped out of the trapped area based on the image information captured of the trapped robot.

[0039] If the trapped robot is not included in the image information captured by all electronic devices in the trapped area, it is determined that the position of the trapped robot is blocked. At this time, the server controls the trapped robot to standby, and controls the electronic devices in the trapped area to stop capturing images, and sends a message to the user terminal that the robot is trapped and cannot escape by itself.

[0040] An embodiment of the present invention provides a collaborative escape method based on other electronic devices in the robot's working area. The electronic devices in the trapped area collect image information of the area where the trapped robot is located, that is, the robot's trapped image. The trapped image includes images of obstacles blocking the robot's movement, which can accurately reflect the robot's trapped situation. Therefore, by controlling the robot to execute escape movement through the trapped image, the robot's escape success rate can be effectively improved.

[0041] In any of the above technical solutions, the step of determining the image sensor includes: in a case where the trapped robot is equipped with an image acquisition device, determining the image acquisition device as the image sensor.

[0042] In this technical solution, after the server receives the trapped signal sent by the trapped robot, the server determines whether an image sensor device is provided on the trapped robot.

[0043] The image sensor device may be a robot visual sensor, such as a camera, etc. The server may send a query command to the trapped robot, and after receiving the query command, the robot returns a corresponding query result to the server, informing the server whether the server itself is provided with an image sensor device.

[0044] In the case where the robot is provided with an image sensing device, the server determines the image sensing device provided on the robot as an image sensor, and collects the trapped image of the robot by calling the image sensor on the robot. After obtaining the trapped image of the robot, the obstacle information near the robot can be determined, such as the obstacle position, shape, height, etc. The obstacle information can help the trapped robot plan an escape route, thereby helping the trapped robot to perform an escape action.

[0045] It can be understood that the image acquisition device set on the robot can be determined as the only image sensor, or after the image acquisition device set on the robot is determined as the image sensor, at least one electronic device in the trapped area of ​​the robot can be further determined, and the image sensor set on the electronic device can be determined as the image sensor used to capture the trapped image.

[0046] In any of the above technical solutions, the step of controlling the trapped robot to perform escape motion according to the image includes:

[0047] The image is sent to the target terminal; the remote control signal sent by the target terminal is received; the remote control signal is sent to the trapped robot, so that the trapped robot can perform an escape movement according to the remote control signal.

[0048] In this technical solution, the target terminal is specifically a terminal held by the user, such as the user's mobile phone, remote control, tablet computer, etc.

[0049] After receiving the trapped image of the trapped robot captured by the electronic device, the server sends the trapped image to the target terminal. It is understandable that the trapped image can be a real-time video stream. The server can compress the video stream file to reduce the demand for network bandwidth and reduce traffic consumption.

[0050] After receiving the trapped image, the target terminal displays the trapped image through the home device management application. The user can understand the specific situation of the robot being trapped through the trapped image, and according to the trapped image displayed by the target terminal, by inputting remote control commands, the remote control robot moves in the direction of avoiding obstacles, thereby achieving escape.

[0051] After the target terminal receives the remote control command input by the user, the target terminal generates a corresponding remote control signal according to the control command and sends the remote control signal to the server. After receiving the remote control signal, the server sends the remote control signal to the trapped robot, and the trapped robot performs the corresponding escape movement according to the remote control signal, such as the "forward" signal or the "backward" signal.

[0052] The embodiment of the present invention controls the electronic equipment in the trapped area where the trapped robot is located to capture the trapped image of the trapped robot, and sends the trapped image to the target terminal held by the user, thereby helping the user to grasp the trapped situation of the trapped robot based on the trapped image and remotely control the trapped robot to escape, thereby effectively improving the success rate of the robot's escape.

[0053] In any of the above technical solutions, the step of controlling the trapped robot to perform escape motion according to the image includes:

[0054] A corresponding escape map is generated according to the image; an escape route is determined according to the escape map; and the escape route is sent to the trapped robot so that the trapped robot can execute an escape movement according to the escape route.

[0055] In this technical solution, the trapped image can be picture data or video stream data. After the server receives the trapped image of the trapped robot taken by the electronic device, the position of the trapped robot and the positions of obstacles near the trapped robot are identified in the trapped image through an image recognition algorithm.

[0056] The server generates an escape map near the escape robot through a mapping algorithm based on the identified robot position and obstacle position. The escape map records the position and posture of the trapped robot and also includes the location information of nearby obstacles.

[0057] Exemplarily, the escape map may be grid map information including obstacle information.

[0058] After the escape map is generated, an escape route for helping the trapped robot escape is generated through a route algorithm based on the robot position and obstacle positions recorded in the escape map.

[0059] The server sends the escape route to the trapped robot, and the trapped robot drives its own running mechanism, such as the running wheels, according to the movement mode indicated by the escape route, and performs the escape movement according to the escape route.

[0060] The embodiment of the present invention constructs an escape map near the trapped robot through the trapped image acquired by the electronic device in the trapped area, and calculates an escape route suitable for the trapped robot according to the escape map, which can effectively improve the success rate of the robot's escape.

[0061] In any of the above technical solutions, determining the trapped area of ​​the trapped robot according to the trapped signal includes:

[0062] Determine the position information of the robot according to the trapped signal; obtain the working map of the trapped robot; and determine the trapped area within the working area of ​​the trapped robot according to the position information and the working map.

[0063] In this technical solution, the robot's work map can be a map created by the robot's own sensors during the work process, or a preset map. The work map records the area information of the robot's work area, and marks each sub-area in the work area based on preset annotation information.

[0064] Taking the robot's working area as the user's home interior as an example, the working map is a complete map of the interior, in which each sub-area is marked, such as the living room, bedroom, kitchen, etc.

[0065] After the robot is trapped, the robot sends a trapped signal to the server. The server determines the current area of ​​the robot, i.e., the trapped area of ​​the robot, based on the location information carried in the trapped signal and the robot's work map.

[0066] Taking the trapped area as the living room as an example, after the server determines the trapped area of ​​the trapped robot based on the working map and the location information carried in the trapped signal, it determines at least one electronic device with an image sensor in the trapped area, that is, the living room, and uses the electronic device to capture the trapped image of the trapped robot, and uses the trapped image to help the trapped robot escape.

[0067] The embodiment of the present invention determines the trapped area of ​​the trapped robot based on the robot's work map and the position information of the robot when it is trapped, and then controls the electronic equipment in the trapped area to obtain the trapped image of the trapped robot. The robot is controlled to escape through the trapped image, which can improve the success rate of the robot's escape.

[0068] In any of the above technical solutions, determining at least one electronic device in the trapped area includes:

[0069] A query signal is sent to a target gateway, and the target gateway is in communication connection with a plurality of candidate electronic devices in the trapped area; and at least one electronic device is determined from the plurality of candidate electronic devices according to a device list received from the target gateway.

[0070] In this technical solution, the target gateway can be a smart gateway in the user's home. Multiple smart home appliances, surveillance cameras, robots and other electronic devices in the user's home are all connected to the smart gateway and access the Internet through the smart gateway.

[0071] When the server receives the trapped signal sent by the trapped robot, it identifies the target gateway in the Internet of Things environment where the trapped robot is located according to the identity information of the trapped robot carried in the trapped signal.

[0072] After determining the target gateway, the server sends a query signal to the target gateway, and the query signal carries the server's identity information and the information of the trapped area. After receiving the query signal, the target gateway authenticates the server according to the identity information. After the authentication is successful, the target gateway determines multiple candidate electronic devices located in the trapped area from the electronic devices connected to the target gateway according to the information of the trapped area.

[0073] The candidate electronic devices are all electronic devices connected to the target gateway, such as smart lamps, smart TVs, electric curtains, somatosensory game consoles, and inspection robots. The target gateway sends the device list of the above electronic devices to the server.

[0074] After receiving the device list, the server selects at least one electronic device with an image sensor from the device list, uses the at least one electronic device with an image sensor to capture a trapped image of the trapped robot, and uses the trapped image to help the robot escape.

[0075] The embodiment of the present invention obtains a device list of alternative electronic devices in the trapped area through a target gateway in the same home scene as the trapped robot, and then determines at least one electronic device equipped with an image sensor in the device list, and helps the trapped robot escape through the trapped image captured by the electronic device, thereby improving the success rate of the robot's escape.

[0076] A second aspect of the present invention provides a control method for a mobile robot, which is applied to the robot. The control method comprises:

[0077] When it is detected that the robot is in a trapped state, a trapped signal is sent to the server; a remote control signal sent by the target terminal is received, the remote control signal is a signal generated by the target terminal according to the received remote control input when the trapped image is displayed, the trapped image is an image captured by an image sensor, the field of view of the image sensor covers the trapped robot or the image sensor is set on the trapped robot; and an escape movement is executed according to the remote control signal.

[0078] In this technical solution, the robot may be a robot with mobility used in the home, such as a sweeping robot, a mopping robot or a patrol robot. When the robot is performing a task, the robot may be blocked by an obstacle. In this case, the robot will first try to cross the obstacle or adjust the direction of travel to bypass the obstacle.

[0079] If the robot fails to cross or bypass an obstacle after multiple attempts, the robot determines that it has entered a trapped state, and then the robot sends a trapped signal to the server.

[0080] The server can be a cloud server, a robot base station, or a device such as an intelligent gateway that establishes a wireless network connection with the robot.

[0081] After the server receives the trapped signal sent by the trapped robot, the server determines the image sensor, which may be an image sensor in the area where the trapped robot is located, or an image sensor disposed on the robot.

[0082] The server obtains the image captured by the image sensor, which can reflect the trapped situation of the trapped robot. Therefore, the trapped robot is controlled to escape according to the image captured by the image sensor, so that the trapped robot can perform targeted escape movements according to the trapped situation, specifically the location of the obstacle, the stuck part, etc.

[0083] Exemplarily, the server determines the identity and location information of the trapped robot based on the information carried in the trapped signal, such as determining that the robot is a sweeping robot and the trapped area is the living room.

[0084] After determining the trapped area of ​​the trapped robot based on the trapped signal, the server searches for at least one electronic device in the trapped area, wherein the electronic device is an electronic device with an image sensor. The server sends an escape signal to the electronic device with the image sensor, and the escape signal carries the identification information and trapped position of the trapped robot. The electronic device photographs the trapped position of the trapped robot based on the escape signal, thereby acquiring a trapped image of the robot.

[0085] Exemplarily, the image sensor is an image sensor provided on the trapped robot, and the trapped condition of the robot from a first perspective is collected by the image sensor on the robot, and the trapped state of the robot is determined by the trapped image from the first perspective.

[0086] After receiving the trapped image, the server sends the trapped image to the target terminal, wherein the target terminal is specifically a terminal held by the user, such as the user's mobile phone, remote control, tablet computer, etc.

[0087] After receiving the trapped image, the target terminal displays the trapped image through the home device management application. The user can understand the specific situation of the robot being trapped through the trapped image, and according to the trapped image displayed by the target terminal, the user can input remote control commands to remotely control the robot to move in the direction of avoiding obstacles.

[0088] After the target terminal receives the remote control command input by the user, the target terminal generates a corresponding remote control signal according to the control command and sends the remote control signal to the server. After receiving the remote control signal, the server sends the remote control signal to the trapped robot, and the trapped robot performs the corresponding escape movement according to the remote control signal, such as the "forward" signal or the "backward" signal.

[0089] The embodiment of the present invention helps the user to understand the trapped situation of the trapped robot according to the trapped image by sending the trapped image to the target terminal held by the user, and remotely control the trapped robot to escape, thereby effectively improving the success rate of the robot's escape.

[0090] A third aspect of the present invention provides a control method for a mobile robot, which is applied to the robot. The control method comprises:

[0091] When it is detected that the robot is in a trapped state, a trapped signal is sent to the server; an escape route sent by the server is received, the escape route is a route determined by the server based on the trapped image, the trapped image is an image captured by an image sensor, the field of view of the image sensor covers the trapped robot or the image sensor is set on the trapped robot; and an escape movement is executed according to the escape route.

[0092] In this technical solution, the robot may be a robot with mobility used in the home, such as a sweeping robot, a mopping robot or a patrol robot. When the robot is performing a task, the robot may be blocked by an obstacle. In this case, the robot will first try to cross the obstacle or adjust the direction of travel to bypass the obstacle.

[0093] If the robot fails to cross or bypass an obstacle after multiple attempts, the robot determines that it has entered a trapped state, and then the robot sends a trapped signal to the server.

[0094] The server can be a cloud server, a robot base station, or a device such as an intelligent gateway that establishes a wireless network connection with the robot.

[0095] After the server receives the trapped signal sent by the trapped robot, the server determines the image sensor, which may be an image sensor in the area where the trapped robot is located, or an image sensor disposed on the robot.

[0096] The server obtains the image captured by the image sensor, which can reflect the trapped situation of the trapped robot. Therefore, the trapped robot is controlled to escape according to the image captured by the image sensor, so that the trapped robot can perform targeted escape movements according to the trapped situation, specifically the location of the obstacle, the stuck part, etc.

[0097] Exemplarily, the server determines the identity and location information of the trapped robot based on the information carried in the trapped signal, such as determining that the robot is a sweeping robot and the trapped area is the living room.

[0098] After determining the trapped area of ​​the trapped robot based on the trapped signal, the server searches for at least one electronic device in the trapped area, wherein the electronic device is an electronic device with an image sensor. The server sends an escape signal to the electronic device with the image sensor, and the escape signal carries the identification information and trapped position of the trapped robot. The electronic device photographs the trapped position of the trapped robot based on the escape signal, thereby acquiring a trapped image of the robot.

[0099] Exemplarily, the image sensor is an image sensor provided on the trapped robot, and the trapped condition of the robot from a first perspective is collected by the image sensor on the robot, and the trapped state of the robot is determined by the trapped image from the first perspective.

[0100] After receiving the trapped image, the server identifies the position of the trapped robot and the positions of obstacles near the trapped robot in the trapped image through an image recognition algorithm. In some embodiments, the escape map can be a map generated by updating the working map of the trapped robot based on the position and posture of the trapped robot and the position information of nearby obstacles.

[0101] The server generates an escape map near the escape robot through a mapping algorithm based on the identified robot position and obstacle position. The escape map records the position and posture of the trapped robot and also includes the location information of nearby obstacles.

[0102] Exemplarily, the escape map may be grid map information including obstacle information.

[0103] After the escape map is generated, an escape route for helping the trapped robot escape is generated through a route algorithm based on the robot position and obstacle positions recorded in the escape map.

[0104] The server sends the escape route to the trapped robot, and the trapped robot drives its own running mechanism, such as the running wheels, according to the movement mode indicated by the escape route, and performs the escape movement according to the escape route.

[0105] The embodiment of the present invention constructs an escape map near the trapped robot through the trapped image, and calculates an escape route suitable for the trapped robot according to the escape map, which can effectively improve the success rate of the robot's escape.

[0106] A fourth aspect of the present invention provides a control method for a mobile robot, which is applied to a terminal. The control method includes:

[0107] Receive a trapped image sent by a server, where the trapped image is an image captured by an image sensor, and the field of view of the image sensor covers the trapped robot or the image sensor is set on the trapped robot; display the trapped image and receive remote control input; determine the remote control signal in response to the remote control input; send the remote control signal to the server, so that the server can send the remote control signal to the trapped robot, so that the trapped robot can perform escape movement according to the remote control signal.

[0108] In this technical solution, the robot may be a robot with mobility used in the home, such as a sweeping robot, a mopping robot or a patrol robot. When the robot is performing a task, the robot may be blocked by an obstacle. In this case, the robot will first try to cross the obstacle or adjust the direction of travel to bypass the obstacle.

[0109] If the robot fails to cross or bypass an obstacle after multiple attempts, the robot determines that it has entered a trapped state, and the robot sends a trapped signal to the server. When the server receives the trapped signal sent by the trapped robot, the server determines the image sensor, which can be an image sensor in the area where the trapped robot is located, or an image sensor set on the robot.

[0110] The server obtains the image captured by the image sensor, which can reflect the trapped situation of the trapped robot. Therefore, the trapped robot is controlled to escape according to the image captured by the image sensor, so that the trapped robot can perform targeted escape movements according to the trapped situation, specifically the location of the obstacle, the stuck part, etc.

[0111] Exemplarily, the server determines the identity and location information of the trapped robot based on the information carried in the trapped signal, such as determining that the robot is a sweeping robot and the trapped area is the living room.

[0112] At least one electronic device is found in the trapped area. The electronic device may be a home surveillance camera or a home appliance with a camera, such as a smart TV, a somatosensory game console, a smart refrigerator, a smart air conditioner, etc. The electronic device may also be other robots in the home. For example, if the trapped robot is a sweeping robot, the electronic device may also be a mopping robot or a patrol robot that is not trapped.

[0113] The electronic device photographs the trapped position of the trapped robot according to the escape signal, thereby collecting the trapped image of the robot, and sends the trapped image to the server.

[0114] Exemplarily, the image sensor is an image sensor provided on the trapped robot, and the trapped condition of the robot from a first perspective is collected by the image sensor on the robot, and the trapped state of the robot is determined by the trapped image from the first perspective.

[0115] After receiving the trapped image, the terminal displays the trapped image through the home device management application. The user can understand the specific situation of the robot being trapped through the trapped image, and according to the trapped image displayed by the terminal, by inputting remote control commands, the remote control robot moves in the direction of avoiding obstacles, thereby achieving escape. After the server receives the trapped image of the trapped robot taken by the electronic device, the trapped image is sent to the terminal. It can be understood that the trapped image can be a real-time video stream. The server can compress the video stream file to reduce the demand for network bandwidth and reduce traffic consumption.

[0116] After the terminal receives the remote control command input by the user, the terminal generates a corresponding remote control signal according to the control command and sends the remote control signal to the server. After receiving the remote control signal, the server sends the remote control signal to the trapped robot, and the trapped robot performs the corresponding escape movement according to the remote control signal, such as the "forward" signal or the "backward" signal.

[0117] The embodiment of the present invention helps the user to understand the trapped situation of the trapped robot according to the trapped image by sending the trapped image to the terminal held by the user, and remotely control the trapped robot to escape, thereby effectively improving the success rate of the robot's escape.

[0118] A fifth aspect of the present invention provides a control device for a mobile robot, which is applied to a server, and the control device includes:

[0119] An acquisition module, used for acquiring a trapped signal of the trapped robot;

[0120] A determination module, used to determine an image sensor, the field of view of the image sensor covers the trapped robot or the image sensor is arranged on the trapped robot;

[0121] The acquisition module is also used to acquire the image of the image sensor;

[0122] The control module is used to control the trapped robot to perform escape movement according to the image.

[0123] In this technical solution, the robot may be a robot with mobility used in the home, such as a sweeping robot, a mopping robot or a patrol robot. When the robot is performing a task, the robot may be blocked by an obstacle. In this case, the robot will first try to cross the obstacle or adjust the direction of travel to bypass the obstacle.

[0124] If the robot fails to cross or bypass an obstacle after multiple attempts, the robot determines that it has entered a trapped state, and then the robot sends a trapped signal to the server.

[0125] Exemplarily, the trapped signal carries information such as the trapped state, location information, and identification information of the first robot.

[0126] The server can be a cloud server, a robot base station, or a device such as an intelligent gateway that establishes a wireless network connection with the robot.

[0127] After receiving the trapped signal of the mobile robot, the server determines the image sensor, which may be an image sensor in the area where the trapped robot is located, or an image sensor arranged on the robot.

[0128] The server obtains the image captured by the image sensor, which can reflect the trapped situation of the trapped robot. Therefore, the trapped robot is controlled to escape according to the image captured by the image sensor, so that the trapped robot can perform targeted escape movements according to the trapped situation, specifically the location of the obstacle, the stuck part, etc.

[0129] Exemplarily, the image sensor is an image sensor in the area where the trapped robot is located. The image sensor in the area where the trapped robot is located is used to capture an image of the trapped robot from a third perspective. The trapped condition of the robot is determined using the trapped image from the third perspective, and the robot is controlled to perform an escape movement.

[0130] Exemplarily, the image sensor is an image sensor provided on the trapped robot, which collects the trapped condition of the robot from a first perspective through the image sensor on the robot, determines the trapped state of the robot based on the trapped image from the first perspective, and controls the robot to execute an escape movement.

[0131] The embodiment of the present invention can effectively improve the robot's escape efficiency by acquiring a trapped image of the robot through an image sensor when the robot is trapped and controlling the robot to escape based on the trapped image.

[0132] A sixth aspect of the present invention provides a control device for a mobile robot, which is applied to the robot, and the control device comprises:

[0133] The sending module is used to send a trapped signal to the server when the robot is detected to be in a trapped state;

[0134] A receiving module, used for receiving a remote control signal sent by a target terminal, the remote control signal being a signal generated by the target terminal according to a received remote control input when displaying a trapped image, the trapped image being an image captured by an image sensor, the field of view of the image sensor covering the trapped robot or the image sensor being arranged on the trapped robot;

[0135] The execution module is used to execute the escape movement according to the remote control signal.

[0136] In this technical solution, the robot may be a robot with mobility used in the home, such as a sweeping robot, a mopping robot or a patrol robot. When the robot is performing a task, the robot may be blocked by an obstacle. In this case, the robot will first try to cross the obstacle or adjust the direction of travel to bypass the obstacle.

[0137] If the robot fails to cross or bypass an obstacle after multiple attempts, the robot determines that it has entered a trapped state, and then the robot sends a trapped signal to the server.

[0138] The server can be a cloud server, a robot base station, or a device such as an intelligent gateway that establishes a wireless network connection with the robot.

[0139] After the server receives the trapped signal sent by the trapped robot, the server determines the image sensor, which may be an image sensor in the area where the trapped robot is located, or an image sensor disposed on the robot.

[0140] The server obtains the image captured by the image sensor, which can reflect the trapped situation of the trapped robot. Therefore, the trapped robot is controlled to escape according to the image captured by the image sensor, so that the trapped robot can perform targeted escape movements according to the trapped situation, specifically the location of the obstacle, the stuck part, etc.

[0141] Exemplarily, the server determines the identity and location information of the trapped robot based on the information carried in the trapped signal, such as determining that the robot is a sweeping robot and the trapped area is the living room.

[0142] After determining the trapped area of ​​the trapped robot based on the trapped signal, the server searches for at least one electronic device in the trapped area, wherein the electronic device is an electronic device with an image sensor. The server sends an escape signal to the electronic device with the image sensor, and the escape signal carries the identification information and trapped position of the trapped robot. The electronic device photographs the trapped position of the trapped robot based on the escape signal, thereby acquiring a trapped image of the robot.

[0143] Exemplarily, the image sensor is an image sensor provided on the trapped robot, and the trapped condition of the robot from a first perspective is collected by the image sensor on the robot, and the trapped state of the robot is determined by the trapped image from the first perspective.

[0144] After receiving the trapped image, the server sends the trapped image to the target terminal, wherein the target terminal is specifically a terminal held by the user, such as the user's mobile phone, remote control, tablet computer, etc.

[0145] After receiving the trapped image, the target terminal displays the trapped image through the home device management application. The user can understand the specific situation of the robot being trapped through the trapped image, and according to the trapped image displayed by the target terminal, the user can input remote control commands to remotely control the robot to move in the direction of avoiding obstacles.

[0146] After the target terminal receives the remote control command input by the user, the target terminal generates a corresponding remote control signal according to the control command and sends the remote control signal to the server. After receiving the remote control signal, the server sends the remote control signal to the trapped robot, and the trapped robot performs the corresponding escape movement according to the remote control signal, such as the "forward" signal or the "backward" signal.

[0147] The embodiment of the present invention helps the user to understand the trapped situation of the trapped robot according to the trapped image by sending the trapped image to the target terminal held by the user, and remotely control the trapped robot to escape, thereby effectively improving the success rate of the robot's escape.

[0148] A seventh aspect of the present invention provides a control device for a mobile robot, which is applied to the robot, and the control device comprises:

[0149] The sending module is used to send a trapped signal to the server when the robot is detected to be in a trapped state;

[0150] A receiving module is used to receive an escape route sent by a server, where the escape route is a route determined by the server according to the trapped image, where the trapped image is an image captured by an image sensor, where the field of view of the image sensor covers the trapped robot or the image sensor is arranged on the trapped robot;

[0151] The execution module is used to execute the escape movement according to the escape route.

[0152] In this technical solution, the robot may be a robot with mobility used in the home, such as a sweeping robot, a mopping robot or a patrol robot. When the robot is performing a task, the robot may be blocked by an obstacle. In this case, the robot will first try to cross the obstacle or adjust the direction of travel to bypass the obstacle.

[0153] If the robot fails to cross or bypass an obstacle after multiple attempts, the robot determines that it has entered a trapped state, and then the robot sends a trapped signal to the server.

[0154] The server can be a cloud server, a robot base station, or a device such as an intelligent gateway that establishes a wireless network connection with the robot.

[0155] After the server receives the trapped signal sent by the trapped robot, the server determines the image sensor, which may be an image sensor in the area where the trapped robot is located, or an image sensor disposed on the robot.

[0156] The server obtains the image captured by the image sensor, which can reflect the trapped situation of the trapped robot. Therefore, the trapped robot is controlled to escape according to the image captured by the image sensor, so that the trapped robot can perform targeted escape movements according to the trapped situation, specifically the location of the obstacle, the stuck part, etc.

[0157] Exemplarily, the server determines the identity and location information of the trapped robot based on the information carried in the trapped signal, such as determining that the robot is a sweeping robot and the trapped area is the living room.

[0158] After determining the trapped area of ​​the trapped robot based on the trapped signal, the server searches for at least one electronic device in the trapped area, wherein the electronic device is an electronic device with an image sensor. The server sends an escape signal to the electronic device with the image sensor, and the escape signal carries the identification information and trapped position of the trapped robot. The electronic device photographs the trapped position of the trapped robot based on the escape signal, thereby acquiring a trapped image of the robot.

[0159] Exemplarily, the image sensor is an image sensor provided on the trapped robot, and the trapped condition of the robot from a first perspective is collected by the image sensor on the robot, and the trapped state of the robot is determined by the trapped image from the first perspective.

[0160] After receiving the trapped image of the trapped robot taken by the electronic device, the server identifies the position of the trapped robot and the positions of obstacles near the trapped robot in the trapped image through an image recognition algorithm.

[0161] The server generates an escape map near the escape robot through a mapping algorithm based on the identified robot position and obstacle position. The escape map records the position and posture of the trapped robot and also includes the location information of nearby obstacles.

[0162] Exemplarily, the escape map may be grid map information including obstacle information.

[0163] After the escape map is generated, an escape route for helping the trapped robot escape is generated through a route algorithm based on the robot position and obstacle positions recorded in the escape map.

[0164] The server sends the escape route to the trapped robot, and the trapped robot drives its own running mechanism, such as the running wheels, according to the movement mode indicated by the escape route, and performs the escape movement according to the escape route.

[0165] The embodiment of the present invention constructs an escape map near the trapped robot through the trapped image, and calculates an escape route suitable for the trapped robot according to the escape map, which can effectively improve the success rate of the robot's escape.

[0166] An eighth aspect of the present invention provides a control device for a mobile robot, which is applied to a terminal, and the control device includes:

[0167] A receiving module, used for receiving a trapped image sent by a server, where the trapped image is an image captured by an image sensor, the field of view of the image sensor covers the trapped robot or the image sensor is arranged on the trapped robot;

[0168] A display module, used to display the trapped image and receive remote control input;

[0169] a determination module, configured to determine a remote control signal in response to a remote control input;

[0170] The sending module is used to send the remote control signal to the server, so that the server sends the remote control signal to the trapped robot, so that the trapped robot can perform the escape movement according to the remote control signal.

[0171] In this technical solution, the robot may be a robot with mobility used in the home, such as a sweeping robot, a mopping robot or a patrol robot. When the robot is performing a task, the robot may be blocked by an obstacle. In this case, the robot will first try to cross the obstacle or adjust the direction of travel to bypass the obstacle.

[0172] If the robot fails to cross or bypass an obstacle after multiple attempts, the robot determines that it has entered a trapped state, and the robot sends a trapped signal to the server. When the server receives the trapped signal sent by the trapped robot, the server determines the image sensor, which can be an image sensor in the area where the trapped robot is located, or an image sensor set on the robot.

[0173] The server obtains the image captured by the image sensor, which can reflect the trapped situation of the trapped robot. Therefore, the trapped robot is controlled to escape according to the image captured by the image sensor, so that the trapped robot can perform targeted escape movements according to the trapped situation, specifically the location of the obstacle, the stuck part, etc.

[0174] Exemplarily, the server determines the identity and location information of the trapped robot based on the information carried in the trapped signal, such as determining that the robot is a sweeping robot and the trapped area is the living room.

[0175] At least one electronic device is found in the trapped area. The electronic device may be a home surveillance camera or a home appliance with a camera, such as a smart TV, a somatosensory game console, a smart refrigerator, a smart air conditioner, etc. The electronic device may also be other robots in the home. For example, if the trapped robot is a sweeping robot, the electronic device may also be a mopping robot or a patrol robot that is not trapped.

[0176] The electronic device photographs the trapped position of the trapped robot according to the escape signal, thereby collecting the trapped image of the robot, and sends the trapped image to the server.

[0177] Exemplarily, the image sensor is an image sensor provided on the trapped robot, and the trapped condition of the robot from a first perspective is collected by the image sensor on the robot, and the trapped state of the robot is determined by the trapped image from the first perspective.

[0178] After receiving the trapped image, the terminal displays the trapped image through the home device management application. The user can understand the specific situation of the robot being trapped through the trapped image, and according to the trapped image displayed by the terminal, by inputting remote control commands, the remote control robot moves in the direction of avoiding obstacles, thereby achieving escape. After the server receives the trapped image of the trapped robot taken by the electronic device, the trapped image is sent to the terminal. It can be understood that the trapped image can be a real-time video stream. The server can compress the video stream file to reduce the demand for network bandwidth and reduce traffic consumption.

[0179] After the terminal receives the remote control command input by the user, the terminal generates a corresponding remote control signal according to the control command and sends the remote control signal to the server. After receiving the remote control signal, the server sends the remote control signal to the trapped robot, and the trapped robot performs the corresponding escape movement according to the remote control signal, such as the "forward" signal or the "backward" signal.

[0180] The embodiment of the present invention helps the user to understand the trapped situation of the trapped robot according to the trapped image by sending the trapped image to the terminal held by the user, and remotely control the trapped robot to escape, thereby effectively improving the success rate of the robot's escape.

[0181] The ninth aspect of the present invention provides a control device for a mobile robot, comprising: a memory for storing programs or instructions; a processor for implementing the steps of the control method for the mobile robot provided in any of the above-mentioned technical solutions when executing the programs or instructions, and therefore also including all its beneficial effects, which will not be described here to avoid repetition.

[0182] The tenth aspect of the present invention provides a readable storage medium having a program or instruction stored thereon. When the program or instruction is executed by a processor, the steps of the mobile robot control method provided in any of the above-mentioned technical solutions are implemented, and therefore all its beneficial effects are also included. To avoid repetition, they will not be described here.

[0183] The eleventh aspect of the present invention provides a mobile robot, including a control device of the mobile robot as provided in any of the above technical solutions; and / or a readable storage medium as provided in any of the above technical solutions, and therefore also includes all its beneficial effects, which will not be repeated here to avoid repetition. BRIEF DESCRIPTION OF THE DRAWINGS

[0184] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0185] Figure 1 A flowchart of a method for controlling a mobile robot according to an embodiment of the present invention is shown;

[0186] Figure 2 A schematic diagram showing a trapped image captured by an electronic device according to an embodiment of the present invention;

[0187] Figure 3 A schematic diagram of an interface of a target terminal according to an embodiment of the present invention is shown;

[0188] Figure 4 A second flowchart of a method for controlling a mobile robot according to an embodiment of the present invention is shown;

[0189] Figure 5 A third flowchart of the control method of the mobile robot according to an embodiment of the present invention is shown;

[0190] Figure 6 A fourth flowchart of a method for controlling a mobile robot according to an embodiment of the present invention is shown;

[0191] Figure 7 shows one of the structural block diagrams of the control device of the mobile robot according to an embodiment of the present invention;

[0192] Figure 8 A second structural block diagram of the control device for a mobile robot according to an embodiment of the present invention is shown;

[0193] Fig. 9 The third structural block diagram of the control device of the mobile robot according to the embodiment of the present invention is shown;

[0194] Fig.10 A fourth structural block diagram of the control device for a mobile robot according to an embodiment of the present invention is shown;

[0195] Fig.11 A fifth structural block diagram of the control device for a mobile robot according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0196] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0197] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0198] Refer to the following Figures 1 to 11 The control method, device, storage medium and mobile robot of the mobile robot according to some embodiments of the present invention are described.

[0199] In some embodiments of the present invention, a control method for a mobile robot is provided, which is applied to a server. Figure 1 FIG. 1 shows one of the flow charts of the control method of the mobile robot according to an embodiment of the present invention. Figure 1 As shown, the control method includes:

[0200] Step 102, obtaining a trapped signal of the trapped robot;

[0201] Step 104, determining an image sensor, the field of view of the image sensor covers the trapped robot or the image sensor is set on the trapped robot;

[0202] Step 106, acquiring an image from an image sensor;

[0203] Step 108, controlling the trapped robot to perform escape motion according to the image.

[0204] In the embodiment of the present invention, the robot may be a robot with mobility used in the home, such as a sweeping robot, a mopping robot or an inspection robot. When the robot performs a task, the robot may be blocked by an obstacle. In this case, the robot will first try to cross the obstacle or adjust the direction of travel to bypass the obstacle.

[0205] If the robot fails to cross or bypass an obstacle after multiple attempts, the robot determines that it has entered a trapped state, and then the robot sends a trapped signal to the server.

[0206] Exemplarily, the trapped signal carries information such as the trapped state, location information, and identification information of the first robot.

[0207] The server can be a cloud server, a robot base station, or a device such as an intelligent gateway that establishes a wireless network connection with the robot.

[0208] After receiving the trapped signal of the mobile robot, the server determines the image sensor, which may be an image sensor in the area where the trapped robot is located, or an image sensor arranged on the robot.

[0209] The server obtains the image captured by the image sensor, which can reflect the trapped situation of the trapped robot. Therefore, the trapped robot is controlled to escape according to the image captured by the image sensor, so that the trapped robot can perform targeted escape movements according to the trapped situation, specifically the location of the obstacle, the stuck part, etc.

[0210] Exemplarily, the image sensor is an image sensor in the area where the trapped robot is located. The image sensor in the area where the trapped robot is located is used to capture an image of the trapped robot from a third perspective. The trapped condition of the robot is determined using the trapped image from the third perspective, and the robot is controlled to perform an escape movement.

[0211] Exemplarily, the image sensor is an image sensor provided on the trapped robot, which collects the trapped condition of the robot from a first perspective through the image sensor on the robot, determines the trapped state of the robot based on the trapped image from the first perspective, and controls the robot to execute an escape movement.

[0212] The embodiment of the present invention can effectively improve the robot's escape efficiency by acquiring a trapped image of the robot through an image sensor when the robot is trapped and controlling the robot to escape based on the trapped image.

[0213] Based on any of the above embodiments, the step of determining the image sensor includes: determining a trapped area of ​​the trapped robot according to the trapped signal; determining at least one electronic device in the trapped area, wherein the at least one electronic device is provided with an image sensor;

[0214] The step of acquiring an image from an image sensor includes: controlling the electronic device to collect a trapped image corresponding to the trapped robot through the image sensor.

[0215] In an embodiment of the present invention, after the server receives a trapped signal sent by the trapped robot, the server determines the identity and location information of the trapped robot based on the information carried in the trapped signal, such as determining that the robot is a sweeping robot and the trapped area is the living room.

[0216] After determining the trapped area of ​​the trapped robot according to the trapped signal, the server searches for at least one electronic device in the trapped area, wherein the electronic device is an electronic device having an image sensor.

[0217] For example, the electronic device can be a home surveillance camera, or a home appliance with a camera, such as a smart TV, a somatosensory game console, a smart refrigerator, a smart air conditioner, etc. The electronic device can also be other robots in the home. For example, if the trapped robot is a sweeping robot, the electronic device can also be a mopping robot or a patrol robot that is not trapped.

[0218] The above electronic devices form an Internet of Things through an intelligent gateway. After determining the trapped area of ​​the robot, the server sends a query request to the home Internet of Things through a network connection with the intelligent gateway. After successfully authenticating the request to the server, the home Internet of Things sends a device list of electronic devices with cameras in the trapped area, such as the living room, to the server.

[0219] The server determines at least one electronic device in the trapped area that can capture the trapped situation of the robot based on the received device list, and sends an escape signal to these electronic devices via the Internet.

[0220] After receiving the escape signal, the electronic device in the trapped area first authenticates the server to determine whether the server has the authority to obtain image data. After successful authentication, the electronic device captures image information in the trapped area according to the escape signal, specifically capturing the trapped image of the trapped robot.

[0221] Figure 2 FIG. 4 is a schematic diagram showing a trapped image captured by an electronic device according to an embodiment of the present invention. Figure 2 As shown, in the trapped area 200 of the trapped robot 202 , there is an electronic device 204 , on which an image sensor 2042 is disposed. The electronic device 204 collects a trapped image of the trapped robot 202 through the image sensor 2042 .

[0222] Exemplarily, the escape signal carries the identification information and trapped position of the trapped robot, and the electronic device photographs the trapped position of the trapped robot according to the escape signal, thereby acquiring an image of the trapped robot.

[0223] After obtaining the trapped image of the robot, the obstacle information near the robot can be determined, such as the obstacle position, shape, height, etc. Through this obstacle information, the trapped robot can be helped to plan an escape route, thereby helping the trapped robot to perform escape actions.

[0224] It is understandable that in some specific scenarios, the robot may be blocked by obstacles, such as the robot being stuck under a sofa. At this time, the server can determine whether the image information captured by the electronic devices in the current trapped area includes the trapped robot. If at least one electronic device can capture the trapped robot, the trapped robot is helped out of the trapped area based on the image information captured of the trapped robot.

[0225] If the trapped robot is not included in the image information captured by all electronic devices in the trapped area, it is determined that the position of the trapped robot is blocked. At this time, the server controls the trapped robot to standby, and controls the electronic devices in the trapped area to stop capturing images, and sends a message to the user terminal that the robot is trapped and cannot escape by itself.

[0226] An embodiment of the present invention provides a collaborative escape method based on other electronic devices in the robot's working area. The electronic devices in the trapped area collect image information of the area where the trapped robot is located, that is, the robot's trapped image. The trapped image includes images of obstacles blocking the robot's movement, which can accurately reflect the robot's trapped situation. Therefore, by controlling the robot to execute escape movement through the trapped image, the robot's escape success rate can be effectively improved.

[0227] Based on any of the above embodiments, the step of determining the image sensor includes: when the trapped robot is equipped with an image acquisition device, determining the image acquisition device as the image sensor.

[0228] In the embodiment of the present invention, after the server receives the trapped signal sent by the trapped robot, the server determines whether an image sensor device is provided on the trapped robot.

[0229] The image sensor device may be a robot visual sensor, such as a camera, etc. The server may send a query command to the trapped robot, and after receiving the query command, the robot returns a corresponding query result to the server, informing the server whether the server itself is provided with an image sensor device.

[0230] In the case where the robot is provided with an image sensing device, the server determines the image sensing device provided on the robot as an image sensor, and collects the trapped image of the robot by calling the image sensor on the robot. After obtaining the trapped image of the robot, the obstacle information near the robot can be determined, such as the obstacle position, shape, height, etc. The obstacle information can help the trapped robot plan an escape route, thereby helping the trapped robot to perform an escape action.

[0231] It can be understood that the image acquisition device set on the robot can be determined as the only image sensor, or after the image acquisition device set on the robot is determined as the image sensor, at least one electronic device in the trapped area of ​​the robot can be further determined, and the image sensor set on the electronic device can be determined as the image sensor used to capture the trapped image.

[0232] Based on any of the above embodiments, the step of controlling the trapped robot to perform escape motion according to the image includes:

[0233] The image is sent to the target terminal; the remote control signal sent by the target terminal is received; the remote control signal is sent to the trapped robot, so that the trapped robot can perform an escape movement according to the remote control signal.

[0234] In the embodiment of the present invention, the target terminal is specifically a terminal held by a user, such as a mobile phone, a remote controller, a tablet computer, etc. of the user.

[0235] After receiving the trapped image of the trapped robot captured by the electronic device, the server sends the trapped image to the target terminal. It is understandable that the trapped image can be a real-time video stream. The server can compress the video stream file to reduce the demand for network bandwidth and reduce traffic consumption.

[0236] After receiving the trapped image, the target terminal displays the trapped image through the home device management application. The user can understand the specific situation of the robot being trapped through the trapped image, and according to the trapped image displayed by the target terminal, by inputting remote control commands, the remote control robot moves in the direction of avoiding obstacles, thereby achieving escape.

[0237] After the target terminal receives the remote control command input by the user, the target terminal generates a corresponding remote control signal according to the control command and sends the remote control signal to the server. After receiving the remote control signal, the server sends the remote control signal to the trapped robot, and the trapped robot performs the corresponding escape movement according to the remote control signal, such as the "forward" signal or the "backward" signal.

[0238] For example, Figure 3 FIG. 4 shows a schematic diagram of an interface of a target terminal according to an embodiment of the present invention. Figure 3 As shown, a trapped image 304 is displayed in the display interface 302 of the target terminal 300 .

[0239] The embodiment of the present invention controls the electronic equipment in the trapped area where the trapped robot is located to capture the trapped image of the trapped robot, and sends the trapped image to the target terminal held by the user, thereby helping the user to grasp the trapped situation of the trapped robot based on the trapped image and remotely control the trapped robot to escape, thereby effectively improving the success rate of the robot's escape.

[0240] Based on any of the above embodiments, the step of controlling the trapped robot to perform escape motion according to the image includes:

[0241] A corresponding escape map is generated according to the image; an escape route is determined according to the escape map; and the escape route is sent to the trapped robot so that the trapped robot can execute an escape movement according to the escape route.

[0242] In the embodiment of the present invention, the trapped image can be picture data or video stream data. After the server receives the trapped image of the trapped robot taken by the electronic device, the position of the trapped robot and the positions of obstacles near the trapped robot are identified in the trapped image through an image recognition algorithm.

[0243] The server generates an escape map near the escape robot through a mapping algorithm based on the identified robot position and obstacle position. The escape map records the position and posture of the trapped robot and also includes the location information of nearby obstacles.

[0244] Exemplarily, the escape map may be grid map information including obstacle information.

[0245] After the escape map is generated, an escape route for helping the trapped robot escape is generated through a route algorithm based on the robot position and obstacle positions recorded in the escape map.

[0246] The server sends the escape route to the trapped robot, and the trapped robot drives its own running mechanism, such as the running wheels, according to the movement mode indicated by the escape route, and performs the escape movement according to the escape route.

[0247] The embodiment of the present invention constructs an escape map near the trapped robot through the trapped image, and calculates an escape route suitable for the trapped robot according to the escape map, which can effectively improve the success rate of the robot's escape.

[0248] Based on any of the above embodiments, determining the trapped area of ​​the trapped robot according to the trapped signal includes:

[0249] Determine the position information of the robot according to the trapped signal; obtain the working map of the trapped robot; and determine the trapped area within the working area of ​​the trapped robot according to the position information and the working map.

[0250] In the embodiment of the present invention, the working map of the robot may be a map created by the robot's own sensors during the working process, or may be a preset map. The working map records the regional information of the robot's working area, and marks each sub-area in the working area based on preset annotation information.

[0251] Taking the robot's working area as the user's home interior as an example, the working map is a complete map of the interior, in which each sub-area is marked, such as the living room, bedroom, kitchen, etc.

[0252] After the robot is trapped, the robot sends a trapped signal to the server. The server determines the current area of ​​the robot, i.e., the trapped area of ​​the robot, based on the location information carried in the trapped signal and the robot's work map.

[0253] Taking the trapped area as the living room as an example, after the server determines the trapped area of ​​the trapped robot based on the working map and the location information carried in the trapped signal, it determines at least one electronic device with an image sensor in the trapped area, that is, the living room, and uses the electronic device to capture the trapped image of the trapped robot, and uses the trapped image to help the trapped robot escape.

[0254] The embodiment of the present invention determines the trapped area of ​​the trapped robot based on the robot's work map and the position information of the robot when it is trapped, and then controls the electronic equipment in the trapped area to obtain the trapped image of the trapped robot. The robot is controlled to escape through the trapped image, which can improve the success rate of the robot's escape.

[0255] Based on any of the above embodiments, determining at least one electronic device in the trapped area includes:

[0256] A query signal is sent to a target gateway, and the target gateway is in communication connection with a plurality of candidate electronic devices in the trapped area; and at least one electronic device is determined from the plurality of candidate electronic devices according to a device list received from the target gateway.

[0257] In an embodiment of the present invention, the target gateway may be a smart gateway in a user's home. Multiple smart home appliances, surveillance cameras, robots and other electronic devices in the user's home are all connected to the smart gateway for communication and access the Internet through the smart gateway.

[0258] When the server receives the trapped signal sent by the trapped robot, it identifies the target gateway in the Internet of Things environment where the trapped robot is located according to the identity information of the trapped robot carried in the trapped signal.

[0259] After determining the target gateway, the server sends a query signal to the target gateway, and the query signal carries the server's identity information and the information of the trapped area. After receiving the query signal, the target gateway authenticates the server according to the identity information. After the authentication is successful, the target gateway determines multiple candidate electronic devices located in the trapped area from the electronic devices connected to the target gateway according to the information of the trapped area.

[0260] The candidate electronic devices are all electronic devices connected to the target gateway, such as smart lamps, smart TVs, electric curtains, somatosensory game consoles, and inspection robots. The target gateway sends the device list of the above electronic devices to the server.

[0261] After receiving the device list, the server selects at least one electronic device with an image sensor from the device list, uses the at least one electronic device with an image sensor to capture a trapped image of the trapped robot, and uses the trapped image to help the robot escape.

[0262] The embodiment of the present invention obtains a device list of alternative electronic devices in the trapped area through a target gateway in the same home scene as the trapped robot, and then determines at least one electronic device equipped with an image sensor in the device list, and helps the trapped robot escape through the trapped image captured by the electronic device, thereby improving the success rate of the robot's escape.

[0263] In some embodiments of the present invention, a control method for a mobile robot is provided, which is applied to the robot. Figure 4 FIG. 2 shows a second flow chart of a control method for a mobile robot according to an embodiment of the present invention. Figure 4 As shown, the control method includes:

[0264] Step 402, when it is detected that the robot is in a trapped state, sending a trapped signal to the server;

[0265] Step 404, receiving a remote control signal sent by the target terminal;

[0266] The remote control signal is a signal generated by the target terminal according to the received remote control input when the trapped image is displayed, and the trapped image is an image captured by an image sensor, and the field of view of the image sensor covers the trapped robot or the image sensor is arranged on the trapped robot;

[0267] Step 406, executing an escape motion according to the remote control signal.

[0268] In the embodiment of the present invention, the robot may be a robot with mobility used in the home, such as a sweeping robot, a mopping robot or an inspection robot. When the robot performs a task, the robot may be blocked by an obstacle. In this case, the robot will first try to cross the obstacle or adjust the direction of travel to bypass the obstacle.

[0269] If the robot fails to cross or bypass an obstacle after multiple attempts, the robot determines that it has entered a trapped state, and then the robot sends a trapped signal to the server.

[0270] The server can be a cloud server, a robot base station, or a device such as an intelligent gateway that establishes a wireless network connection with the robot.

[0271] After the server receives the trapped signal sent by the trapped robot, the server determines the image sensor, which may be an image sensor in the area where the trapped robot is located, or an image sensor disposed on the robot.

[0272] The server obtains the image captured by the image sensor, which can reflect the trapped situation of the trapped robot. Therefore, the trapped robot is controlled to escape according to the image captured by the image sensor, so that the trapped robot can perform targeted escape movements according to the trapped situation, specifically the location of the obstacle, the stuck part, etc.

[0273] Exemplarily, the server determines the identity and location information of the trapped robot based on the information carried in the trapped signal, such as determining that the robot is a sweeping robot and the trapped area is the living room.

[0274] After determining the trapped area of ​​the trapped robot based on the trapped signal, the server searches for at least one electronic device in the trapped area, wherein the electronic device is an electronic device with an image sensor. The server sends an escape signal to the electronic device with the image sensor, and the escape signal carries the identification information and trapped position of the trapped robot. The electronic device photographs the trapped position of the trapped robot based on the escape signal, thereby acquiring a trapped image of the robot.

[0275] Exemplarily, the image sensor is an image sensor provided on the trapped robot, and the trapped condition of the robot from a first perspective is collected by the image sensor on the robot, and the trapped state of the robot is determined by the trapped image from the first perspective.

[0276] After receiving the trapped image, the server sends the trapped image to the target terminal, wherein the target terminal is specifically a terminal held by the user, such as the user's mobile phone, remote control, tablet computer, etc.

[0277] After receiving the trapped image, the target terminal displays the trapped image through the home device management application. The user can understand the specific situation of the robot being trapped through the trapped image, and according to the trapped image displayed by the target terminal, the user can input remote control commands to remotely control the robot to move in the direction of avoiding obstacles.

[0278] After the target terminal receives the remote control command input by the user, the target terminal generates a corresponding remote control signal according to the control command and sends the remote control signal to the server. After receiving the remote control signal, the server sends the remote control signal to the trapped robot, and the trapped robot performs the corresponding escape movement according to the remote control signal, such as the "forward" signal or the "backward" signal.

[0279] The embodiment of the present invention helps the user to understand the trapped situation of the trapped robot according to the trapped image by sending the trapped image to the target terminal held by the user, and remotely control the trapped robot to escape, thereby effectively improving the success rate of the robot's escape.

[0280] In some embodiments of the present invention, a control method for a mobile robot is provided, which is applied to the robot. Figure 5 FIG. 3 shows a flow chart of a control method for a mobile robot according to an embodiment of the present invention. Figure 5 As shown, the control method includes:

[0281] Step 502, when it is detected that the robot is in a trapped state, sending a trapped signal to the server;

[0282] Step 504, receiving the escape route sent by the server;

[0283] The escape route is a route determined by the server according to the trapped image, the trapped image is an image captured by an image sensor, the field of view of the image sensor covers the trapped robot or the image sensor is arranged on the trapped robot;

[0284] Step 506, executing escape motion according to the escape route.

[0285] In the embodiment of the present invention, the robot may be a robot with mobility used in the home, such as a sweeping robot, a mopping robot or an inspection robot. When the robot performs a task, the robot may be blocked by an obstacle. In this case, the robot will first try to cross the obstacle or adjust the direction of travel to bypass the obstacle.

[0286] If the robot fails to cross or bypass an obstacle after multiple attempts, the robot determines that it has entered a trapped state, and then the robot sends a trapped signal to the server.

[0287] The server can be a cloud server, a robot base station, or a device such as an intelligent gateway that establishes a wireless network connection with the robot.

[0288] After the server receives the trapped signal sent by the trapped robot, the server determines the image sensor, which may be an image sensor in the area where the trapped robot is located, or an image sensor disposed on the robot.

[0289] The server obtains the image captured by the image sensor, which can reflect the trapped situation of the trapped robot. Therefore, the trapped robot is controlled to escape according to the image captured by the image sensor, so that the trapped robot can perform targeted escape movements according to the trapped situation, specifically the location of the obstacle, the stuck part, etc.

[0290] Exemplarily, the server determines the identity and location information of the trapped robot based on the information carried in the trapped signal, such as determining that the robot is a sweeping robot and the trapped area is the living room.

[0291] After determining the trapped area of ​​the trapped robot based on the trapped signal, the server searches for at least one electronic device in the trapped area, wherein the electronic device is an electronic device with an image sensor. The server sends an escape signal to the electronic device with the image sensor, and the escape signal carries the identification information and trapped position of the trapped robot. The electronic device photographs the trapped position of the trapped robot based on the escape signal, thereby acquiring a trapped image of the robot.

[0292] Exemplarily, the image sensor is an image sensor provided on the trapped robot, and the trapped condition of the robot from a first perspective is collected by the image sensor on the robot, and the trapped state of the robot is determined by the trapped image from the first perspective.

[0293] After receiving the trapped image of the trapped robot taken by the electronic device, the server identifies the position of the trapped robot and the positions of obstacles near the trapped robot in the trapped image through an image recognition algorithm.

[0294] The server generates an escape map near the escape robot through a mapping algorithm based on the identified robot position and obstacle position. The escape map records the position and posture of the trapped robot and also includes the location information of nearby obstacles. In some embodiments, the escape map can be a map generated by updating the working map of the trapped robot based on the position and posture of the trapped robot and the location information of nearby obstacles.

[0295] Exemplarily, the escape map may be grid map information including obstacle information.

[0296] After the escape map is generated, an escape route for helping the trapped robot escape is generated through a route algorithm based on the robot position and obstacle positions recorded in the escape map.

[0297] The server sends the escape route to the trapped robot, and the trapped robot drives its own running mechanism, such as the running wheels, according to the movement mode indicated by the escape route, and performs the escape movement according to the escape route.

[0298] The embodiment of the present invention constructs an escape map near the trapped robot through the trapped image acquired by the electronic device in the trapped area, and calculates an escape route suitable for the trapped robot according to the escape map, which can effectively improve the success rate of the robot's escape.

[0299] In some embodiments of the present invention, a control method for a mobile robot is provided, which is applied to a terminal. Figure 6 FIG. 4 shows a flow chart of a control method for a mobile robot according to an embodiment of the present invention. Figure 6 As shown, the control method includes:

[0300] Step 602, receiving the trapped image sent by the server;

[0301] The trapped image is an image captured by an image sensor, the field of view of the image sensor covers the trapped robot or the image sensor is arranged on the trapped robot;

[0302] Step 604, displaying the trapped image and receiving remote control input;

[0303] Step 606, in response to the remote control input, determining a remote control signal;

[0304] Step 608, sending the remote control signal to the server, so that the server sends the remote control signal to the trapped robot, so that the trapped robot performs an escape movement according to the remote control signal.

[0305] In the embodiment of the present invention, the robot may be a robot with mobility used in the home, such as a sweeping robot, a mopping robot or an inspection robot. When the robot performs a task, the robot may be blocked by an obstacle. In this case, the robot will first try to cross the obstacle or adjust the direction of travel to bypass the obstacle.

[0306] If the robot fails to cross or bypass an obstacle after multiple attempts, the robot determines that it has entered a trapped state, and the robot sends a trapped signal to the server. When the server receives the trapped signal sent by the trapped robot, the server determines the image sensor, which can be an image sensor in the area where the trapped robot is located, or an image sensor set on the robot.

[0307] The server obtains the image captured by the image sensor, which can reflect the trapped situation of the trapped robot. Therefore, the trapped robot is controlled to escape according to the image captured by the image sensor, so that the trapped robot can perform targeted escape movements according to the trapped situation, specifically the location of the obstacle, the stuck part, etc.

[0308] Exemplarily, the server determines the identity and location information of the trapped robot based on the information carried in the trapped signal, such as determining that the robot is a sweeping robot and the trapped area is the living room.

[0309] At least one electronic device is found in the trapped area. The electronic device may be a home surveillance camera or a home appliance with a camera, such as a smart TV, a somatosensory game console, a smart refrigerator, a smart air conditioner, etc. The electronic device may also be other robots in the home. For example, if the trapped robot is a sweeping robot, the electronic device may also be a mopping robot or a patrol robot that is not trapped.

[0310] The electronic device photographs the trapped position of the trapped robot according to the escape signal, thereby collecting the trapped image of the robot, and sends the trapped image to the server.

[0311] Exemplarily, the image sensor is an image sensor provided on the trapped robot, and the trapped condition of the robot from a first perspective is collected by the image sensor on the robot, and the trapped state of the robot is determined by the trapped image from the first perspective.

[0312] After receiving the trapped image, the terminal displays the trapped image through the home device management application. The user can understand the specific situation of the robot being trapped through the trapped image, and according to the trapped image displayed by the terminal, by inputting remote control commands, the remote control robot moves in the direction of avoiding obstacles, thereby achieving escape. After the server receives the trapped image of the trapped robot taken by the electronic device, the trapped image is sent to the terminal. It can be understood that the trapped image can be a real-time video stream. The server can compress the video stream file to reduce the demand for network bandwidth and reduce traffic consumption.

[0313] After the terminal receives the remote control command input by the user, the terminal generates a corresponding remote control signal according to the control command and sends the remote control signal to the server. After receiving the remote control signal, the server sends the remote control signal to the trapped robot, and the trapped robot performs the corresponding escape movement according to the remote control signal, such as the "forward" signal or the "backward" signal.

[0314] The embodiment of the present invention helps the user to understand the trapped situation of the trapped robot according to the trapped image by sending the trapped image to the terminal held by the user, and remotely control the trapped robot to escape, thereby effectively improving the success rate of the robot's escape.

[0315] In some embodiments of the present invention, a control device for a mobile robot is provided, which is applied to a server. Figure 7 FIG. 1 shows one of the structural block diagrams of the control device of the mobile robot according to an embodiment of the present invention. Figure 7 As shown, the control device 700 of the mobile robot includes:

[0316] An acquisition module 702 is used to acquire a trapped signal of the trapped robot;

[0317] A determination module 704 is used to determine an image sensor, the field of view of the image sensor covers the trapped robot or the image sensor is arranged on the trapped robot;

[0318] The acquisition module 702 is also used to acquire an image from the image sensor;

[0319] The control module 706 is used to control the trapped robot to perform escape motion according to the image.

[0320] In the embodiment of the present invention, the robot may be a robot with mobility used in the home, such as a sweeping robot, a mopping robot or an inspection robot. When the robot performs a task, the robot may be blocked by an obstacle. In this case, the robot will first try to cross the obstacle or adjust the direction of travel to bypass the obstacle.

[0321] If the robot fails to cross or bypass an obstacle after multiple attempts, the robot determines that it has entered a trapped state, and then the robot sends a trapped signal to the server.

[0322] Exemplarily, the trapped signal carries information such as the trapped state, location information, and identification information of the first robot.

[0323] The server can be a cloud server, a robot base station, or a device such as an intelligent gateway that establishes a wireless network connection with the robot.

[0324] After receiving the trapped signal of the mobile robot, the server determines the image sensor, which may be an image sensor in the area where the trapped robot is located, or an image sensor arranged on the robot.

[0325] The server obtains the image captured by the image sensor, which can reflect the trapped situation of the trapped robot. Therefore, the trapped robot is controlled to escape according to the image captured by the image sensor, so that the trapped robot can perform targeted escape movements according to the trapped situation, specifically the location of the obstacle, the stuck part, etc.

[0326] Exemplarily, the image sensor is an image sensor in the area where the trapped robot is located. The image sensor in the area where the trapped robot is located is used to capture an image of the trapped robot from a third perspective. The trapped condition of the robot is determined using the trapped image from the third perspective, and the robot is controlled to perform an escape movement.

[0327] Exemplarily, the image sensor is an image sensor provided on the trapped robot, which collects the trapped condition of the robot from a first perspective through the image sensor on the robot, determines the trapped state of the robot based on the trapped image from the first perspective, and controls the robot to execute an escape movement.

[0328] The embodiment of the present invention can effectively improve the robot's escape efficiency by acquiring a trapped image of the robot through an image sensor when the robot is trapped and controlling the robot to escape based on the trapped image.

[0329] Based on any of the above embodiments, the control module is further used to control the electronic device to collect a trapped image corresponding to the trapped robot through the image sensor.

[0330] In an embodiment of the present invention,

[0331] When the server receives the trapped signal sent by the trapped robot, the server determines the identity and location information of the trapped robot according to the information carried in the trapped signal, such as determining that the robot is a sweeping robot and the trapped area is the living room.

[0332] After determining the trapped area of ​​the trapped robot according to the trapped signal, the server searches for at least one electronic device in the trapped area, wherein the electronic device is an electronic device having an image sensor.

[0333] For example, the electronic device can be a home surveillance camera, or a home appliance with a camera, such as a smart TV, a somatosensory game console, a smart refrigerator, a smart air conditioner, etc. The electronic device can also be other robots in the home. For example, if the trapped robot is a sweeping robot, the electronic device can also be a mopping robot or a patrol robot that is not trapped.

[0334] The above electronic devices form an Internet of Things through an intelligent gateway. After determining the trapped area of ​​the robot, the server sends a query request to the home Internet of Things through a network connection with the intelligent gateway. After successfully authenticating the request to the server, the home Internet of Things sends a device list of electronic devices with cameras in the trapped area, such as the living room, to the server.

[0335] The server determines at least one electronic device in the trapped area that can capture the trapped situation of the robot based on the received device list, and sends an escape signal to these electronic devices via the Internet.

[0336] After receiving the escape signal, the electronic device in the trapped area first authenticates the server to determine whether the server has the authority to obtain image data. After successful authentication, the electronic device captures image information in the trapped area according to the escape signal, specifically capturing the trapped image of the trapped robot.

[0337] Figure 2 FIG. 4 is a schematic diagram showing a trapped image captured by an electronic device according to an embodiment of the present invention. Figure 2 As shown, in the trapped area 200 of the trapped robot 202 , there is an electronic device 204 , on which an image sensor 2042 is disposed. The electronic device 204 collects a trapped image of the trapped robot 202 through the image sensor 2042 .

[0338] Exemplarily, the escape signal carries the identification information and trapped position of the trapped robot, and the electronic device photographs the trapped position of the trapped robot according to the escape signal, thereby acquiring an image of the trapped robot.

[0339] After obtaining the trapped image of the robot, the obstacle information near the robot can be determined, such as the obstacle position, shape, height, etc. Through this obstacle information, the trapped robot can be helped to plan an escape route, thereby helping the trapped robot to perform escape actions.

[0340] It is understandable that in some specific scenarios, the robot may be blocked by obstacles, such as the robot being stuck under a sofa. At this time, the server can determine whether the image information captured by the electronic devices in the current trapped area includes the trapped robot. If at least one electronic device can capture the trapped robot, the trapped robot is helped out of the trapped area based on the image information captured of the trapped robot.

[0341] If the trapped robot is not included in the image information captured by all electronic devices in the trapped area, it is determined that the position of the trapped robot is blocked. At this time, the server controls the trapped robot to standby, and controls the electronic devices in the trapped area to stop capturing images, and sends a message to the user terminal that the robot is trapped and cannot escape by itself.

[0342] An embodiment of the present invention provides a collaborative escape method based on other electronic devices in the robot's working area. The electronic devices in the trapped area collect image information of the area where the trapped robot is located, that is, the robot's trapped image. The trapped image includes images of obstacles blocking the robot's movement, which can accurately reflect the robot's trapped situation. Therefore, by controlling the robot to execute escape movement through the trapped image, the robot's escape success rate can be effectively improved.

[0343] Based on any of the above embodiments, the control device further includes:

[0344] A sending module, used for sending the image to a target terminal;

[0345] The control device also includes a receiving module for receiving a remote control signal sent by the target terminal;

[0346] The first sending module is also used to send a remote control signal to the trapped robot, so that the trapped robot can perform an escape movement according to the remote control signal.

[0347] In the embodiment of the present invention, the target terminal is specifically a terminal held by a user, such as a mobile phone, a remote controller, a tablet computer, etc. of the user.

[0348] After receiving the trapped image of the trapped robot captured by the electronic device, the server sends the trapped image to the target terminal. It is understandable that the trapped image can be a real-time video stream. The server can compress the video stream file to reduce the demand for network bandwidth and reduce traffic consumption.

[0349] After receiving the trapped image, the target terminal displays the trapped image through the home device management application. The user can understand the specific situation of the robot being trapped through the trapped image, and according to the trapped image displayed by the target terminal, by inputting remote control commands, the remote control robot moves in the direction of avoiding obstacles, thereby achieving escape.

[0350] After the target terminal receives the remote control command input by the user, the target terminal generates a corresponding remote control signal according to the control command and sends the remote control signal to the server. After receiving the remote control signal, the server sends the remote control signal to the trapped robot, and the trapped robot performs the corresponding escape movement according to the remote control signal, such as the "forward" signal or the "backward" signal.

[0351] The embodiment of the present invention controls the electronic equipment in the trapped area where the trapped robot is located to capture the trapped image of the trapped robot, and sends the trapped image to the target terminal held by the user, thereby helping the user to grasp the trapped situation of the trapped robot based on the trapped image and remotely control the trapped robot to escape, thereby effectively improving the success rate of the robot's escape.

[0352] Based on any of the above embodiments, the control device further includes:

[0353] A generation module, used to generate a corresponding escape map according to the image;

[0354] The determination module is also used to determine an escape route based on the escape map;

[0355] The sending module is also used to send the escape route to the trapped robot, so that the trapped robot can execute the escape movement according to the escape route.

[0356] In the embodiment of the present invention, the trapped image can be picture data or video stream data. After the server receives the trapped image of the trapped robot taken by the electronic device, the position of the trapped robot and the positions of obstacles near the trapped robot are identified in the trapped image through an image recognition algorithm.

[0357] The server generates an escape map near the escape robot through a mapping algorithm based on the identified robot position and obstacle position. The escape map records the position and posture of the trapped robot and also includes the location information of nearby obstacles.

[0358] Exemplarily, the escape map may be grid map information including obstacle information.

[0359] After the escape map is generated, an escape route for helping the trapped robot escape is generated through a route algorithm based on the robot position and obstacle positions recorded in the escape map.

[0360] The server sends the escape route to the trapped robot, and the trapped robot drives its own running mechanism, such as the running wheels, according to the movement mode indicated by the escape route, and performs the escape movement according to the escape route.

[0361] The embodiment of the present invention constructs an escape map near the trapped robot through the trapped image, and calculates an escape route suitable for the trapped robot according to the escape map, which can effectively improve the success rate of the robot's escape.

[0362] Based on any of the above embodiments, the determination module is further used to determine the position information of the robot according to the trapped signal;

[0363] The control device also includes an acquisition module for acquiring a working map of the trapped robot;

[0364] The first determination module is further used to determine the trapped area within the working area of ​​the trapped robot according to the position information and the working map.

[0365] In the embodiment of the present invention, the working map of the robot may be a map created by the robot's own sensors during the working process, or may be a preset map. The working map records the regional information of the robot's working area, and marks each sub-area in the working area based on preset annotation information.

[0366] Taking the robot's working area as the user's home interior as an example, the working map is a complete map of the interior, in which each sub-area is marked, such as the living room, bedroom, kitchen, etc.

[0367] After the robot is trapped, the robot sends a trapped signal to the server. The server determines the current area of ​​the robot, i.e., the trapped area of ​​the robot, based on the location information carried in the trapped signal and the robot's work map.

[0368] Taking the trapped area as the living room as an example, after the server determines the trapped area of ​​the trapped robot based on the working map and the location information carried in the trapped signal, it determines at least one electronic device with an image sensor in the trapped area, that is, the living room, and uses the electronic device to capture the trapped image of the trapped robot, and uses the trapped image to help the trapped robot escape.

[0369] The embodiment of the present invention determines the trapped area of ​​the trapped robot based on the robot's work map and the position information of the robot when it is trapped, and then controls the electronic equipment in the trapped area to obtain the trapped image of the trapped robot. The robot is controlled to escape through the trapped image, which can improve the success rate of the robot's escape.

[0370] On the basis of any of the above embodiments, the first sending module is further used to send a query signal to a target gateway, and the target gateway is communicatively connected with a plurality of candidate electronic devices in the trapped area;

[0371] The first determination module is further configured to determine at least one electronic device from a plurality of candidate electronic devices according to the received device list sent by the target gateway.

[0372] In an embodiment of the present invention, the target gateway may be a smart gateway in a user's home. Multiple smart home appliances, surveillance cameras, robots and other electronic devices in the user's home are all connected to the smart gateway for communication and access the Internet through the smart gateway.

[0373] When the server receives the trapped signal sent by the trapped robot, it identifies the target gateway in the Internet of Things environment where the trapped robot is located according to the identity information of the trapped robot carried in the trapped signal.

[0374] After determining the target gateway, the server sends a query signal to the target gateway, and the query signal carries the server's identity information and the information of the trapped area. After receiving the query signal, the target gateway authenticates the server according to the identity information. After the authentication is successful, the target gateway determines multiple candidate electronic devices located in the trapped area from the electronic devices connected to the target gateway according to the information of the trapped area.

[0375] The candidate electronic devices are all electronic devices connected to the target gateway, such as smart lamps, smart TVs, electric curtains, somatosensory game consoles, and inspection robots. The target gateway sends the device list of the above electronic devices to the server.

[0376] After receiving the device list, the server selects at least one electronic device with an image sensor from the device list, uses the at least one electronic device with an image sensor to capture a trapped image of the trapped robot, and uses the trapped image to help the robot escape.

[0377] The embodiment of the present invention obtains a device list of alternative electronic devices in the trapped area through a target gateway in the same home scene as the trapped robot, and then determines at least one electronic device equipped with an image sensor in the device list, and helps the trapped robot escape through the trapped image captured by the electronic device, thereby improving the success rate of the robot's escape.

[0378] In some embodiments of the present invention, a control device for a mobile robot is provided, which is applied to the robot. Figure 8 FIG. 2 shows a second structural block diagram of a control device for a mobile robot according to an embodiment of the present invention. Figure 8 As shown, the control device 800 of the mobile robot includes:

[0379] The sending module 802 is used to send a trapped signal to the server when detecting that the robot is in a trapped state;

[0380] The receiving module 804 is used to receive a remote control signal sent by the target terminal, where the remote control signal is a signal generated by the target terminal according to the received remote control input when displaying the trapped image, and the trapped image is an image captured by an image sensor, and the field of view of the image sensor covers the trapped robot or the image sensor is set on the trapped robot;

[0381] The execution module 806 is used to execute the escape movement according to the remote control signal.

[0382] In the embodiment of the present invention, the robot may be a robot with mobility used in the home, such as a sweeping robot, a mopping robot or an inspection robot. When the robot performs a task, the robot may be blocked by an obstacle. In this case, the robot will first try to cross the obstacle or adjust the direction of travel to bypass the obstacle.

[0383] If the robot fails to cross or bypass an obstacle after multiple attempts, the robot determines that it has entered a trapped state, and then the robot sends a trapped signal to the server.

[0384] The server can be a cloud server, a robot base station, or a device such as an intelligent gateway that establishes a wireless network connection with the robot.

[0385] After the server receives the trapped signal sent by the trapped robot, the server determines the image sensor, which may be an image sensor in the area where the trapped robot is located, or an image sensor disposed on the robot.

[0386] The server obtains the image captured by the image sensor, which can reflect the trapped situation of the trapped robot. Therefore, the trapped robot is controlled to escape according to the image captured by the image sensor, so that the trapped robot can perform targeted escape movements according to the trapped situation, specifically the location of the obstacle, the stuck part, etc.

[0387] Exemplarily, the server determines the identity and location information of the trapped robot based on the information carried in the trapped signal, such as determining that the robot is a sweeping robot and the trapped area is the living room.

[0388] After determining the trapped area of ​​the trapped robot based on the trapped signal, the server searches for at least one electronic device in the trapped area, wherein the electronic device is an electronic device with an image sensor. The server sends an escape signal to the electronic device with the image sensor, and the escape signal carries the identification information and trapped position of the trapped robot. The electronic device photographs the trapped position of the trapped robot based on the escape signal, thereby acquiring a trapped image of the robot.

[0389] Exemplarily, the image sensor is an image sensor provided on the trapped robot, and the trapped condition of the robot from a first perspective is collected by the image sensor on the robot, and the trapped state of the robot is determined by the trapped image from the first perspective.

[0390] After receiving the trapped image, the server sends the trapped image to the target terminal, wherein the target terminal is specifically a terminal held by the user, such as the user's mobile phone, remote control, tablet computer, etc.

[0391] After receiving the trapped image, the target terminal displays the trapped image through the home device management application. The user can understand the specific situation of the robot being trapped through the trapped image, and according to the trapped image displayed by the target terminal, the user can input remote control commands to remotely control the robot to move in the direction of avoiding obstacles.

[0392] After the target terminal receives the remote control command input by the user, the target terminal generates a corresponding remote control signal according to the control command and sends the remote control signal to the server. After receiving the remote control signal, the server sends the remote control signal to the trapped robot, and the trapped robot performs the corresponding escape movement according to the remote control signal, such as the "forward" signal or the "backward" signal.

[0393] The embodiment of the present invention helps the user to understand the trapped situation of the trapped robot according to the trapped image by sending the trapped image to the target terminal held by the user, and remotely control the trapped robot to escape, thereby effectively improving the success rate of the robot's escape.

[0394] In some embodiments of the present invention, a control device for a mobile robot is provided, which is applied to the robot. Fig. 9FIG. 3 shows a third structural block diagram of a control device for a mobile robot according to an embodiment of the present invention. Fig. 9 As shown, the control device 900 of the mobile robot includes:

[0395] The sending module 902 is used to send a trapped signal to the server when detecting that the robot is in a trapped state;

[0396] The receiving module 904 is used to receive the escape route sent by the server, where the escape route is a route determined by the server according to the trapped image, where the trapped image is an image captured by an image sensor, where the field of view of the image sensor covers the trapped robot or the image sensor is set on the trapped robot;

[0397] The execution module 906 is used to execute the escape movement according to the escape route.

[0398] In the embodiment of the present invention, the robot may be a robot with mobility used in the home, such as a sweeping robot, a mopping robot or an inspection robot. When the robot performs a task, the robot may be blocked by an obstacle. In this case, the robot will first try to cross the obstacle or adjust the direction of travel to bypass the obstacle.

[0399] If the robot fails to cross or bypass an obstacle after multiple attempts, the robot determines that it has entered a trapped state, and then the robot sends a trapped signal to the server.

[0400] The server can be a cloud server, a robot base station, or a device such as an intelligent gateway that establishes a wireless network connection with the robot.

[0401] After the server receives the trapped signal sent by the trapped robot, the server determines the image sensor, which may be an image sensor in the area where the trapped robot is located, or an image sensor disposed on the robot.

[0402] The server obtains the image captured by the image sensor, which can reflect the trapped situation of the trapped robot. Therefore, the trapped robot is controlled to escape according to the image captured by the image sensor, so that the trapped robot can perform targeted escape movements according to the trapped situation, specifically the location of the obstacle, the stuck part, etc.

[0403] Exemplarily, the server determines the identity and location information of the trapped robot based on the information carried in the trapped signal, such as determining that the robot is a sweeping robot and the trapped area is the living room.

[0404] After determining the trapped area of ​​the trapped robot based on the trapped signal, the server searches for at least one electronic device in the trapped area, wherein the electronic device is an electronic device with an image sensor. The server sends an escape signal to the electronic device with the image sensor, and the escape signal carries the identification information and trapped position of the trapped robot. The electronic device photographs the trapped position of the trapped robot based on the escape signal, thereby acquiring a trapped image of the robot.

[0405] Exemplarily, the image sensor is an image sensor provided on the trapped robot, and the trapped condition of the robot from a first perspective is collected by the image sensor on the robot, and the trapped state of the robot is determined by the trapped image from the first perspective.

[0406] After receiving the trapped image of the trapped robot taken by the electronic device, the server identifies the position of the trapped robot and the positions of obstacles near the trapped robot in the trapped image through an image recognition algorithm.

[0407] The server generates an escape map near the escape robot through a mapping algorithm based on the identified robot position and obstacle position. The escape map records the position and posture of the trapped robot and also includes the location information of nearby obstacles.

[0408] Exemplarily, the escape map may be grid map information including obstacle information.

[0409] After the escape map is generated, an escape route for helping the trapped robot escape is generated through a route algorithm based on the robot position and obstacle positions recorded in the escape map.

[0410] The server sends the escape route to the trapped robot, and the trapped robot drives its own running mechanism, such as the running wheels, according to the movement mode indicated by the escape route, and performs the escape movement according to the escape route.

[0411] The embodiment of the present invention constructs an escape map near the trapped robot through the trapped image acquired by the electronic device in the trapped area, and calculates an escape route suitable for the trapped robot according to the escape map, which can effectively improve the success rate of the robot's escape.

[0412] In some embodiments of the present invention, a control device for a mobile robot is provided, which is applied to a terminal. Fig.10 FIG. 4 shows a fourth structural block diagram of a control device for a mobile robot according to an embodiment of the present invention. Fig.10 As shown, the control device 1000 of the mobile robot includes:

[0413] The receiving module 1002 is used to receive the trapped image sent by the server, where the trapped image is an image captured by an image sensor, the field of view of the image sensor covers the trapped robot or the image sensor is arranged on the trapped robot;

[0414] The display module 1004 displays the trapped image and receives the remote control input;

[0415] The determination module 1006 determines the remote control signal in response to the remote control input;

[0416] The sending module 1008 is used to send the remote control signal to the server, so that the server sends the remote control signal to the trapped robot, so that the trapped robot performs the escape movement according to the remote control signal.

[0417] In the embodiment of the present invention, the robot may be a robot with mobility used in the home, such as a sweeping robot, a mopping robot or an inspection robot. When the robot performs a task, the robot may be blocked by an obstacle. In this case, the robot will first try to cross the obstacle or adjust the direction of travel to bypass the obstacle.

[0418] If the robot fails to cross or bypass an obstacle after multiple attempts, the robot determines that it has entered a trapped state, and the robot sends a trapped signal to the server. When the server receives the trapped signal sent by the trapped robot, the server determines the image sensor, which can be an image sensor in the area where the trapped robot is located, or an image sensor set on the robot.

[0419] The server obtains the image captured by the image sensor, which can reflect the trapped situation of the trapped robot. Therefore, the trapped robot is controlled to escape according to the image captured by the image sensor, so that the trapped robot can perform targeted escape movements according to the trapped situation, specifically the location of the obstacle, the stuck part, etc.

[0420] Exemplarily, the server determines the identity and location information of the trapped robot based on the information carried in the trapped signal, such as determining that the robot is a sweeping robot and the trapped area is the living room.

[0421] At least one electronic device is found in the trapped area. The electronic device may be a home surveillance camera or a home appliance with a camera, such as a smart TV, a somatosensory game console, a smart refrigerator, a smart air conditioner, etc. The electronic device may also be other robots in the home. For example, if the trapped robot is a sweeping robot, the electronic device may also be a mopping robot or a patrol robot that is not trapped.

[0422] The electronic device photographs the trapped position of the trapped robot according to the escape signal, thereby collecting the trapped image of the robot, and sends the trapped image to the server.

[0423] Exemplarily, the image sensor is an image sensor provided on the trapped robot, and the trapped condition of the robot from a first perspective is collected by the image sensor on the robot, and the trapped state of the robot is determined by the trapped image from the first perspective.

[0424] After receiving the trapped image, the terminal displays the trapped image through the home device management application. The user can understand the specific situation of the robot being trapped through the trapped image, and according to the trapped image displayed by the terminal, by inputting remote control commands, the remote control robot moves in the direction of avoiding obstacles, thereby achieving escape. After the server receives the trapped image of the trapped robot taken by the electronic device, the trapped image is sent to the terminal. It can be understood that the trapped image can be a real-time video stream. The server can compress the video stream file to reduce the demand for network bandwidth and reduce traffic consumption.

[0425] After the terminal receives the remote control command input by the user, the terminal generates a corresponding remote control signal according to the control command and sends the remote control signal to the server. After receiving the remote control signal, the server sends the remote control signal to the trapped robot, and the trapped robot performs the corresponding escape movement according to the remote control signal, such as the "forward" signal or the "backward" signal.

[0426] The embodiment of the present invention helps the user to understand the trapped situation of the trapped robot according to the trapped image by sending the trapped image to the terminal held by the user, and remotely control the trapped robot to escape, thereby effectively improving the success rate of the robot's escape.

[0427] In some embodiments of the present invention, a control device for a mobile robot is provided. Fig.11 FIG. 5 shows a fifth structural block diagram of a control device for a mobile robot according to an embodiment of the present invention. Fig.11 As shown, the control device 1100 of the mobile robot includes: a memory 1102, which is used to store programs or instructions; a processor 1104, which is used to implement the steps of the mobile robot control method provided in any of the above embodiments when executing the programs or instructions, and therefore also includes all its beneficial effects, which will not be repeated here to avoid repetition.

[0428] In some embodiments of the present invention, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the mobile robot control method provided in any of the above embodiments are implemented, and therefore all its beneficial effects are also included. To avoid repetition, they will not be repeated here.

[0429] In some embodiments of the present invention, a mobile robot is provided, including a control device of the mobile robot as provided in any of the above embodiments; and / or a readable storage medium as provided in any of the above embodiments, and therefore also including all of its beneficial effects, which will not be described here to avoid repetition.

[0430] In the description of the present invention, the term "plurality" refers to two or more than two. Unless otherwise clearly defined, the orientation or positional relationship indicated by the terms "upper" and "lower" is based on the orientation or positional relationship described in the accompanying drawings. It is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention. The terms "connection", "installation", "fixation", etc. should be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0431] In the description of the present invention, the description of the terms "one embodiment", "some embodiments", "specific embodiments", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In the present invention, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0432] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A control method for a mobile robot, applied to a server, characterized in that: include: Obtaining the trapped signal of the trapped robot; Determine an image sensor, the field of view of the image sensor covers the trapped robot or the image sensor is arranged on the trapped robot; acquiring an image from the image sensor; and The trapped robot is controlled to perform an escape movement according to the image.

2. The control method according to claim 1, characterized in that: The step of determining the image sensor comprises: Determine a trapped area of ​​the trapped robot according to the trapped signal; At least one electronic device is determined in the trapped area, wherein the at least one electronic device is provided with the image sensor; and the step of acquiring the image of the image sensor comprises: The electronic device is controlled to collect a trapped image corresponding to the trapped robot through the image sensor.

3. The control method according to claim 1, characterized in that: The step of determining the image sensor comprises: In the case that the trapped robot is equipped with an image acquisition device, the image acquisition device is determined to be the image sensor.

4. The control method according to claim 1, characterized in that: The step of controlling the trapped robot to perform escape motion according to the image comprises: sending the image to a target terminal; Receiving a remote control signal sent by the target terminal; The remote control signal is sent to the trapped robot so that the trapped robot performs the escape movement according to the remote control signal.

5. The control method according to claim 1, characterized in that: The controlling the trapped robot to perform an escape movement according to the image comprises: generating a corresponding escape map according to the image; Determining an escape route according to the escape map; The escape route is sent to the trapped robot so that the trapped robot can execute the escape movement according to the escape route.

6. The control method according to claim 2, characterized in that: The step of determining the trapped area of ​​the trapped robot according to the trapped signal comprises: Determine the position information of the robot according to the trapped signal; Obtaining a working map of the trapped robot; The trapped area is determined within the working area of ​​the trapped robot according to the position information and the working map.

7. The control method according to claim 2, characterized in that: The step of determining at least one electronic device in the trapped area includes: sending a query signal to a target gateway, the target gateway being communicatively connected to a plurality of candidate electronic devices in the trapped area; The at least one electronic device is determined from the plurality of candidate electronic devices according to the received device list sent by the target gateway.

8. A control method for a mobile robot, applied to the robot, characterized in that: include: When detecting that the robot is in a trapped state, sending a trapped signal to a server; Receiving a remote control signal sent by a target terminal, wherein the remote control signal is a signal generated by the target terminal according to a received remote control input when displaying a trapped image, wherein the trapped image is an image captured by an image sensor, and the field of view of the image sensor covers the trapped robot or the image sensor is disposed on the trapped robot; The escape movement is performed according to the remote control signal.

9. A control method for a mobile robot, applied to the robot, characterized in that: include: When detecting that the robot is in a trapped state, sending a trapped signal to a server; Receiving an escape route sent by the server, wherein the escape route is a route determined by the server according to the trapped image, wherein the trapped image is an image captured by an image sensor, wherein the field of view of the image sensor covers the trapped robot or the image sensor is disposed on the trapped robot; The escape movement is performed according to the escape route.

10. A control method for a mobile robot, applied to a terminal, characterized in that: include: Receiving a trapped image sent by a server, wherein the trapped image is an image captured by an image sensor, the field of view of the image sensor covers the trapped robot or the image sensor is disposed on the trapped robot; displaying the trapped image and receiving remote control input; In response to the remote control input, determining a remote control signal; The remote control signal is sent to the server, so that the server sends the remote control signal to the trapped robot, so that the trapped robot performs an escape movement according to the remote control signal.

11. A control device for a mobile robot, applied to a server, characterized in that: include: An acquisition module, used for acquiring a trapped signal of the trapped robot; A determination module, used to determine an image sensor, the field of view of the image sensor covers the trapped robot or the image sensor is arranged on the trapped robot; The acquisition module is further used to acquire the image of the image sensor; The control module is used to control the trapped robot to perform escape movement according to the image.

12. A control device for a mobile robot, applied to the robot, characterized in that: include: A sending module, used for sending a trapped signal to a server when detecting that the robot is in a trapped state; A receiving module, configured to receive a remote control signal sent by a target terminal, wherein the remote control signal is a signal generated by the target terminal according to a received remote control input when displaying a trapped image, wherein the trapped image is an image captured by an image sensor, and the field of view of the image sensor covers the trapped robot or the image sensor is disposed on the trapped robot; An execution module is used to execute the escape movement according to the remote control signal.

13. A control device for a mobile robot, applied to the robot, characterized in that: include: A sending module, used for sending a trapped signal to a server when detecting that the robot is in a trapped state; A receiving module, used for receiving an escape route sent by the server, wherein the escape route is a route determined by the server according to the trapped image, wherein the trapped image is an image captured by an image sensor, and the field of view of the image sensor covers the trapped robot or the image sensor is arranged on the trapped robot; An execution module is used to execute an escape motion according to the escape route.

14. A control device for a mobile robot, applied to a terminal, characterized in that: include: A receiving module, used for receiving a trapped image sent by a server, wherein the trapped image is an image captured by an image sensor, the field of view of the image sensor covers the trapped robot or the image sensor is arranged on the trapped robot; A display module, used to display the trapped image and receive remote control input; a determination module, configured to determine a remote control signal in response to the remote control input; The sending module is used to send the remote control signal to the server, so that the server sends the remote control signal to the trapped robot, so that the trapped robot performs the escape movement according to the remote control signal.

15. A control device for a mobile robot, characterized in that: include: Memory, used to store programs or instructions; A processor, configured to implement the steps of the mobile robot control method as described in any one of claims 1 to 10 when executing the program or instruction.

16. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by a processor, the steps of the method for controlling a mobile robot according to any one of claims 1 to 10 are implemented.

17. A mobile robot, characterized in that: include: A control device for a mobile robot as claimed in any one of claims 11 to 15; and / or The readable storage medium of claim 16.