Robot control method and device, robot, program product and medium

Abstract

The invention provides a robot control method and device and the like. According to the method, after the robot receives a first instruction used for indicating automatic object obtaining or automatic object placing, namely, the robot is in an automatic object taking mode or an automatic object placing mode, and before a mechanical arm is controlled to obtain or place the object, the robot is driven to move to the automatic object taking mode or the automatic object placing mode. The robot is firstly controlled to execute at least one of movement actions such as advancing, retreating and rotating of the robot body, posture adjustment of the mechanical arm and the like, so that the robot has a wider visual field, and more complete information of a front object or a front environment is obtained; therefore, the recognition accuracy of the front object or the front environment can be improved (for example, for the scene of obtaining the object, the more accurate object position, object type and the like can be determined, and for the scene of placing the object, whether a storage container or an obstacle exists in the front environment or not can be accurately recognized). Therefore, the object can be better acquired or placed subsequently.

Description

Technical Field

[0001] The present application relates to robotics technology, and is related to, but not limited to, robot control methods and devices, robots, program products, and media. Background Art

[0002] As a representative product in the smart home field, sweeping robots have gradually become the mainstream solution for home cleaning in recent years thanks to their automated cleaning, intelligent navigation and scenario-based function iterations. Summary of the Invention

[0003] In a first aspect, an embodiment of the present application provides a method for controlling a robot, which is applied to the robot, wherein the robot includes a body and a robotic arm connected to the body; the method includes: receiving a first instruction, wherein the first instruction is used to instruct the robotic arm to automatically dispose of an object, wherein the disposing of the object includes acquiring the object or placing the object; in response to the first instruction, controlling the robot to move, and controlling the robot to perform at least one motion action to identify the object and obtain an object recognition result or to identify the front environment and obtain an environment recognition result; wherein the at least one motion action includes at least one of the following: the body moves forward, backward, or rotates, and the robotic arm adjusts its posture; acquiring the object or placing the object according to the recognition result.

[0004] In some embodiments, the controlling the robot to perform at least one motion action includes: when the first instruction indicates to automatically acquire an object through the robotic arm, according to predefined motion rules, controlling the body to perform at least one of the following motion actions in sequence: retreating a first distance, rotating a first angle in a first direction, and rotating a second angle in a second direction; wherein, the second angle is greater than the first angle, and the first direction is opposite to the second direction.

[0005] Further, in some embodiments, the controlling of the fuselage body to sequentially perform at least one of the following motion actions according to predefined motion rules: retreating a first distance, rotating a first angle in a first direction, and rotating a second angle in a second direction includes: identifying the rear environment of the fuselage body and determining a second distance; wherein, the second distance is the distance between the fuselage body and the rear obstacle, or the second distance is the distance between the fuselage body and the edge of the rear plane, and the plane is the plane where the fuselage body is located; if the second distance is greater than or equal to the first threshold, controlling the fuselage body to sequentially perform at least one of the following motion actions according to predefined motion rules: retreating a first distance, rotating a first angle in a first direction, and rotating a second angle in a second direction.

[0006] In some embodiments, controlling the robot to perform at least one motion action includes at least one of the following steps, or, before receiving the first instruction, the method further includes at least one of the following steps:

[0007] receiving a second instruction, and controlling the fuselage body to retreat a third distance according to the second instruction;

[0008] receiving a third instruction, and controlling the main body of the fuselage to move forward a fourth distance according to the third instruction;

[0009] receiving a fourth instruction, and controlling the main body of the fuselage to rotate in the first direction by a third angle according to the fourth instruction;

[0010] receiving a fifth instruction, and controlling the main body of the fuselage to rotate in the second direction by a fourth angle according to the fifth instruction;

[0011] Receive a posture adjustment instruction, and adjust the posture of the robotic arm according to the posture adjustment instruction; wherein, the robotic arm is provided with at least one sensor for collecting environmental data.

[0012] In some embodiments, the method further includes: collecting environmental data using at least one sensor on the fuselage body, and / or collecting environmental data using at least one sensor on the robotic arm; identifying the object based on the collected environmental data to obtain an object recognition result or identifying the front environment to obtain an environmental recognition result.

[0013] In some embodiments, acquiring the object based on the object recognition result includes: determining the target rest position of the fuselage body and the target posture of the robotic arm based on the object recognition result; controlling the fuselage body to move to the target rest position, and controlling the robotic arm to adjust to the target posture; after controlling the fuselage body to move to the target rest position and controlling the robotic arm to adjust to the target posture, controlling the end effector of the robotic arm to perform an acquisition action.

[0014] In some embodiments, the environment recognition result includes the number of sampling points in the point cloud and the height of the point cloud; placing the object according to the environment recognition result includes: if it is determined that the point cloud meets a preset condition based on the number of sampling points and the height of the point cloud is within a predefined first height range, determining that there is a storage container in the front environment; and controlling the robotic arm to place the acquired object in the storage container.

[0015] In some embodiments, the environment recognition result includes the number of sampling points in a point cloud and the height of the point cloud; placing the object according to the environment recognition result includes: if it is determined that the point cloud meets a preset condition according to the number of sampling points and the height of the point cloud is within a predefined second height range, determining that there is an obstacle in the front environment; controlling the main body of the fuselage to rotate or move, or controlling the robotic arm to rotate, so that the robotic arm is not obstructed by the obstacle during the process of placing the object.

[0016] In some embodiments, the received instruction is sent based on a touch operation received by a control key of the UI interface.

[0017] In a second aspect, an embodiment of the present application provides a control device for a robot, wherein the device is applied to a robot, the robot including a body and a robotic arm connected to the body; the device includes:

[0018] a first receiving unit configured to receive a first instruction, wherein the first instruction is used to instruct the robotic arm to automatically dispose of an object, wherein disposing of the object includes acquiring the object or placing the object;

[0019] a first control unit configured to control the robot to move in response to the first instruction, and to control the robot to perform at least one motion action to identify the object to obtain an object recognition result or to identify the front environment to obtain an environment recognition result; wherein the at least one motion action includes at least one of the following: the main body moving forward, backward, or rotating, and the robot arm adjusting its posture;

[0020] The second control unit is configured to obtain or place the object according to the recognition result.

[0021] In some embodiments, the controlling the robot to perform at least one motion action includes: when the first instruction indicates to automatically acquire an object through the robotic arm, according to predefined motion rules, controlling the body to perform at least one of the following motion actions in sequence: retreating a first distance, rotating a first angle in a first direction, and rotating a second angle in a second direction; wherein, the second angle is greater than the first angle, and the first direction is opposite to the second direction.

[0022] In some embodiments, the controlling of the fuselage body to sequentially perform at least one of the following motion actions according to predefined motion rules: retreating a first distance, rotating a first angle in a first direction, and rotating a second angle in a second direction includes: identifying the rear environment of the fuselage body and determining a second distance; wherein, the second distance is the distance between the fuselage body and the rear obstacle, or the second distance is the distance between the fuselage body and the edge of the rear plane, and the plane is the plane where the fuselage body is located; if the second distance is greater than or equal to a first threshold, the controlling of the fuselage body to sequentially perform at least one of the following motion actions according to predefined motion rules: retreating a first distance, rotating a first angle in a first direction, and rotating a second angle in a second direction.

[0023] In some embodiments, controlling the robot to perform at least one motion action includes at least one of the following steps, or the device further performs at least one of the following steps before receiving the first instruction:

[0024] receiving a second instruction, and controlling the fuselage body to retreat a third distance according to the second instruction;

[0025] receiving a third instruction, and controlling the main body of the fuselage to move forward a fourth distance according to the third instruction;

[0026] receiving a fourth instruction, and controlling the main body of the fuselage to rotate in the first direction by a third angle according to the fourth instruction;

[0027] receiving a fifth instruction, and controlling the main body of the fuselage to rotate in the second direction by a fourth angle according to the fifth instruction;

[0028] Receive a posture adjustment instruction, and adjust the posture of the robotic arm according to the posture adjustment instruction; wherein, the robotic arm is provided with at least one sensor for collecting environmental data.

[0029] In some embodiments, the device also includes a data acquisition unit and an identification unit; wherein: the data acquisition unit is configured to collect environmental data using at least one sensor on the fuselage body, and / or to collect environmental data using at least one sensor on the robotic arm; the identification unit is configured to identify the object based on the collected environmental data to obtain an object identification result or to identify the front environment to obtain an environmental identification result.

[0030] In some embodiments, acquiring the object based on the object recognition result includes: determining the target rest position of the fuselage body and the target posture of the robotic arm based on the object recognition result; controlling the fuselage body to move to the target rest position, and controlling the robotic arm to adjust to the target posture; after controlling the fuselage body to move to the target rest position and controlling the robotic arm to adjust to the target posture, controlling the end effector of the robotic arm to perform an acquisition action.

[0031] In some embodiments, the environment recognition result includes the number of sampling points in the point cloud and the height of the point cloud; placing the object according to the environment recognition result includes: if it is determined that the point cloud meets a preset condition based on the number of sampling points and the height of the point cloud is within a predefined first height range, determining that there is a storage container in the front environment; and controlling the robotic arm to place the acquired object in the storage container.

[0032] In some embodiments, the environment recognition result includes the number of sampling points in a point cloud and the height of the point cloud; placing the object according to the environment recognition result includes: if it is determined that the point cloud meets a preset condition according to the number of sampling points and the height of the point cloud is within a predefined second height range, determining that there is an obstacle in the front environment; controlling the main body of the fuselage to rotate or move, or controlling the robotic arm to rotate, so that the robotic arm is not obstructed by the obstacle during the process of placing the object.

[0033] In some embodiments, the received instruction is sent based on a touch operation received by a control key of the UI interface.

[0034] In a third aspect, an embodiment of the present application provides a robot comprising a robotic arm, a body, a first memory, and a first processor, wherein the robotic arm is connected to the body; the robotic arm is used to obtain and place items to be disposed of; the first memory stores a computer program that can be run on a first processor, and when the first processor executes the program, the steps in the method described in the first aspect are implemented.

[0035] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor or a robot, implements the method described in the first aspect of the present application.

[0036] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising a computer program or instructions, which, when executed by a processor or a robot, implements the method described in the first aspect of the present application.

[0037] In a sixth aspect, an embodiment of the present application provides a computer program, which enables a processor or a robot to execute the method described in the first aspect.

[0038] It can be understood that in the embodiment of the present application, after the robot receives the first instruction for instructing automatic acquisition of an object or automatic placement of an object (that is, in the automatic object retrieval mode or in the automatic object placement mode), and before controlling the robotic arm to acquire or place the object, the robot is first controlled to perform at least one of the following movement actions, such as the main body moving forward, backward, rotating, and the robotic arm adjusting the posture, so that the robot has a wider field of view to obtain more complete information about the object in front or the front environment, which is beneficial to improving the recognition accuracy of the object in front or the front environment (for example, for the scene of acquiring objects, it is beneficial to determine a more accurate object position and object type, etc., and for the scene of placing objects, it is beneficial to accurately identify whether there are storage containers or obstacles in the front environment, etc.), so as to facilitate the subsequent better acquisition or placement of objects.

[0039] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings herein are incorporated into and constitute a part of this specification. These drawings illustrate embodiments consistent with the present application and, together with the specification, serve to illustrate the technical solutions of the present application. Obviously, the drawings described below are merely some embodiments of the present application. Those skilled in the art can, without inventive effort, derive other drawings from these drawings.

[0041] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0042] Figure 1 Schematic diagram of the structure of the robot provided in the embodiment of this application Figure 1 ;

[0043] Figure 2 Schematic diagram of the structure of the robot provided in the embodiment of this application Figure 2 ;

[0044] Figure 3 A diagram of a system architecture that may be applicable to the embodiments of this application;

[0045] Figure 4 Schematic diagram of the implementation process of the robot control method provided in the embodiment of the application Figure 1 ;

[0046] Figure 5 Schematic diagram of the UI interface of the remote control device provided in the embodiment of the present application Figure 1 ;

[0047] Figure 6 A top view showing the relative position relationship between the fuselage body and the object to be captured;

[0048] Figure 7 A schematic diagram of the implementation process of controlling a robotic arm to acquire an object provided in an embodiment of the present application;

[0049] Figure 8A Schematic diagram of the UI interface of the remote control device provided in the embodiment of the present application Figure 2 ;

[0050] Figure 8B Schematic diagram of the UI interface of the remote control device provided in the embodiment of the application Figure 3 ;

[0051] Figure 9 Schematic diagram of the implementation process of the robot control method provided in the embodiment of the application Figure 2 ;

[0052] Figure 10 A schematic diagram of the sweeping robot provided in an embodiment of the present application after grabbing an object;

[0053] Figure 11 An example diagram of a trash can provided in an embodiment of the present application;

[0054] Figure 12 Schematic diagram of the UI interface of the remote control device provided in the embodiment of the application Figure 4 ;

[0055] Figure 13 A top view of the features of a trash can identified by a TOF sensor provided in an embodiment of the present application;

[0056] Figure 14 A schematic diagram of a point cloud of features of a trash can identified by a TOF sensor from the perspective of a sweeper provided in an embodiment of the present application;

[0057] Figure 15 A schematic diagram of a point cloud of wall features identified by a TOF sensor from the perspective of a sweeper provided in an embodiment of the present application;

[0058] Figure 16 Schematic diagram of the structure of the robot control device provided in the embodiment of the present application Figure 1 ;

[0059] Figure 17Schematic diagram of the structure of the robot control device provided in the embodiment of the present application Figure 2 ;

[0060] Figure 18 Schematic diagram of the structure of the robot provided in the embodiment of this application Figure 3 ;

[0061] Figure 19 A schematic diagram of the structure of the remote control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] To make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the specific technical solutions of the present application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application but are not intended to limit the scope of the present application.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0064] In the following description, references to “some embodiments,” “this embodiment,” “embodiments of the present application,” and examples, etc., describe a subset of all possible embodiments. However, it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

[0065] Descriptions such as "first, second, third" appearing in the embodiments of the present application do not have a specific meaning (such as there is no distinction in order, nor does it indicate a special limitation on the number of devices in the embodiments of the present application). They are only for the convenience of clearly describing the embodiments of the present application and do not constitute any limitation on the embodiments of the present application.

[0066] Figure 1 Schematic diagram of the structure of the robot provided in the embodiment of this application Figure 1 .like Figure 1 As shown, the robot 100 includes a body 101, a robotic arm 102 and a storage compartment 103 for the robotic arm 102;

[0067] A storage compartment 103 for storing the robotic arm 102;

[0068] The robot arm 102 is connected to the body 101 and is used to handle objects, such as obtaining objects and placing objects.

[0069] In some embodiments, as Figure 1As shown, the robot 100 further includes a memory 104 and a processor 105; wherein the memory 104 stores a computer program that can be run on the processor 105, and when the processor 105 executes the program, it implements the control method of one or more combinations of the following embodiments.

[0070] It should be noted that the memory 104 is configured to store instructions and applications executable by the processor 105, and can also cache data to be processed or processed by the processor 105 and each unit in the robot 100, which can be implemented through flash memory (FLASH) or random access memory (RAM).

[0071] It should also be noted that in the embodiment of the present application, there is no limitation on the type of robot 100. The robot 100 may be a sweeping robot (also known as a cleaning robot, sweeping machine) or other types of robots having a robotic arm and a storage compartment for storing the robotic arm.

[0072] It should also be noted that in the embodiment of the present application, there is no limitation on the structure of the robotic arm 102 , as long as it can achieve the acquisition and placement of objects.

[0073] For example, in some embodiments, Figure 2 As shown, the robotic arm 102 of the robot 100 includes an end effector 1021, a first joint M1, a second joint M2, a third joint M3, a fourth joint M4, a fifth joint M5 and a sixth joint M6; wherein, the first joint M1 is connected to the bottom of the storage cabin, the second joint M2 is connected to the first joint M1, the second joint M2 and the third joint M3 are connected via a first link L1, the third joint M3 and the fourth joint M4 are connected via a second link L2, the fourth joint M4 and the fifth joint M5 are connected via a third link L3, the fifth joint and the sixth joint are connected via a fourth link L4, and the sixth joint M6 is connected to the end effector 1021; the sixth joint M6 is used to control the action of the end effector 1021. The end effector 1021 is used to acquire and release objects. For example, the end effector 1021 is an adsorption device or a gripper, etc. As shown Figure 2 As shown, the robot 100 further includes a cleaning component 201, a laser radar sensor 202, and a camera 203. There can be one or more laser radar sensors and cameras, respectively. The robot's main body is equipped with a laser radar sensor and / or camera. In some embodiments, the robot's mechanical arm is also equipped with a laser radar and / or camera.

[0074] Of course, in the embodiment of the present application, the robot arm is not limited to Figure 2 In some embodiments, the robot arm may be a robot hand, for example, Figure 2 The end effector of the robot arm shown is a gripper.

[0075] It should be understood that Figure 2 As shown, the robotic arm 102 can be completely folded and retracted into the robot's internal space (i.e., the storage compartment), allowing the robot to maintain an ultra-thin body, fully displaying its structural aesthetics and improving cleaning coverage. The new bionic manipulator has 5 joints and 6 degrees of freedom, allowing the robotic arm to fold, rotate, and control its posture flexibly, such as the fingers at the end of the robotic arm. The agile robotic arm can remove obstacles such as shoes and socks to clean the bottom, improving the floor cleaning coverage, and evolving the cleaning dimension from plane to three-dimensional; the robot intelligently identifies scattered objects, and AI calculates the best grasping position, classifying and storing them in customized areas or stone storage boxes, saying goodbye to manual sorting; with the dual field of view composed of the end camera of the robotic arm and the body camera, you can also make video calls at high and low angles; intelligent control of object picking, intelligent retrieval of distant objects, bringing users a better smart home experience.

[0076] Figure 3 A possible applicable system architecture diagram provided for the embodiment of this application; Figure 3 As shown, the system architecture includes a remote control device 300 and a robot 100; the remote control device 300 and the robot 100 can communicate. For example, the remote control device 300 sends instructions to the robot 100 to control the movement or rotation of the robot 100's body, or instructs the robot 100 to automatically acquire or place an object through the remote control device 300, or controls the robot's mechanical arm 102 to extend, retract, acquire, and / or place an object through the remote control device 300. For another example, the robot 100 can also provide feedback to the remote control device 300 on robot-related information, such as the robot's current posture, whether it has successfully acquired an object, and / or successfully placed an object.

[0077] It should be noted that the remote control device 300 and the robot 100 can communicate by accessing the network or by D2D. This application does not limit the communication method between the two.

[0078] In the embodiment of the present application, the type of remote control device 300 is not limited, and the remote control device can be a variety of devices with information processing and communication capabilities. For example, the remote control device 300 can be a mobile phone, tablet computer, laptop computer, desktop computer, smart watch, or remote control.

[0079] The following embodiment describes a method for controlling a robot, wherein a remote control device is used to remotely control the movements of the robot's body and / or robotic arms, thereby achieving the acquisition and placement of objects such as paper balls, socks, or shoes, thereby improving the user experience.

[0080] Figure 4 Schematic diagram of the implementation process of the robot control method provided in the embodiment of the application Figure 1 ;like Figure 4 As shown, the method is applied to the robot 100, and the method includes the following steps 401 to 403:

[0081] Step 401: The robot receives a first instruction, wherein the first instruction is used to instruct the robot arm to automatically handle an object, wherein the handling of the object includes acquiring the object or automatically placing the object;

[0082] In step 402, the robot controls movement in response to the first instruction, and controls execution of at least one motion action to identify the object and obtain an object recognition result or to identify the environment ahead and obtain an environment recognition result; wherein the at least one motion action includes at least one of the following: the robot body moves forward, backward, or rotates, and the robot arm adjusts its posture;

[0083] Step 403: The robot obtains or places the object according to the recognition result.

[0084] It can be understood that in the embodiment of the present application, after receiving the first instruction for instructing automatic acquisition of an object or automatic placement of an object (that is, in the automatic object retrieval mode or in the automatic object placement mode), and before controlling the robotic arm to acquire or place the object, the robot is first controlled to perform at least one of the following motions, such as the body moving backward, the body rotating in the first direction, the body rotating in the second direction, and the robotic arm adjusting the posture, so that the robot has a wider field of view to obtain more complete information about the object in front or the environment in front, which is beneficial to improving the recognition accuracy of the object in front or the environment in front (such as for the scene of acquiring objects, it is beneficial to determine a more accurate object position and object type, etc., and for the scene of placing objects, it is beneficial to accurately identify whether there are storage containers or obstacles in the environment in front, etc.), so as to facilitate the subsequent better acquisition or placement of objects.

[0085] The following describes further optional implementations and related terms of each of the above steps.

[0086] In step 401 , the robot receives a first instruction, where the first instruction is used to instruct the robot arm to automatically handle an object, where the handling of the object includes acquiring the object or placing the object.

[0087] In some embodiments, the first instruction may be sent via a remote control device, or the first instruction may be sent via a UI interface on the robot's body when a touch operation is detected.

[0088] In step 402, the robot controls the movement of the robot in response to the first instruction, and controls the robot to perform at least one motion action to identify the object and obtain an object recognition result or to identify the front environment and obtain an environment recognition result; wherein, the at least one motion action includes at least one of the following: the body moves forward, the body moves backward, the body rotates, and the robotic arm adjusts its posture.

[0089] In an embodiment of the present application, the robot collects environmental data while performing at least one motion action (the robot can collect environmental data while performing the motion action), and identifies the object to be acquired based on the collected environmental data to obtain an object recognition result, or identifies the environment in front to obtain an environmental recognition result. In an embodiment of the present application, after receiving the first instruction, the robot can automatically perform at least one motion action according to a predefined motion rule, that is, there is no need for the user to instruct the robot which motion action to perform through the remote control device or the robot's UI interface. In other embodiments, after receiving the first instruction, the robot can also perform the corresponding motion action according to the second instruction, third instruction, etc. sent by the subsequent user through the remote control device or the robot's UI interface. In some other embodiments, the robot can also automatically perform at least one motion action according to the predefined motion rule. If the robot receives an instruction sent by the remote control device or the robot's UI interface such as the second instruction, third instruction, etc., the robot interrupts the automatically executed motion action and executes the motion action corresponding to the instruction, or does not interrupt the automatically executed motion action, but executes the motion action indicated by the instruction after automatically executing the predefined motion action according to the predefined motion rule. The following will describe in detail each of the above possible implementations.

[0090] In step 402 , the robot controls the movement of the robot in response to the first instruction, and controls the robot to perform at least one motion action to identify the object to obtain an object recognition result or to identify the front environment to obtain an environment recognition result.

[0091] Among them, for the feature of controlling the robot to perform at least one motion action, in some embodiments, it can further include: the robot controls the body to perform at least one of the following motion actions in sequence according to a predefined motion rule: retreat a first distance, rotate a first angle in a first direction, and rotate a second angle in a second direction; wherein, the second angle is greater than the first angle; the predefined motion rule defines at least one of the following motion actions: retreat a first distance, rotate a first angle in a first direction, and rotate a second angle in a second direction, and the first direction is opposite to the second direction.

[0092] Furthermore, in some embodiments, the feature of controlling the robot to perform at least one motion action may further include: when the first instruction indicates to automatically acquire an object through the robotic arm, the robot controls the body to perform at least one of the following motion actions in sequence according to predefined motion rules: retreat a first distance, rotate a first angle in a first direction, and rotate a second angle in a second direction; wherein, the second angle is greater than the first angle.

[0093] It is understood that the above embodiments include a variety of combination schemes. In one combination scheme, the predefined motion rule is that the main body of the robot sequentially moves backward a first distance, rotates in a first direction by a first angle, and rotates in a second direction by a second angle. In this way, the main body of the robot moves backward first and then rotates. Compared with rotating first and then moving backward, the main body of the robot is more likely to be damaged or destroyed due to collisions. This is because if the robot rotates first and then moves backward, the robot may collide with or rub against surrounding objects or walls due to the robot being too close to them during the rotation process.

[0094] In the embodiments of the present application, the first direction and the second direction are not limited, and the two directions can be opposite. For example, the first direction is counterclockwise and the second direction is clockwise; or in another example, the first direction is clockwise and the second direction is counterclockwise.

[0095] It can be understood that before the robot retreats the first distance, there may be an obstacle behind the robot (such as a wall or other obstacle, etc.), and there may also be a "cliff" behind the robot (such as the robot is on the outer edge of the steps). Therefore, in order to enhance the safety of the robot, further, in some embodiments, the robot controls the body to perform at least one of the following motion actions in sequence according to predefined motion rules: retreating the first distance, rotating the first angle in the first direction, and rotating the second angle in the second direction, including: the robot identifies the rear environment of the body and determines the second distance; wherein the second distance is the distance between the body and the rear obstacle, or the second distance is the distance between the body and the edge of the rear plane, and the plane is the plane where the body is located; if the second distance is greater than or equal to the first threshold, the robot controls the body to perform at least one of the following motion actions in sequence according to the predefined motion rules: retreating the first distance, rotating the first angle in the first direction, and rotating the second angle in the second direction.

[0096] It can be understood that if the second distance is greater than or equal to the first threshold, it means that the environment behind the robot's main body is safe; after confirming that the rear environment is safe, the main body is controlled to perform at least one of the following motion actions in sequence according to the predefined motion rules: retreat a first distance, rotate a first angle in a first direction, and rotate a second angle in a second direction; in this way, the robot is less likely to be damaged due to colliding with obstacles in the rear environment or falling off a "cliff" when retreating.

[0097] In other embodiments, the feature of "controlling the robot to perform at least one motion action" described in step 402 may also be that the user instructs the robot to perform the motion action through a remote control device or the robot's UI (i.e., user-assisted). In this embodiment, controlling the robot to perform at least one motion action may further include at least one of the following steps:

[0098] (1) The robot receives a second instruction and controls the body to retreat a third distance according to the second instruction; for example, the second instruction may be sent by a remote control device, or may be sent after a touch operation is received on the UI interface of the robot;

[0099] (2) The robot receives a third instruction and controls the body to move forward a fourth distance according to the third instruction; for example, the third instruction may be sent by a remote control device, or may be sent after a touch operation is received on a UI interface of the robot;

[0100] (3) The robot receives a fourth instruction and controls the body to rotate in the first direction by a third angle according to the fourth instruction; for example, the fourth instruction may be sent by a remote control device or may be sent after a touch operation is received on a UI interface of the robot;

[0101] (4) The robot receives a fifth instruction and controls the body to rotate in the second direction by a fourth angle according to the fifth instruction; for example, the fifth instruction may be sent by a remote control device or may be sent after a touch operation is received on the UI interface of the robot;

[0102] (5) The robot receives a posture adjustment instruction and adjusts the posture of the robotic arm according to the posture adjustment instruction; wherein the robotic arm is provided with at least one sensor for collecting environmental data. For example, the posture adjustment instruction may be sent by a remote control device or may be sent after a touch operation is received on the UI interface of the robot;

[0103] It should be understood that the specific number of the third through fifth instructions and the posture adjustment instructions that the robot receives depends on the user's operation. In other words, the robot may receive any one or more of the third through fifth instructions and the posture adjustment instructions. Upon receiving a particular instruction, the robot may respond to the instruction and perform the corresponding movement action.

[0104] In some other embodiments, for the feature "controlling the robot to perform at least one motion action" described in step 402, the at least one motion action includes not only the actions defined by the predefined motion rules, but also the motion actions indicated by the user through the remote control device or the UI interface of the robot. In this way, it is beneficial for the robot to collect complete information about objects in the environment through sensors, thereby improving the accuracy of object recognition, and further helping the robot to perform subsequent correct actions based on more accurate object information (such as object type and object position). That is, in this embodiment, controlling the robot to perform at least one motion action may further include: the robot controls the body to perform at least one of the following motion actions in sequence according to the predefined motion rules: retreating a first distance, rotating a first angle in a first direction, and rotating a second angle in a second direction; wherein the second angle is greater than the first angle; and controlling the execution of at least one motion action also includes at least one of the following steps:

[0105] (1) The robot receives a second instruction and controls the body to retreat a third distance according to the second instruction;

[0106] (2) The robot receives a third instruction and controls the body to move forward a fourth distance according to the third instruction;

[0107] (3) The robot receives a fourth instruction and controls the body to rotate toward the first direction by a third angle according to the fourth instruction;

[0108] (4) The robot receives the fifth instruction and controls the body to rotate in the second direction by a fourth angle according to the fifth instruction;

[0109] (5) The robot receives a posture adjustment instruction and adjusts the posture of the robotic arm according to the posture adjustment instruction; wherein, the robotic arm is provided with at least one sensor for collecting environmental data.

[0110] For the above-mentioned embodiment in which both user assistance and robot automatic execution of motion actions are provided, it is not limited whether the above-mentioned steps (1)-(5) are performed after or before the robot controls the body to sequentially perform the motion actions defined in the motion rules according to the predefined motion rules, or whether they are performed during the process in which the robot controls the body to sequentially perform the defined motion actions according to the predefined motion rules.

[0111] In some other embodiments, the user may also adjust the robot's position and the posture of the robot arm (i.e., the joint angle) before issuing the first instruction through the remote control device or the robot's UI interface. After adjusting to the appropriate position or posture, the user clicks the control key corresponding to the first instruction to send the first instruction to the robot. That is, in this embodiment, before the robot receives the first instruction, the method further includes at least one of the following steps:

[0112] (1) The robot receives a second instruction and controls the body to retreat a third distance according to the second instruction;

[0113] (2) The robot receives a third instruction and controls the body to move forward a fourth distance according to the third instruction;

[0114] (3) The robot receives a fourth instruction and controls the body to rotate toward the first direction by a third angle according to the fourth instruction;

[0115] (4) The robot receives the fifth instruction and controls the body to rotate in the second direction by a fourth angle according to the fifth instruction;

[0116] (5) The robot receives a posture adjustment instruction and adjusts the posture of the robotic arm according to the posture adjustment instruction; wherein, the robotic arm is provided with at least one sensor for collecting environmental data.

[0117] For example, for the remote control device, in the scene of the object to be acquired, such as Figure 5As shown, the remote control device displays a UI interface 500, which includes: a background image 501, a main viewing area 502 of the robot (i.e., a blue area), prompt information 503, a position adjustment control key 504, and an execution control key 505; wherein, the background image 501 is an image captured by the robot at its current position; when the execution control key 505 is used to control the automatic acquisition of an object, the prompt information is used to prompt the robot to move so that the object to be acquired in the background image is in the main viewing area; the position adjustment control key is used to control the body to perform movement actions.

[0118] Furthermore, in some embodiments, Figure 5 As shown, the position adjustment control key 504 includes a first control key 5041, a second control key 5042, a third control key 5043 and a fourth control key 5044; wherein, the first control key 5041 is used to control the robot's body to move forward, the second control key 5042 is used to control the robot's body to move backward, the third control key 5043 is used to control the robot's body to rotate clockwise, and the fourth control key 5044 is used to control the robot's body to rotate counterclockwise.

[0119] In a possible implementation, the viewing angle when capturing the background image 501 is greater than the viewing angle of the main viewing area 502. For example, Figure 6 It is a top view of the relative position relationship between the fuselage body and the object to be captured, and the main viewing area and the active area of the robotic arm at least partially overlap.

[0120] As mentioned above, in step 402, the robot controls the execution of at least one motion action while collecting environmental data in response to the first instruction.

[0121] Furthermore, in some embodiments, the method further includes: the robot using at least one sensor on the main body to collect environmental data; and / or the robot using at least one sensor on the robotic arm to collect environmental data; and identifying the object based on the collected environmental data to obtain an object recognition result or identifying the environment ahead to obtain an environment recognition result. In conjunction with the above embodiments, whether the user manually or the robot actively (i.e., automatically) adjusts the position or angle of the main body, or the user manually or the robot actively (i.e., automatically) adjusts the posture of the robotic arm, one of the purposes is to help the robot obtain a wider and more diverse perspective through the sensors on the main body or the robotic arm, thereby obtaining information on more surfaces of the same object in the environment, thereby improving the accuracy of object recognition or the accuracy of recognition of the environment ahead, helping the robot to better decide on the next operation / action, and enhancing the robot's intelligence.

[0122] Step 403 : acquiring an object or placing an object according to the recognition result, that is, acquiring an object according to the object recognition result or placing an object according to the environment recognition result.

[0123] I understand. Figure 4 The control method of the robot shown actually includes two schemes, one scheme is a scheme for acquiring an object based on the object recognition result, and the other scheme is a scheme for placing an object based on the environment recognition result.

[0124] Among them, in the scheme for acquiring an object, it includes: receiving a first instruction, the first instruction is used to instruct automatic acquisition of an object; in response to the first instruction, controlling the robot to move, and controlling the robot to perform at least one motion action to identify the object and obtain an object recognition result; acquiring the object according to the object recognition result; wherein, the at least one motion action includes at least one of the following: the fuselage body moves forward, backward, rotates, and the robotic arm adjusts its posture.

[0125] Among them, in the scheme for placing an object, it includes: receiving a first instruction, the first instruction is used to instruct the automatic placement of an object; in response to the first instruction, controlling the robot to move, and controlling the robot to perform at least one motion action to identify the front environment and obtain an environment recognition result; wherein, the at least one motion action includes at least one of the following: the fuselage body moves forward, backward, and rotates, and the robotic arm adjusts its posture; placing the object according to the environment recognition result.

[0126] Furthermore, in some embodiments, the object is obtained according to the object recognition result, such as Figure 7 As shown, it includes the following steps 701 to 703:

[0127] Step 701: The robot determines a target rest position of the body and a target posture of the robotic arm based on the object recognition result.

[0128] Step 702: The robot controls the body to move to the target rest position, and controls the robotic arm to adjust to the target posture.

[0129] In step 703 , after controlling the body to move to the target rest position and controlling the manipulator to adjust to the target posture, the robot controls the end effector of the manipulator to perform an acquisition action.

[0130] Furthermore, in some embodiments, the environment recognition result includes the number of sampling points in the point cloud and the height of the point cloud; placing the object according to the environment recognition result includes: if it is determined that the point cloud meets a preset condition based on the number of sampling points, and the height of the point cloud is within a predefined first height range, determining that there is a storage container in the front environment; and controlling the robotic arm to place the acquired object in the storage container.

[0131] In a possible implementation, determining that the point cloud satisfies a preset condition based on the number of sampling points includes: determining that the point cloud satisfies the preset condition when the number of sampling points in the point cloud is greater than or equal to a second threshold; wherein the preset condition includes that the number of sampling points in the point cloud is greater than or equal to the second threshold.

[0132] In another possible implementation, determining whether the point cloud satisfies a preset condition based on the number of sampling points includes: determining a density of the point cloud based on the number of sampling points in the point cloud; and determining that the point cloud satisfies the preset condition when the density of the point cloud is greater than or equal to a third threshold; wherein the preset condition includes that the density of the point cloud is greater than or equal to the third threshold.

[0133] In another possible implementation, determining whether the point cloud satisfies a preset condition based on the number of sampling points includes: determining a density of the point cloud based on the number of sampling points in the point cloud; and determining that the point cloud satisfies the preset condition if the number of sampling points in the point cloud is greater than or equal to a second threshold and the density of the point cloud is greater than or equal to a third threshold; wherein the preset condition includes: the number of sampling points in the point cloud is greater than or equal to the second threshold, and the density of the point cloud is greater than or equal to a third threshold.

[0134] It can be understood that the point cloud meets the preset conditions, which means that the point cloud is available, that is, the sampling points in the point cloud are not noise points, but sampling points of physical substances. The availability of the point cloud can also be understood as the possible existence of obstacles or storage containers in the front environment.

[0135] It can be understood that if the collected point cloud of the front environment meets the preset conditions, it means that there may be obstacles or storage containers in the front environment, and the height of the storage container is usually a known specific height, for example, the height of the trash can used with the robot is 30 cm; therefore, in the embodiment of the present application, the condition for determining the presence of a storage container in the front environment is that the point cloud is available and the point cloud height is within a predefined first height range (such as 25 cm-30 cm); this is beneficial to improving the recognition accuracy of the storage container and reducing the occurrence of misidentification and missed identification, so that the robot can place the acquired objects reasonably, thereby improving the intelligence of the robot and the user experience.

[0136] In the embodiment of the present application, the type of the storage container is not limited. For a scenario where the object captured by the end effector is garbage, the storage container can be a trash can. For a scenario where the object captured by the end effector is clothing, the storage container can be a trash can.

[0137] Furthermore, in some other embodiments, the environment recognition result includes the number of sampling points in the point cloud and the height of the point cloud; placing the object according to the environment recognition result includes: if it is determined that the point cloud meets a preset condition according to the number of sampling points, and the height of the point cloud is within a predefined second height range, determining that there is an obstacle in the front environment; controlling the main body of the fuselage to rotate or move, or controlling the robotic arm to rotate, so that the robotic arm is not obstructed by the obstacle during the placement of the object, thereby placing the object after the end effector of the robotic arm is away from the obstacle.

[0138] In an embodiment of the present application, the first height interval and the second height interval are two different height intervals, and the first height interval and the second height interval do not intersect. In some embodiments, the lower limit of the second height interval is greater than the upper limit of the first height interval, or the upper limit of the second height interval is less than the lower limit of the first height interval.

[0139] In the embodiments of the present application, various methods can be used to control the end effector of the robotic arm away from the obstacle. The purpose is to prevent the robotic arm from hitting a wall or obstacle while placing an object, causing damage. In one possible implementation, the robot can control the main body of the robot to rotate or move toward the obstacle to move the robotic arm away from the obstacle. In another possible implementation, the robot can control the robotic arm to rotate, thereby actually moving the robotic arm and the end effector away from the obstacle.

[0140] It should be noted that the posture adjustment instructions mentioned above are used to control the posture of the robotic arm (such as the arm span, joint angles and / or the orientation of the robotic arm, etc.). In this application, there is no limitation on the posture adjustment instructions, and the posture adjustment instructions may include at least one of the following instructions:

[0141] (1) a rotation instruction; wherein the rotation instruction is used to control the end effector of the robotic arm to rotate in an indicated direction;

[0142] (2) an arm span adjustment instruction; wherein the arm span adjustment instruction is used to control the extension and / or contraction of the robotic arm;

[0143] (3) Joint adjustment instructions; wherein the joint adjustment instructions are used to control the angle of a specified joint (different joint adjustment instructions may correspond to different joints); the joints are joints of the robotic arm.

[0144] It can be understood that adjusting the joint angle actually adjusts the angle between the connecting rods at both ends of the joint. Figure 2 Taking the joint M3 (M3 joint) shown as an example, adjusting the angle of the joint M3 actually adjusts the angle between the connecting rod L1 and the connecting rod L2; adjusting the angle of the joint M4 (M4 joint) actually adjusts the angle between the connecting rod L2 and the connecting rod L3.

[0145] It should be noted that in the embodiments of the present application, there is no limitation on which type of instructions the rotation instructions, arm span adjustment instructions and joint adjustment instructions further include. For further embodiments of these instructions, please refer to the description below.

[0146] Figure 8A Schematic diagram of the UI interface of the remote control device provided in the embodiment of the application Figure 2 ;like Figure 8A As shown, the UI interface 800 includes the following control keys:

[0147] The fifth control key 801 is used to control the end effector (such as the gripper) of the robot arm to rotate counterclockwise;

[0148] The sixth control key 802 is used to control the end effector (such as the gripper) of the robot arm to rotate clockwise;

[0149] The seventh control key 803 is used to control the robot arm to extend in the third direction; Figure 8A The arrow in the seventh control button 803 is shown, and the third direction is a direction away from the top surface and perpendicular to the top surface of the robot;

[0150] The eighth control key 804 is used to control the robot arm to retract in the fourth direction; Figure 8AAs shown in the arrow of the eighth control key 804, the fourth direction is a direction close to the top surface and perpendicular to the top surface;

[0151] The ninth control key 805 is used to control the robot arm to extend in the fifth direction; Figure 8A The arrow in the ninth control key 805 is shown, and the fifth direction is a direction away from the top surface and parallel to the top surface;

[0152] The tenth control key 806 is used to control the robot arm to retract in the sixth direction; Figure 8A As shown in the arrow of the tenth control key 806, the sixth direction is a direction close to the top surface and parallel to the top surface.

[0153] Figure 8B Schematic diagram of the UI interface of the remote control device provided in the embodiment of the application Figure 3 ;like Figure 8B As shown, the UI interface 830 includes the following control keys:

[0154] The eleventh control key 807 is used to control the angle of the fourth joint; Figure 8B As shown, the arrow in the eleventh control key 807 is used to prompt the user that the control key is used to increase the angle of the fourth joint;

[0155] The twelfth control key 808 is used to control the angle of the fourth joint; Figure 8B As shown, the arrow in the twelfth control key 808 is used to remind the user that the control key is used to reduce the angle of the fourth joint;

[0156] The thirteenth control key 809 is used to control the angle of the third joint of the instruction; Figure 8B As shown, the arrow in the thirteenth control key 809 is used to prompt the user that the control key is used to increase the angle of the third joint;

[0157] The fourteenth control key 810 is used to control the angle of the third joint of the adjustment instruction; Figure 8B As shown, the arrow in the fourteenth control key 810 is used to prompt the user that the control key is used to reduce the angle of the third joint.

[0158] Taking the end effector as a gripper as an example, for example, Figure 8A and 8B As shown, the UI interface 800 and the UI interface 830 also include a fifteenth control key 811 and / or a sixteenth control key 812; wherein the fifteenth control key 811 is used to control the robot to close the gripper; the sixteenth control key 812 is used to control the robot to open the gripper.

[0159] In some embodiments, the instructions described in the above embodiments, i.e., the instructions received by the robot, are sent based on touch operations received by control keys on a UI interface. The UI interface can be a display interface on the robot's screen or a display interface on the screen of a remote control device.

[0160] It can be understood that the above one or more embodiments mainly describe the control method of the robot from the robot side, and the following will describe the control method of the robot from the remote control device side.

[0161] Figure 9 Schematic diagram of the implementation process of the robot control method provided in the embodiment of the application Figure 2 ;like Figure 9 As shown, the method includes the following steps 901 to 902:

[0162] Step 901: The remote control device displays a UI interface, wherein the UI interface includes at least one of the following: a background image, a main viewing area of the robot, prompt information, a position adjustment control key, and an execution control key; wherein the background image is an image captured by the robot at its current position; when the execution control key is used to control automatic object acquisition, the prompt information is used to prompt the robot to move so that the object to be acquired in the background image is within the main viewing area; and the position adjustment control key is used to control the robot body to perform a movement action.

[0163] In step 902 , the remote control device sends a first instruction to the robot in response to the touch operation received by the execution control key; wherein the first instruction is used to instruct automatic acquisition of an object or automatic placement of an object.

[0164] It can be understood that in the embodiment of the present application, if the background image and the main viewing area are displayed on the UI interface, the user can be more intuitively guided to adjust the position and angle of the robot's body, so that the body is finally in a suitable position (that is, at this position, the object to be acquired is in the robot's main viewing area, for example Figure 5 As shown, the paper ball of the object to be obtained is in the main viewing area 502), which is beneficial for the robot to accurately obtain the object to be obtained and reduces the probability of empty grasping.

[0165] For example, the position adjustment control key includes at least one of the following: a first control key, a second control key, a third control key, and a fourth control key. Figure 5 .

[0166] In some embodiments, the method further comprises at least one of the following steps:

[0167] (6) The remote control device sends a second instruction to the robot in response to the touch operation received by the first control key, wherein the second instruction is used to instruct the robot to move the main body backward;

[0168] (7) the remote control device sends a third instruction to the robot in response to the touch operation received by the second control key, wherein the third instruction is used to instruct the robot to move the main body forward;

[0169] (8) The remote control device sends a fourth instruction to the robot in response to the touch operation received by the third control key, wherein the fourth instruction is used to instruct the robot to rotate the body in the first direction;

[0170] (9) The remote control device sends a fifth instruction to the robot in response to the touch operation received by the fourth control key, and the fifth instruction is used to instruct the robot to rotate the body in a second direction; wherein the second direction is opposite to the first direction.

[0171] In some embodiments, the UI interface also includes a gesture adjustment control key, see Figure 8A and Figure 8B .

[0172] In some embodiments, the method further includes: the remote control device sending a posture adjustment instruction to the robot in response to the touch operation received by the posture adjustment control key, wherein the posture adjustment instruction is used to instruct adjustment of the posture of the robotic arm.

[0173] In some embodiments, the robot control method described in one or more of the above embodiments may be a control method for a cleaning robot (such as a sweeper), and the robot may be a cleaning robot.

[0174] The following examples describe possible implementations of the robot control method described in one or more of the above embodiments.

[0175] Figure 6 and Figure 5 They are respectively the user's perspective diagram and the UI interface diagram of the remote control device corresponding to the main perspective of the sweeper. First, the user follows the prompt to remotely control the sweeper to the object to be grasped. As a result, the paper ball to be grasped is in Figure 5 In the main viewing area 502, the user clicks the execution control key 505 (i.e., indicates to start grabbing), and the sweeping robot starts to automatically grab the object. Further, in some embodiments, the sweeping robot automatically grabs the object through the following steps 1 to 3:

[0176] In step 1, after receiving the automatic grasping command, the robot vacuum cleaner needs to perform a recognition operation to identify the object in the area ahead. First, based on perception information and positioning information (environmental map, its own position and / or surroundings), the robot vacuum cleaner confirms that the rear of the main body is safe. It then begins to move back 175mm, then turns left 30 degrees, then right 60 degrees and back. These recognition operations are to accurately identify the object in front. Once the object is identified, the robot stops the remaining recognition operations and begins preparing to grasp it.

[0177] Furthermore, during the process of the sweeping machine implementing step 1, the user can also control the sweeping machine to move or rotate by remote control, thereby assisting the sweeping machine to accurately identify objects in the surrounding environment. Figure 5 In the state shown, the user can further remotely control the relative distance, relative angle and posture of the robot arm between the sweeper and the object, so as to obtain more complete and comprehensive information about the object, which is beneficial to improve the recognition accuracy of objects in the environment.

[0178] Step 2: The robot calculates the target dwell position and the gripping angles of each joint of the robotic arm based on the identified object information (such as object type and object location coordinates);

[0179] Step 3: The robot moves to the target stop position and starts to grasp the object at the grasping angle calculated in step 2.

[0180] In addition, users can use the remote control to automatically put down the grabbed objects. Figure 10 A schematic diagram of the sweeping robot provided in an embodiment of the present application after grabbing an object; Figure 11 This is an example of a trash can provided in the embodiment of the present application. The trash can is 30 cm high, 20 cm long, and 20 cm wide. After the user remotely controls the sweeper to automatically grab an object, the UI interface of the remote control device is as follows: Figure 12 As shown, an execution control key 1201 is displayed on the interface, and the execution control key 1201 is used to instruct the sweeping machine to automatically place objects.

[0181] When in use, the user can click the position adjustment key (such as Figure 12 The first control button 5041, the second control button 5042, the third control button 5043, and the fourth control button 5044 are shown in the figure, thereby controlling the robot to move to the specified location. When the user clicks the execution control button 1201, the robot automatically places the object. This operation has two options: one is to place the object directly on the ground, and the other is to place it in a trash can. The robot uses a sensor to detect whether there is a trash can in front of it and selects different placement actions. The following details the process of automatic placement (i.e., automatic object placement).

[0182] After receiving the command to put down, the robot vacuum first detects whether the space in front of it has the characteristics of a trash can (such as Figure 11 When the robot vacuum cleaner is close to the trash can, most of the point clouds are concentrated in the front area of the robot vacuum cleaner, and the point cloud height is in the range of 25cm-30cm. Based on these features, the trash can can be confirmed. When the user clicks the execution control key 1201 (i.e. Figure 12 When the lower control button is shown), the robot arm will do the action of throwing the trash can instead of putting the object on the ground; Figure 13 The top view (point cloud diagram) of the features of the trash can identified by the TOF sensor provided in the embodiment of the present application is as follows: Figure 13 As shown, 1301 is a sweeping robot and 1302 is point cloud data of a trash can. Figure 14 A schematic diagram of a point cloud of a trash can feature identified by a TOF sensor from the perspective of a sweeper provided in an embodiment of the present application is shown as follows: Figure 14 As shown, 1301 is a sweeping robot and 1302 is point cloud data of a trash can.

[0183] When the robot vacuum cleaner determines that there is no trash can feature in front of it, it will detect whether there is an obstacle in front of it, such as a wall / obstacle in front of it. To prevent the robot arm from hitting the wall / obstacle and causing damage, the robot vacuum cleaner will rotate in place to a direction without obstacles and then perform the lowering action. Figure 15 The schematic diagram of the point cloud of the wall features identified by the TOF sensor from the perspective of the sweeper provided in the embodiment of the present application is as follows: Figure 15 As shown, 1301 is a sweeping robot, and 1502 is point cloud data of a wall.

[0184] It should be noted that although the steps of the method of the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all steps must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps; or steps in different embodiments may be combined to form a new technical solution.

[0185] Based on the same inventive concept as the aforementioned method embodiment applied to the robot side, an embodiment of the present application provides a robot control device, which is applied to the robot. Figure 16 Schematic diagram of the structure of the robot control device provided in the embodiment of the present application Figure 1 ;like Figure 16 As shown, the robot's control device 1600 includes:

[0186] A first receiving unit 1601 is configured to receive a first instruction, where the first instruction is used to instruct the robotic arm to automatically dispose of an object, where disposing of the object includes acquiring the object or placing the object;

[0187] The first control unit 1602 is configured to control the robot to move in response to the first instruction, and to control the robot to perform at least one motion action to identify the object to obtain an object recognition result or to identify the front environment to obtain an environment recognition result; wherein the at least one motion action includes at least one of the following: the main body moving forward, backward, or rotating, and the robot arm adjusting its posture;

[0188] The second control unit 1603 is configured to obtain an object according to the object recognition result or place an object according to the environment recognition result.

[0189] The description of the above device embodiment is similar to the description of the above robot-side method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of this application, please refer to the description of the method embodiment of this application for understanding.

[0190] Based on the same inventive concept as the aforementioned method embodiment applied to a remote control device, an embodiment of the present application provides a robot control device, which is applied to a remote control device. Figure 17 Schematic diagram of the structure of the robot control device provided in the embodiment of the present application Figure 2 ;like Figure 17 As shown, the robot's control device 1700 includes:

[0191] The display unit 1701 is configured to display a UI interface, wherein the UI interface includes at least one of the following: a background image, a main viewing area of the robot, prompt information, a position adjustment control key, and an execution control key; wherein the background image is an image captured by the robot at its current position; when the execution control key is used to control automatic object acquisition, the prompt information is used to prompt the robot to move so that the object to be acquired in the background image is within the main viewing area; and the position adjustment control key is used to control the robot body to perform a movement action.

[0192] The first sending unit 1702 is configured to send a first instruction to the robot in response to the touch operation received by the execution control key; wherein the first instruction is used to instruct automatic acquisition of an object or automatic placement of an object.

[0193] The description of the above device embodiment is similar to the description of the method embodiment of the remote control device, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of this application, please refer to the description of the method embodiment of this application for understanding.

[0194] It should be noted that the division of modules in the embodiments of the present application is schematic and is merely a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units. They may also be implemented in the form of a combination of software and hardware.

[0195] It should be noted that, in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling an electronic device to execute all or part of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0196] The embodiment of the present application provides a robot, Figure 18 Schematic diagram of the structure of the robot provided in the embodiment of this application Figure 3 ;like Figure 18 As shown, the robot 100 includes a robotic arm 102, a body 101, a first memory 1801 and a first processor 1802, and the robotic arm 102 is connected to the body 101; the robotic arm 102 is used to obtain and place objects; the first memory 1801 stores a computer program that can be run on the first processor 1802, and when the first processor 1802 executes the program, it implements the steps in the method on the robot side.

[0197] The present application provides a remote control device. Figure 19 A schematic diagram of the structure of the remote control device provided in the embodiment of the present application; Figure 19 As shown, the remote control device 300 includes a second memory 1901 and a second processor 1902. The second memory 1901 stores a computer program that can be run on the second processor 1902. When the second processor 1902 executes the program, the steps in the method on the remote control device side are implemented.

[0198] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program.

[0199] Optionally, the computer-readable storage medium can be applied to the processor or robot in the embodiment of the present application, and the computer program enables the processor or robot to execute the various methods on the robot side of the embodiment of the present application. For the sake of brevity, they are not repeated here.

[0200] Optionally, the computer-readable storage medium can be applied to the processor or remote control device in the embodiment of the present application, and the computer program enables the processor or remote control device to execute the various methods on the remote control device side of the embodiment of the present application. For the sake of brevity, they are not repeated here.

[0201] An embodiment of the present application also provides a computer program product, including computer program instructions.

[0202] Optionally, the computer program product can be applied to the processor or robot in the embodiments of the present application, and the computer program instructions enable the processor or robot to execute the various methods on the robot side of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0203] Optionally, the computer program product can be applied to the processor or remote control device in the embodiments of the present application, and the computer program instructions enable the processor or remote control device to execute the various methods on the remote control device side of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0204] The embodiment of the present application also provides a computer program.

[0205] Optionally, the computer program can be applied to the processor or robot in the embodiments of the present application. When the computer program runs on the processor or robot, the processor or robot executes the various methods on the robot side of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0206] Optionally, the computer program can be applied to the processor or remote control device in the embodiments of the present application. When the computer program runs on the processor or remote control device, the processor or remote control device executes the various methods on the remote control device side of the embodiments of the present application. For the sake of brevity, they are not described here.

[0207] It should be noted that the descriptions of the above storage medium, computer program product, and computer program embodiments are similar to the descriptions of the above method embodiments and have similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium, computer program product, and computer program embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0208] It should be understood that "one embodiment" or "an embodiment" or "some embodiments" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other. For the sake of brevity, they will not be repeated here.

[0209] The term "and / or" in this article is only a description of the association relationship between associated objects, indicating that there can be three relationships. For example, object A and / or object B can mean: object A exists alone, object A and object B exist at the same time, and object B exists alone.

[0210] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, object, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, object, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, object, or apparatus comprising the element.

[0211] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or modules can be electrical, mechanical or other forms.

[0212] The modules described above as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules; they may be located in one place or distributed across multiple network units; some or all of the modules may be selected according to actual needs to achieve the purpose of this embodiment.

[0213] In addition, all functional modules in the embodiments of the present application can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the above-mentioned integrated modules can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0214] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, and other media that can store program codes.

[0215] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling an electronic device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks or optical disks.

[0216] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0217] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0218] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0219] The above is merely an embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A robot control method, characterized in that: The robot includes a body and a robotic arm connected to the body; the method includes: receiving a first instruction for instructing the robotic arm to automatically dispose of an object, wherein disposing of the object includes acquiring the object or placing the object; In response to the first instruction, the robot is controlled to move and to perform at least one motion action to identify the object and obtain an object recognition result or to identify the front environment and obtain an environment recognition result; wherein the at least one motion action includes at least one of the following: the main body moves forward, backward, or rotates, and the robot arm adjusts its posture; The object is acquired or placed according to the recognition result.

2. The method according to claim 1, characterized in that The controlling the robot to perform at least one motion action comprises: When the first instruction indicates that the object is automatically acquired by the robotic arm, the body is controlled to perform at least one of the following motion actions in sequence according to predefined motion rules: retreat a first distance, rotate a first angle in a first direction, and rotate a second angle in a second direction; wherein the second angle is greater than the first angle, and the first direction is opposite to the second direction.

3. The method according to claim 2, characterized in that The controlling the main body to sequentially perform at least one of the following motions according to a predefined motion rule: retreating a first distance, rotating in a first direction by a first angle, and rotating in a second direction by a second angle includes: Identifying the environment behind the fuselage body and determining a second distance; wherein the second distance is the distance between the fuselage body and a rear obstacle, or the second distance is the distance between the fuselage body and an edge of a rear plane, where the plane is the plane on which the fuselage body is located; If the second distance is greater than or equal to the first threshold, the fuselage body is controlled to perform at least one of the following motion actions in sequence according to a predefined motion rule: retreat a first distance, rotate a first angle in a first direction, and rotate a second angle in a second direction.

4. The method according to any one of claims 1 to 3, characterized in that The controlling the robot to perform at least one motion action includes at least one of the following steps, or the method further includes at least one of the following steps before receiving the first instruction: receiving a second instruction, and controlling the fuselage body to retreat a third distance according to the second instruction; receiving a third instruction, and controlling the main body of the fuselage to move forward a fourth distance according to the third instruction; receiving a fourth instruction, and controlling the main body of the fuselage to rotate in the first direction by a third angle according to the fourth instruction; receiving a fifth instruction, and controlling the main body of the fuselage to rotate in the second direction by a fourth angle according to the fifth instruction; Receive a posture adjustment instruction, and adjust the posture of the robotic arm according to the posture adjustment instruction; wherein, the robotic arm is provided with at least one sensor for collecting environmental data.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: collecting environmental data using at least one sensor on the fuselage body, and / or collecting environmental data using at least one sensor on the robotic arm; The object is identified according to the collected environmental data to obtain an object identification result, or the front environment is identified to obtain an environment identification result.

6. The method according to any one of claims 1 to 5, characterized in that The acquiring of the object according to the object recognition result includes: Determining a target stopping position of the fuselage body and a target posture of the robotic arm based on the object recognition result; Controlling the body to move to the target rest position, and controlling the robotic arm to adjust to the target posture; After controlling the body to move to the target rest position and controlling the robotic arm to adjust to the target posture, the end effector of the robotic arm is controlled to perform an acquisition action.

7. The method according to any one of claims 1 to 6, characterized in that The environment recognition result includes the number of sampling points in the point cloud and the height of the point cloud; Placing the object according to the environment recognition result includes: If it is determined that the point cloud satisfies a preset condition according to the number of sampling points, and the height of the point cloud is within a predefined first height range, it is determined that a storage container exists in the front environment; The robotic arm is controlled to place the acquired object in the storage container.

8. The method according to any one of claims 1 to 7, characterized in that The environment recognition result includes the number of sampling points in the point cloud and the height of the point cloud; Placing the object according to the environment recognition result includes: If it is determined that the point cloud satisfies a preset condition according to the number of sampling points, and the height of the point cloud is within a predefined second height interval, determining that an obstacle exists in the forward environment; The body is controlled to rotate or move, or the robotic arm is controlled to rotate, so that the robotic arm is not blocked by obstacles during the placement of the object.

9. The method according to any one of claims 1 to 8, characterized in that The received command is sent based on the touch operation received by the control key of the UI interface.

10. A robot control device, characterized in that: The device is applied to a robot, which includes a body and a robotic arm connected to the body; the device includes: a first receiving unit configured to receive a first instruction, wherein the first instruction is used to instruct the robotic arm to automatically dispose of an object, wherein disposing of the object includes acquiring the object or placing the object; a first control unit configured to control the robot to move in response to the first instruction, and to control the robot to perform at least one motion action to identify the object to obtain an object recognition result or to identify the front environment to obtain an environment recognition result; wherein the at least one motion action includes at least one of the following: the main body moving forward, backward, or rotating, and the robot arm adjusting its posture; The second control unit is configured to obtain or place the object according to the recognition result.

11. The device according to claim 10, characterized in that The controlling the robot to perform at least one motion action comprises: When the first instruction indicates that the object is automatically acquired by the robotic arm, the body is controlled to perform at least one of the following motion actions in sequence according to predefined motion rules: retreat a first distance, rotate a first angle in a first direction, and rotate a second angle in a second direction; wherein the second angle is greater than the first angle, and the first direction is opposite to the second direction.

12. The device according to claim 11, characterized in that The controlling the main body to sequentially perform at least one of the following motions according to a predefined motion rule: retreating a first distance, rotating in a first direction by a first angle, and rotating in a second direction by a second angle includes: Identifying the environment behind the fuselage body and determining a second distance; wherein the second distance is the distance between the fuselage body and a rear obstacle, or the second distance is the distance between the fuselage body and an edge of a rear plane, where the plane is the plane on which the fuselage body is located; If the second distance is greater than or equal to the first threshold, the fuselage body is controlled to perform at least one of the following motion actions in sequence according to a predefined motion rule: retreat a first distance, rotate a first angle in a first direction, and rotate a second angle in a second direction.

13. The device according to any one of claims 10 to 12, characterized in that The controlling the robot to perform at least one motion action includes at least one of the following steps, or the device further performs at least one of the following steps before receiving the first instruction: receiving a second instruction, and controlling the fuselage body to retreat a third distance according to the second instruction; receiving a third instruction, and controlling the main body of the fuselage to move forward a fourth distance according to the third instruction; receiving a fourth instruction, and controlling the main body of the fuselage to rotate in the first direction by a third angle according to the fourth instruction; receiving a fifth instruction, and controlling the main body of the fuselage to rotate in the second direction by a fourth angle according to the fifth instruction; Receive a posture adjustment instruction, and adjust the posture of the robotic arm according to the posture adjustment instruction; wherein, the robotic arm is provided with at least one sensor for collecting environmental data.

14. The device according to any one of claims 10 to 13, characterized in that The device further comprises a data acquisition unit and an identification unit; wherein: The data acquisition unit is configured to acquire environmental data using at least one sensor on the fuselage body and / or to acquire environmental data using at least one sensor on the robotic arm; The recognition unit is configured to recognize the object according to the collected environmental data to obtain an object recognition result or recognize the front environment to obtain an environment recognition result.

15. The device according to any one of claims 10 to 14, characterized in that The acquiring of the object according to the object recognition result includes: Determining a target stopping position of the fuselage body and a target posture of the robotic arm based on the object recognition result; Controlling the body to move to the target rest position, and controlling the robotic arm to adjust to the target posture; After controlling the body to move to the target rest position and controlling the robotic arm to adjust to the target posture, the end effector of the robotic arm is controlled to perform an acquisition action.

16. The device according to any one of claims 10 to 15, characterized in that The environment recognition result includes the number of sampling points in the point cloud and the height of the point cloud; Placing the object according to the environment recognition result includes: If it is determined that the point cloud satisfies a preset condition according to the number of sampling points, and the height of the point cloud is within a predefined first height range, it is determined that a storage container exists in the front environment; The robotic arm is controlled to place the acquired object in the storage container.

17. The device according to any one of claims 10 to 16, characterized in that The environment recognition result includes the number of sampling points in the point cloud and the height of the point cloud; Placing the object according to the environment recognition result includes: If it is determined that the point cloud satisfies a preset condition according to the number of sampling points, and the height of the point cloud is within a predefined second height interval, determining that an obstacle exists in the forward environment; The body is controlled to rotate or move, or the robotic arm is controlled to rotate, so that the robotic arm is not blocked by obstacles during the placement of the object.

18. The device according to any one of claims 10 to 17, characterized in that The received command is sent based on the touch operation received by the control key of the UI interface.

19. A robot comprising a robotic arm, a body, a first memory, and a first processor, wherein: The robotic arm is connected to the fuselage body; the robotic arm is used to obtain and place objects; the first memory stores a computer program that can be run on a first processor, and when the first processor executes the program, it implements the steps in the method as described in any one of claims 1 to 9.

20. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instructions are executed by a processor or a robot, the steps in the method according to any one of claims 1 to 9 are implemented.

Images