Control method and apparatus for movable device, movable device, and storage medium

Abstract

The present disclosure relates to the field of smart homes, and provides a control method and apparatus for a movable device, a movable device, and a storage medium. The method comprises: determining at least two gripping points on a target object to be moved; synchronously controlling a mechanical arm of each movable device among at least two movable devices to correspondingly grip one gripping point and lift the target object in the vertical direction; and once the target object has been lifted in place, synchronously controlling a chassis of each movable device to move towards a target position, so as to move the target object to the target position.

Description

Control methods, devices, mobile devices, and storage media for mobile devices

[0001] Cross-reference of related applications

[0002] This disclosure is based on and claims priority to Chinese Patent Application No. 202411534362.9, filed on October 30, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to the field of smart homes, and in particular to a method, apparatus, mobile device, and readable storage medium for controlling a mobile device. Background Technology

[0004] Robotic vacuum cleaners with robotic arms are household robots developed based on the chassis and robotic arms of traditional robotic vacuum cleaners. They can clean floors and organize items on the floor. Compared to traditional robotic vacuum cleaners, which can only avoid obstacles and thus miss large areas, these household robots, with the assistance of their robotic arms, can grasp and move obstacles to clean the area under and around the obstacle, improving floor coverage. Furthermore, these robots can automatically organize themselves; for example, they can automatically identify and mark obstacles on the floor while cleaning, and then automatically sort and organize them after cleaning. Summary of the Invention

[0005] In view of this, the present disclosure provides a control method, apparatus, mobile device, and readable storage medium for a mobile device.

[0006] In a first aspect, embodiments of this disclosure provide a control method for a mobile device, the method comprising:

[0007] Identify at least two gripping points on the target object to be transported;

[0008] The robotic arm of each of at least two mobile devices is synchronously controlled to grasp one of the gripping points and lift the target object in the vertical direction.

[0009] After the target object is lifted into position, the chassis of each mobile device is simultaneously controlled to move toward the target position in order to transport the target object to the target position.

[0010] Secondly, embodiments of this disclosure provide a control device for a mobile device, the device comprising:

[0011] The gripping point determination module is used to determine at least two gripping points on the target object to be transported.

[0012] The control module is used for:

[0013] The robotic arm of each of at least two mobile devices is synchronously controlled to grasp one of the gripping points and lift the target object in the vertical direction.

[0014] After the target object is lifted into position, the chassis of each mobile device is simultaneously controlled to move toward the target position in order to transport the target object to the target position.

[0015] Thirdly, embodiments of this disclosure provide a mobile device including a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the method as described in the first aspect.

[0016] Fourthly, embodiments of this disclosure provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method as described in the first aspect.

[0017] Fifthly, embodiments of this disclosure provide a computer program product including computer program instructions stored in a computer-readable storage medium and adapted to be invoked and executed by a processor to cause a computer device having the processor to perform the steps of the method described in the first aspect.

[0018] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, specific embodiments of this disclosure are described below. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this disclosure, illustrate exemplary embodiments of the present disclosure and are used to explain the disclosure, but do not constitute an undue limitation of the disclosure. In the drawings:

[0020] Figure 1 shows a schematic diagram of the structure of a mobile device according to an embodiment of the present disclosure;

[0021] Figure 2 shows one of the flowcharts illustrating a control method for a mobile device according to an embodiment of the present disclosure;

[0022] Figure 3 shows a schematic diagram of determining the gripping point according to an embodiment of the present disclosure;

[0023] Figure 4 shows a schematic diagram of the movement to the vicinity of the gripping point according to an embodiment of the present disclosure;

[0024] Figure 5 shows a schematic diagram of gripping and lifting the gripping point according to an embodiment of the present disclosure;

[0025] Figure 6 shows a schematic diagram of movement to the target location according to an embodiment of the present disclosure;

[0026] Figure 7 shows a second schematic flowchart of a control method for a mobile device according to an embodiment of the present disclosure;

[0027] Figure 8 shows a schematic diagram of a narrow space according to an embodiment of the present disclosure;

[0028] Figure 9 shows a structural block diagram of a control device for a mobile device according to an embodiment of the present disclosure;

[0029] Figure 10 shows a structural block diagram of a mobile device according to an embodiment of the present disclosure. Detailed Implementation

[0030] The technical solutions of the embodiments of this disclosure will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure are within the scope of protection of this disclosure.

[0031] The terms "first," "second," etc., used in this disclosure and in the claims are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this disclosure can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or at least two. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0032] In related technologies, the capabilities of a single mobile device, such as a household robot, are limited. It struggles to grip large or heavy objects, hindering its ability to complete tasks. For example, using only a single mobile device might result in the object being too large and obstructing the sensors on the robot's chassis, preventing it from functioning properly. Similarly, using only a single mobile device to handle objects exceeding its rated load, such as a weighing scale, might result in failure to grip the object or frequent drops, further impairing its task completion.

[0033] This disclosure provides a scheme for multiple mobile devices to collaboratively grip and transport objects. Through the collaboration of multiple mobile devices, objects that are large in size or weight can be successfully gripped and transported, thereby improving the task completion rate.

[0034] The control method, apparatus, mobile device, and readable storage medium of the present disclosure are described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0035] The mobile device exists in an indoor space, as shown in Figure 1. The mobile device includes a chassis 101, wheels 102, casters 103, and a robotic arm 104 mounted on the chassis 101.

[0036] In one embodiment, the chassis 101 can achieve linear movement along the x-axis and y-axis, linear movement along the z-axis, and rotation along the z-axis via the wheels 102 and casters 103. The x-axis and y-axis are parallel to the horizontal plane (i.e., the ground), and the z-axis is perpendicular to the horizontal plane. The linear movement of the chassis 101 along the z-axis, i.e., the adjustment of the chassis 101's ground clearance, can be achieved through the wheels 102 and / or the casters 103. The wheels 102 and casters 103 can be raised and lowered individually or together, thereby adjusting the chassis 101's ground clearance.

[0037] The end effector of the robotic arm 104 of the mobile device is equipped with a gripper and a camera. The robotic arm 104 includes multiple rotating shafts and connecting rods. The number of rotating shafts determines the flexibility of the robotic arm 104. The robotic arm 104 can be a multi-degree-of-freedom robotic arm, such as three-degree-of-freedom, four-degree-of-freedom, five-degree-of-freedom, etc. The specific number of degrees of freedom is not specifically limited in this disclosure.

[0038] The rotating shaft M1 can rotate the robotic arm 104 to the left or right (in Figure 1, the paper-faced direction is inward or outward). The rotating shaft M2 rotates in the same direction as the rotating shafts M3 and M4. The rotating shafts M1 and M2 are mainly responsible for the robotic arm 104's exit and return. After the robotic arm 104 exits the compartment, the rotating shafts M1 and M2 maintain a fixed angle. A compartment for retrieving the robotic arm 104 can be set on the chassis 101. The robotic arm 104 can be folded and stored in the compartment; this state is the robotic arm 104 returning to the compartment. When the robotic arm is unfolded, it can extend from the compartment; this state is the robotic arm 104 exiting the compartment. The rotating shafts M3 and M4 determine the position of the gripper of the robotic arm 104 in space, and the rotating shaft M5 is the gripper's spin joint, which determines the gripper's posture.

[0039] In one embodiment, the mobile device further includes a camera mounted on the chassis 101 and / or a camera mounted on the robotic arm 104. The camera can be used to identify objects, and after the object is identified, it can be gripped and transported with the assistance of the robotic arm 104.

[0040] In one embodiment, an indoor space may include at least two mobile devices. When a single mobile device cannot handle the gripping and moving of an object, multiple mobile devices can work together to achieve this. These mobile devices can be intelligent cleaning equipment, including robotic vacuum cleaners, robotic mops, combined vacuum and mop machines, and household robots. Through the collaboration of multiple mobile devices, actions such as gripping and moving large or heavy objects can be performed to clean the bottom and surrounding area of ​​the object, increasing the floor cleaning coverage. Alternatively, large or heavy objects can be organized, improving the overall tidying effect of the room.

[0041] This disclosure provides a control method for a mobile device, as shown in FIG2, the method comprising:

[0042] Step 201: Determine at least two gripping points on the target object to be transported.

[0043] In this step, at least two gripping points are determined for the target object that needs to be moved by the mobile device. By determining appropriate gripping points on the target object, the success rate of gripping the target object is increased and the probability of the target object falling during movement is reduced.

[0044] In one embodiment of this disclosure, before determining at least two gripping points on the target object to be transported, the method further includes: controlling at least two mobile devices to acquire object feature information from different angles, and identifying the target object to be transported based on the object feature information from different angles.

[0045] In this embodiment of the disclosure, before grasping an object, multiple mobile devices collaborate to identify object feature information from various angles, and construct a three-dimensional spatial model based on the object feature information to obtain the target object. In some embodiments, object feature information can be identified by sensors such as ultrasonic sensors, infrared ranging sensors, lidar, and vision sensors. Object feature information includes, but is not limited to, shape and size features, material and surface features, color and texture features, position and dynamic features, etc.

[0046] When controlling at least two mobile devices to acquire object feature information, the mobile devices can be moved to the vicinity of the target object to collect information. If the distance to the object is close, they can remain stationary and collect information from the object in place. Regardless of the method, it is necessary to ensure that mobile devices are collecting information from multiple different angles of the target object.

[0047] This embodiment of the invention enables the collection of information about a target object from different angles using at least two mobile devices, thereby more accurately identifying the target object, improving the accuracy of the target object identification, and thus enabling smoother gripping and handling of the target object.

[0048] In one embodiment of this disclosure, at least two gripping points are determined on the target object to be transported, including:

[0049] Generate the external model of the target object based on its contour;

[0050] For the circumscribed model, divide it horizontally into N1 equal parts with the center of the circle as the center, resulting in N2 equal dividing lines, where N1 is an integer greater than or equal to 2 and N2 is an integer greater than or equal to 1.

[0051] For each bisector, determine the intersection point between the bisector and the contour of the target object, and determine the clamping point based on the intersection point.

[0052] In some embodiments, N1 is greater than or equal to N2.

[0053] In this embodiment of the disclosure, after identifying the target object, a circumscribed model of the target object is generated based on its surface contour. This circumscribed model can be a circumscribed circle, a circumscribed cylinder, or a circumscribed sphere, etc. For example, if the height of the target object is less than a second preset height, that is, the target object is thinner in the vertical direction and is a planar object, then a circumscribed circle of the target object can be generated based on its contour. If the height of the target object is greater than or equal to the second preset height, and the target object is a three-dimensional object, then a circumscribed cylinder or a circumscribed sphere of the target object can be generated based on its contour. The second preset height can be the gripper stroke of the robotic arm's gripper, where gripper stroke refers to the maximum distance between the gripper fingertips or the gripper gripping surface after the gripper is opened.

[0054] The circumscribed model is divided into N1 equal parts horizontally, centered on the center of the circle, resulting in N2 dividing lines. N1 is an integer greater than or equal to 2, and N2 is an integer greater than or equal to 1. When the circumscribed model is a circumcircle, circumcylinder, or circumsphere, its projection onto the horizontal plane is a circle, with the center being the center of this projection. Specifically, for a circumcircle, the center is its geometric center; for a circumcylinder, it's the projection of its central axis onto the horizontal plane; and for a circumsphere, it's the projection of its center onto the horizontal plane. Dividing the circumscribed model horizontally into N1 equal parts centered on the center of the circle results in N2 dividing lines that divide the projection of the circumscribed model onto the horizontal plane into N1 sector-shaped regions.

[0055] In one embodiment, when the circumscribed model is a circumcircle, each dividing line can be understood as a line from the center of the circle to a point on the outline of the circumcircle. When the circumscribed model is a 3D model such as a circumscribed cylinder or circumscribed sphere, each dividing line can be understood as an infinitely high, vertical plane whose projection onto a horizontal plane is a line from the center of the circle to a target point, which is a point on the outline of the projection of the circumscribed model onto the horizontal plane. The projections of N² such planes onto the horizontal plane divide the projection of the circumscribed model onto the horizontal plane into N¹ sector regions. In this case, the values ​​of N¹ and N² can be equal.

[0056] In another embodiment, when the circumscribed model is a circumscribed circle, each dividing line can be understood as a line passing through the center of the circle between any two points on the outline of the circumscribed circle. When the circumscribed model is a 3D model such as a circumscribed cylinder or a circumscribed sphere, each dividing line can be understood as an infinitely high, vertical plane. The projection of this plane onto the horizontal plane is a line passing through the center of the circle between the first and second points, where the first and second points are any two points on the outline of the projection of the circumscribed model onto the horizontal plane. The projections of N2 such planes onto the horizontal plane divide the projection of the circumscribed model onto the horizontal plane into N1 sector regions. In this case, the value of N1 is greater than the value of N2.

[0057] Each bisector obtained in the above manner intersects the surface contour of the target object, and the gripping point can be determined based on the intersection of the bisector and the contour of the target object. Alternatively, the direction of the bisector towards the center of the circle can be used as the gripping direction of the robotic arm of each mobile device or the direction of movement of the chassis of each mobile device toward the target object.

[0058] It is understandable that N1 and N2 can be the number of usable mobile devices, or N1 and N2 can be preset values ​​that are related to the properties of the target object. For example, they can be the number of easy gripping positions of the target object. Furthermore, the larger the target object, the larger the values ​​of N1 and N2. That is, more gripping points are determined for larger objects to ensure the stability of transporting larger objects.

[0059] In one embodiment, determining the gripping point based on the intersection includes: if the height of the target object is less than a second preset height, meaning the target object is thinner in the vertical direction, then the intersection of the bisector and the upper edge contour of the target object is determined. The x and y coordinates of the projection point of this intersection on the horizontal plane are used as the x and y coordinates of the gripping point; the z coordinate does not need to be determined to obtain the gripping point. If the height of the target object is greater than or equal to the second preset height, then the intersection of the bisector and the upper, lower, or outermost edge contour of the target object is determined. The x and y coordinates of the projection point of this intersection on the horizontal plane are used as the x and y coordinates of the gripping point. A z coordinate can be randomly determined on the intersection line of the bisector corresponding to the projection point and the contour of the target object. Alternatively, based on the shape of the target object near the position of the bisector corresponding to the projection point, an easy gripping position can be found to determine the z coordinate. Finally, after determining the x, y, and z coordinates, the gripping point is obtained. It should be noted that the gripping points may or may not be on the same height plane.

[0060] The upper edge contour can refer to the edge contour of the upper surface of the target object. The lower edge contour can refer to the edge contour of the lower surface of the target object. The outermost edge contour can refer to the outer boundary line formed by the projection of the target object onto the horizontal plane.

[0061] It should be noted that when the values ​​of N1 and N2 are equal, the number of intersections between each bisector and the upper edge, lower edge, or outermost edge of the target object is 1; when the value of N1 is greater than the value of N2, the number of intersections between each bisector and the upper edge, lower edge, or outermost edge of the target object is 2.

[0062] Taking the collaborative lifting of a simple planar irregular object by four mobile devices as an example, as shown in Figure 3 (the outline of the target object in Figure 3 is the upper edge outline of the target object), the circumcircle 300 of the target object 200 is calculated based on its three-dimensional information. The circumcircle 300 is divided into four equal parts. The direction from each dividing line to the center of the circle is taken as the direction of movement of each mobile device toward the target object 200. The intersection of each dividing line with the upper edge outline of the target object 200 is taken as each gripping point, namely A1, A2, A3 and A4.

[0063] In this embodiment of the disclosure, multiple mobile devices model the target object. By using circumscribed models such as circumscribed circles, circumscribed cylinders, or circumscribed spheres and bisectors, gripping points are determined that are evenly distributed around the target object. This ensures that when multiple mobile devices grip the object according to the gripping points, the resultant force is upward and the resultant torque is as close to zero as possible, so as to stably lift the target object.

[0064] In one embodiment of this disclosure, the method further includes: after determining the gripping point, if the shape of the gripping point position on the target object and / or the gripper stroke of the robotic arm of the movable device do not meet the gripping conditions, then a new gripping point is determined within a preset distance range of the gripping point, and the new gripping point is used to replace the gripping point; or, the circumscribed model is re-divided into M equal parts in the vertical direction, thereby re-determining M gripping points, where M is an integer greater than or equal to 2, and M is equal to or not equal to N1. The method of dividing the circumscribed model into M equal parts in the vertical direction can be, for example, finding the center point of the projection of the circumscribed model onto a certain vertical plane, and dividing the circumscribed model into M equal parts in the vertical direction based on this center point. The specific process is similar to the method of dividing the circumscribed model into N1 equal parts in the horizontal direction with the center of a circle as the center, and will not be described in detail here.

[0065] In this embodiment of the disclosure, after determining the gripping point, if the shape of the gripping point location on the target object and / or the gripper stroke of the robotic arm of the movable device do not meet the gripping conditions, then a replacement gripping point is determined around the gripping point, or the circumscribed model is re-divided equally. For example, after randomly determining the z-coordinate of the gripping point on the intersection of the bisector corresponding to the projection point and the contour, if the shape of the gripping point location is one that is difficult to grip, then it is determined that the gripping conditions are not met; as another example, after randomly determining the z-coordinate of the gripping point on the intersection of the bisector corresponding to the projection point and the contour, or after determining the z-coordinate of the gripping point based on the shape of the target object near the bisector location, if it is determined that the width or height of the gripping point location exceeds the gripper stroke of the robotic arm, then it is determined that the gripping conditions are not met.

[0066] When determining replacement grip points around a previous grip point, they can be determined on the intersection line corresponding to the previous grip point, that is, above or below the previous grip point. Alternatively, they can be determined outside the intersection line, for example, to the left or right of the previous grip point. Of course, to ensure that the grip points are evenly distributed on the target object, it is preferable to determine the new grip points on the intersection line corresponding to the previous grip point.

[0067] When redividing the circumscribed model to redetermine the grip points, the redividing of the circumscribed model can be either dividing it into N1 equal parts (M equals N1) or dividing it into N1 unequal parts (M does not equal N1).

[0068] In this embodiment of the disclosure, after determining the gripping point, the determined gripping point can be confirmed. If the determined gripping point is not suitable, the gripping point is re-determined to ensure that the gripping points are evenly distributed around the target object and that the grippers can grip it.

[0069] Step 202: Synchronously control the robotic arm of each of at least two mobile devices to grasp a corresponding gripping point and lift the target object in the vertical direction.

[0070] In this step, the robotic arm of each of at least two mobile devices is controlled to grasp a corresponding gripping point. For example, as shown in Figure 4, there are four mobile devices (B1, B2, B3, B4) and four gripping points (A1, A2, A3, A4). Each mobile device grips one gripping point, and all four mobile devices grip the gripping points simultaneously. After gripping the gripping points, the target object is simultaneously lifted vertically.

[0071] In one embodiment of this disclosure, synchronously controlling the robotic arm of each of at least two mobile devices to grasp a corresponding gripping point includes:

[0072] For each mobile device, determine the gripping point corresponding to the mobile device, and control the mobile device to move to a preset spatial position in front of its corresponding gripping point;

[0073] After reaching the preset spatial position, the robotic arm of the movable device is controlled to face the center of the target object and adjusted to the preset gripping angle to grasp the corresponding gripping point of the movable device.

[0074] In this embodiment of the disclosure, after determining the positions of each gripping point on the target object, for each mobile device, the gripping point corresponding to the mobile device is determined, that is, the gripping point that the mobile device needs to grip, and the mobile device is controlled to move to a preset spatial position in front of its corresponding gripping point.

[0075] In one embodiment of this disclosure, determining the gripping point corresponding to the mobile device includes: aiming to minimize the travel distance or travel time to the preset spatial position of the gripping point, thus determining the gripping point corresponding to the mobile device. In this embodiment, for each mobile device, the travel distance or travel time between the mobile device's current position and the preset spatial positions of each gripping point is planned, and the gripping point corresponding to the shortest travel distance or travel time among multiple travel distances or travel times is determined as the gripping point that the mobile device needs to grip. This method ensures that the travel distance or travel time of the mobile device in the process of reaching the gripping point is minimized, reducing the energy consumption of the mobile device.

[0076] Furthermore, after the mobile device reaches the preset spatial position, as shown in Figure 4, the robotic arm of the mobile device is controlled to face the center of the target object. For example, the robotic arm can be oriented towards the center of the target object by controlling the chassis of the mobile device. The axis of the robotic arm is also controlled to rotate to a suitable gripping angle (i.e., the preset gripping angle), and the chassis of the mobile device is controlled to move, thereby grasping the corresponding gripping point of the mobile device. The arrow in Figure 4 indicates the direction of movement of the chassis of the mobile device.

[0077] In this embodiment of the disclosure, for each mobile device, its corresponding gripping point is determined, and the mobile device is controlled to move in front of the gripping point to grip the gripping point. This achieves precise control of the mobile device to grip the gripping point, ensuring the efficiency of transporting the target object.

[0078] In one embodiment of this disclosure, lifting the target object in the vertical direction includes:

[0079] After the robotic arm of the mobile device grasps the corresponding gripping point, the robotic arm of each mobile device is controlled to lift the target object in the vertical direction.

[0080] During the lifting process, the first speed at which each robotic arm moves upward is controlled to meet a first preset condition, so that the difference in the vertical movement distance of the grippers of each robotic arm is less than a first preset height. The first preset condition includes that the first speed at which each robotic arm moves upward is the same or the speed difference is less than the first preset speed.

[0081] In this embodiment, after all the grippers of the robotic arms of the mobile devices have successfully gripped the target object, as shown in Figure 5, each gripper simultaneously lifts upwards (as indicated by the arrow), and it is necessary to ensure that the target object remains stable during the lifting process. A stable target object means that, during the lifting process, the difference in the vertical movement distance of each robotic arm's gripper, i.e., the difference in the vertical movement distance of each gripping point, is less than a first preset height. In other words, if the gripping points are on the same horizontal plane at the initial gripping stage, then the distances moved by each gripping point during the lifting process should be as similar as possible, thus keeping the gripping points on the same horizontal plane; if there is a height difference in the vertical direction at the initial gripping stage, then the distances moved by each gripping point during the lifting process should be as similar as possible, thus keeping the gripping points at the same height difference.

[0082] During control, the target object is kept stable during the lifting process by controlling the first speed at which each robotic arm moves upward to be the same or the speed difference to be less than the first preset speed.

[0083] In this embodiment of the disclosure, each mobile device is controlled to simultaneously grip and lift the target object to ensure the stability of the target object and prevent it from tilting.

[0084] In one embodiment of this disclosure, controlling the first speed at which each robotic arm moves upward satisfies a first preset condition, including:

[0085] The robotic arm pose information of the mobile devices is transmitted through communication between the various mobile devices.

[0086] Based on the pose information of the robotic arm of each mobile device, the difference in the vertical movement distance of the gripper of each robotic arm is determined.

[0087] The speed of the robotic arms is controlled by negative feedback based on the difference in movement distance, so that the first speed of each robotic arm moving upward meets the first preset condition.

[0088] In this embodiment of the disclosure, each mobile device can communicate wirelessly via WiFi, Bluetooth or radio frequency to exchange chassis pose information and robotic arm pose information, and convert the chassis pose information and robotic arm pose information to the same world coordinate system to achieve unified control of the mobile devices.

[0089] For any mobile device, based on its robotic arm pose information, the vertical movement distance of its grippers is calculated, and then the difference in movement distance between each gripper is calculated. Based on this difference in movement distance, negative feedback control of the robotic arm's speed is applied, ensuring that the initial upward movement speed of each robotic arm meets a first preset condition, thus achieving a smooth lifting of the target object.

[0090] For example, if the difference in the moving distance of the grippers of two mobile devices is greater than a preset value, the lifting speed of the gripper with the smaller moving distance will be increased and the lifting speed of the gripper with the larger moving distance will be decreased, so as to maintain the resultant force direction upward and the resultant torque small, thereby achieving the smooth lifting of the target object.

[0091] In one embodiment, the robot arm pose information can be determined based on visual information, or the robot arm is at a fixed height at the position connected to the main body, and the robot arm pose information can be calculated based on the rotation angle of each axis and the length of the link.

[0092] In one embodiment, by communicating with each other, the lifting speed information of the grippers of the mobile devices can also be transmitted, so that the lifting speed of each gripper is the same, thereby achieving smooth lifting of the target object.

[0093] In one embodiment of this disclosure, the method further includes: while the robotic arm lifts the target object in the vertical direction, controlling the chassis of the mobile device to move toward the center of the target object, so as to compensate for the coordinate changes of the gripper of the robotic arm in the horizontal direction.

[0094] In this embodiment of the present disclosure, while controlling the robotic arm to lift the target object in the vertical direction, the chassis of the mobile device is controlled to move towards the center of the target object, thereby compensating for the coordinate changes of the gripper of the robotic arm in the horizontal direction, so that each gripper and its corresponding gripping point only move upward at the same speed, without horizontal movement, thus ensuring the stability of the target object.

[0095] Step 203: After the target object is lifted into place, the chassis of each mobile device is simultaneously controlled to move toward the target position in order to transport the target object to the target position.

[0096] In this step, after the target object is lifted into position, the chassis of each mobile device is controlled to move synchronously toward the target location, thereby transporting the target object to the target location. Once the target location is reached, the target object is lowered. The descent of the target object also needs to be smooth. The method to ensure a smooth descent is the same as that for the lifting process, and will not be repeated here.

[0097] It should be noted that the mobile device used for target object information recognition can be all or some of the mobile devices present in the indoor space, and the mobile device used for picking up and moving the target object can be all or some of the mobile devices present in the indoor space. Furthermore, the mobile device used for target object information recognition and the mobile device used for picking up and moving the target object can be the same or different.

[0098] For coordinated control of at least two mobile devices, one master mobile device can control itself and other mobile devices to collaboratively grip and move a target object. Alternatively, in a clockwise or counterclockwise direction, the preceding mobile device transmits a gripping and moving signal to the next mobile device to achieve coordinated gripping and moving of the target object.

[0099] In one embodiment of this disclosure, determining the target object to be raised into position includes: when at least two mobile device robotic arms smoothly raise the target object in the vertical direction, and each gripping point of the target object is raised by the same preset distance, the target object is determined to be raised into position.

[0100] In this embodiment of the disclosure, by controlling the lifting speed, the lifting process of the robotic arm on the target object in the vertical direction is made smooth. Under this premise, when each gripping point of the target object is lifted by the same preset distance, that is, when each gripping point has moved the same distance, the target object is determined to be lifted into place.

[0101] It should be noted that the preset distance can be determined based on factors such as the weight of the target object, the mechanical structure of the robotic arm, and the home environment, such as the height of the threshold. Among these factors, the threshold height limits the minimum preset distance, while the mechanical structure of the robotic arm limits the maximum preset distance.

[0102] In one embodiment of this disclosure, after the target object is lifted into position, the chassis of each mobile device is simultaneously controlled to move toward the target location to transport the target object to the target location, including:

[0103] After the target object is lifted into position, control the chassis of each mobile device to move to the transport posture;

[0104] Synchronously control each chassis to move towards the target position, and during the movement, control the second speed of each chassis to meet the second preset condition. The second preset condition includes that the second speeds of each chassis are the same or the speed difference is less than the second preset speed.

[0105] In this embodiment, after the target object is lifted into position, the chassis of each mobile device is controlled to move into a transport posture, as shown in Figure 6, with each mobile device facing the target position. Then, the chassis move synchronously towards the target position (as indicated by the arrow) to achieve the transport action of the object. During the movement, the second speed of each chassis is controlled to meet a second preset condition so that the target object remains stable during the movement.

[0106] In one embodiment of this disclosure, the method of controlling the second speed of movement of each chassis to satisfy a second preset condition includes:

[0107] The chassis pose information of the mobile devices is transmitted through communication between the various mobile devices.

[0108] During the movement, negative feedback control of the chassis speed is performed based on the chassis pose information of each mobile device, so that the second speed of each chassis movement meets the second preset condition.

[0109] In this embodiment of the disclosure, each mobile device can communicate wirelessly via WiFi, Bluetooth or radio frequency to exchange chassis pose information and robotic arm pose information, and convert the chassis pose information and robotic arm pose information to the same world coordinate system to achieve unified control of the mobile devices.

[0110] For any mobile device, the horizontal movement distance of its chassis is calculated based on its chassis pose information, and then the difference in movement distance between each chassis is calculated. Negative feedback control of the chassis speed is then applied based on this difference in movement distance, ensuring that the second speed of each chassis' horizontal movement meets a second preset condition, thus achieving smooth transport of the target object.

[0111] According to the embodiments of this disclosure, objects in a room can be gripped and moved by at least two mobile devices working together. This is especially true for larger or heavier objects, enabling smooth gripping and moving and improving the completion rate of tasks such as cleaning and tidying.

[0112] As a refinement and extension of the above embodiments, this disclosure provides another method for controlling a mobile device, as shown in FIG7, the method including:

[0113] Step 701: Determine at least two gripping points on the target object to be transported.

[0114] Step 702: Synchronously control the robotic arm of each of at least two mobile devices to grasp a corresponding gripping point and lift the target object in the vertical direction.

[0115] Step 703: After the target object is lifted into place, the chassis of each mobile device is simultaneously controlled to move toward the target position in order to transport the target object to the target position.

[0116] Step 704: During the process of transporting the target object, if it is necessary to pass through a narrow space, control the chassis of each mobile device to move towards the center of the target object, and control the robotic arms of each mobile device to move upward synchronously to compensate for the coordinate changes of the gripper of the robotic arm in the horizontal direction.

[0117] In this embodiment, environmental detection is performed during the transport of a target object by the mobile device to detect whether there are narrow spaces, such as passageways, along the movement route. A narrow space refers to a space whose width is less than or equal to the farthest distance between two opposing mobile devices during the current transport of the target object. That is, the width of the narrow space is less than or equal to the farthest distance between the two opposing mobile devices before they move towards the center of the target object. The existence of a narrow space can be determined by comparing the width of the space with the farthest distance between the two opposing mobile devices before they move towards the center of the target object. If a narrow space is determined to exist and passage through it is necessary, the chassis of each mobile device is controlled to move towards the center of the target object, that is, to move an appropriate distance closer to the center of the outermost model of the target object, so that the farthest distance between the two opposing mobile devices is less than the width of the narrow space, ensuring smooth passage.

[0118] It's worth noting that after determining that a narrow space exists along the movement route, it's further possible to determine whether the narrow space can be successfully traversed if each mobile device's chassis moves towards the center of the target object to its maximum movable distance, thereby reducing the overall width. If, after moving to its maximum movable distance, the furthest distance between two relatively opposite mobile devices is less than the width of the narrow space, then the narrow space can be traversed successfully. If, after moving to its maximum movable distance, the furthest distance between two relatively opposite mobile devices is still greater than the width of the narrow space, then the narrow space cannot be traversed. The maximum movable distance of each mobile device can be determined based on factors such as whether the mobile devices are allowed to compress the target object, the maximum permissible degree of compression of the target object, and the angle of the mobile device's chassis towards the target position.

[0119] When it is determined that a narrow space cannot be traversed, the movement route can be replanned, or an alarm can be issued through the alarm device of the mobile device or through the user terminal.

[0120] If it is determined that the route cannot be replanned (i.e., the narrow space is the only way through), an alarm will be issued through the alarm device of the mobile device or through the user terminal to remind the user to take appropriate measures.

[0121] Furthermore, while the chassis of each mobile device moves toward the center of the target object, the robotic arms of each mobile device are controlled to move upward synchronously to compensate for the coordinate changes of the grippers in the horizontal direction, so that each gripping point moves the same distance only in the height direction, and the relative distance in the horizontal direction remains unchanged.

[0122] For example, as shown in Figure 8, for mobile devices B2 and B4 that are positioned opposite each other, the farthest distance between them is the distance between point B21 on mobile device B2 and point B41 on mobile device B4. Point B21 is the farthest point on mobile device B2 from mobile device B4, and point B41 is the farthest point on mobile device B4 from mobile device B2. If the farthest distance between them is greater than or equal to the width of the narrow space, and it is determined that all four mobile devices can pass through the narrow space after moving towards the center of the target object, then all four mobile devices are controlled to move towards the center of the target object (as indicated by the arrow), thereby reducing the overall width and allowing them to pass through the narrow space smoothly.

[0123] In this embodiment, multiple mobile devices model the target object. By equally dividing the circumcircle, circumcylinder, or circumsphere of the target object, gripping points evenly distributed on the target object are determined, as well as the directions of the chassis and robotic arms in horizontal projection. The robotic arms of the multiple mobile devices simultaneously grip the corresponding gripping points and lift them upwards at the same speed, realizing the gripping action of the target object. Afterwards, the robotic arms maintain their posture, and the chassis simultaneously moves towards the target position at the same speed, realizing the transport action.

[0124] Multiple mobile devices can communicate remotely in real time, exchanging chassis pose information and robotic arm pose information. Based on the robotic arm pose information, a negative feedback algorithm is used to ensure the consistency of the robotic arm's movements, ensuring that the resultant force direction at each gripping point is consistent and that the resultant torque is zero along the target direction, thus ensuring the smooth gripping and lifting of the target object. Based on the chassis pose information, a negative feedback algorithm is used to ensure the consistency of chassis movements, thus ensuring smooth transport.

[0125] During the gripping and handling of the target object, each mobile device can move closer to the center of the target object via its chassis, and the end of the robotic arm can be raised a certain distance to compensate for the changes in the horizontal relative distance between each gripping point.

[0126] When navigating narrow spaces, each mobile device can move its chassis closer to the center of the target object, reducing its overall width and thus navigating the confined environment.

[0127] As a specific implementation of the control method for the aforementioned mobile device, this disclosure provides a control device for a mobile device. As shown in FIG9, the control device 900 for the mobile device includes: a gripping point determination module 901 and a control module 902.

[0128] The gripping point determination module 901 is used to determine at least two gripping points on the target object to be transported.

[0129] Control module 902 is used for:

[0130] Synchronously control the robotic arm of each of at least two mobile devices to grasp a corresponding gripping point and lift the target object in the vertical direction;

[0131] After the target object is lifted into position, the chassis of each mobile device is simultaneously controlled to move toward the target location in order to transport the target object to the target location.

[0132] Furthermore, the control module 902 is also used to: control at least two mobile devices to acquire object feature information from different angles;

[0133] The device also includes an object recognition module, used to identify the target object to be transported based on object feature information from different angles.

[0134] Furthermore, the grip point determination module 901 is specifically used for:

[0135] Generate the external model of the target object based on its contour;

[0136] For the circumscribed model, divide it horizontally into N1 equal parts with the center of the circle as the center, resulting in N2 equal dividing lines, where N1 is an integer greater than or equal to 2 and N2 is an integer greater than or equal to 1.

[0137] For each bisector, determine the intersection point between the bisector and the contour of the target object, and determine the clamping point based on the intersection point.

[0138] Furthermore, the grip point determination module 901 is also used for:

[0139] After determining the gripping point, if the shape of the gripping point position on the target object and / or the gripper stroke of the robotic arm of the movable device do not meet the gripping conditions, a new gripping point is determined within the preset distance range of the gripping point, and the new gripping point is used to replace the gripping point. Alternatively, the external model is re-divided into M equal parts in the vertical direction, thereby re-determining M gripping points, where M is an integer greater than or equal to 2, and M is equal to or not equal to N1.

[0140] Furthermore, the control module 902 is specifically used for:

[0141] For each mobile device, determine the gripping point corresponding to the mobile device, and control the mobile device to move to a preset spatial position in front of its corresponding gripping point;

[0142] After reaching the preset spatial position, the robotic arm of the movable device is controlled to face the center of the target object and adjusted to the preset gripping angle to grasp the corresponding gripping point of the movable device.

[0143] Furthermore, the control module 902 is specifically used to: determine the gripping point corresponding to the mobile device with the goal of minimizing the travel distance or travel time to the preset spatial position of the gripping point.

[0144] Furthermore, the control module 902 is specifically used for:

[0145] After the robotic arm of the mobile device grasps the corresponding gripping point, the robotic arm of each mobile device is controlled to lift the target object in the vertical direction.

[0146] During the lifting process, the first speed at which each robotic arm moves upward is controlled to meet a first preset condition, so that the difference in the vertical movement distance of the grippers of each robotic arm is less than a first preset height. The first preset condition includes that the first speed at which each robotic arm moves upward is the same or the speed difference is less than the first preset speed.

[0147] Furthermore, controlling the first speed at which each robotic arm moves upward satisfies a first preset condition, including:

[0148] The robotic arm pose information of the mobile devices is transmitted through communication between the various mobile devices.

[0149] Based on the pose information of the robotic arm of each mobile device, the difference in the vertical movement distance of the gripper of each robotic arm is determined.

[0150] The speed of the robotic arms is controlled by negative feedback based on the difference in movement distance, so that the first speed of each robotic arm moving upward meets the first preset condition.

[0151] Furthermore, the control module 902 is also used for:

[0152] While the robotic arm lifts the target object vertically, the chassis of the mobile device is controlled to move towards the center of the target object to compensate for the coordinate changes of the robotic arm's gripper in the horizontal direction.

[0153] Furthermore, the control module 902 is specifically used to: determine that the target object has been lifted into place when at least two mobile device robotic arms smoothly lift the target object in the vertical direction, and each gripping point of the target object is lifted by the same preset distance.

[0154] Furthermore, the control module 902 is specifically used for:

[0155] After the target object is lifted into position, control the chassis of each mobile device to move to the transport posture;

[0156] Synchronously control each chassis to move towards the target position, and during the movement, control the second speed of each chassis to meet the second preset condition. The second preset condition includes that the second speeds of each chassis are the same or the speed difference is less than the second preset speed.

[0157] Furthermore, the method of controlling the second speed of movement of each chassis to meet the second preset condition includes:

[0158] The chassis pose information of the mobile devices is transmitted through communication between the various mobile devices.

[0159] During the movement, negative feedback control of the chassis speed is performed based on the chassis pose information of each mobile device, so that the second speed of each chassis movement meets the second preset condition.

[0160] Furthermore, the control module 902 is also used to: control the chassis of each mobile device to move toward the center of the target object if it is necessary to pass through a narrow space during the process of transporting the target object, and control the robotic arms of each mobile device to move upward synchronously to compensate for the coordinate changes of the gripper of the robotic arm in the horizontal direction.

[0161] The control device 900 for the mobile device in this embodiment can be the mobile device itself, or a component within the mobile device, such as an integrated circuit or a chip. The control device 900 for the mobile device provided in this embodiment can implement the various processes implemented in the control method embodiments of the mobile devices in Figures 2 and 7; to avoid repetition, these processes will not be described again here.

[0162] This disclosure also provides a mobile device, as shown in FIG10. The mobile device 1000 includes a processor 1001 and a memory 1002. The memory 1002 stores a program or instructions that can run on the processor 1001. When the program or instructions are executed by the processor 1001, they implement the various steps of the above-described mobile device control method embodiments and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0163] The memory 1002 can be used to store software programs and various data. The memory 1002 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback function, image playback function, etc.). Furthermore, the memory 1002 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1002 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0164] Processor 1001 may include one or at least two processing units; in one embodiment, processor 1001 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the modem processor may also not be integrated into processor 1001.

[0165] This disclosure also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described control method embodiments for the mobile device and achieve the same technical effects. To avoid repetition, these will not be described again here.

[0166] This disclosure also provides a computer program product including computer program instructions stored in a computer-readable storage medium and adapted to be called and executed by a processor to cause a computer device having the processor to perform the steps of the method described in any of the above embodiments.

[0167] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this disclosure is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0168] The embodiments of this disclosure have been described above with reference to the accompanying drawings. However, this disclosure is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this disclosure without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this disclosure.

[0169] All embodiments disclosed herein can be executed individually or in combination with other embodiments, and are all considered to be within the scope of protection claimed by this disclosure.

Claims

1. A control method for a mobile device, characterized in that, The method includes: Identify at least two gripping points on the target object to be transported; The robotic arm of each of at least two mobile devices is synchronously controlled to grasp one of the gripping points and lift the target object in the vertical direction. After the target object is lifted into position, the chassis of each mobile device is simultaneously controlled to move toward the target position in order to transport the target object to the target position.

2. The method according to claim 1, characterized in that, Before determining at least two gripping points on the target object to be transported, the method further includes: The system controls at least two mobile devices to acquire object feature information from different angles, and identifies the target object to be transported based on the object feature information from different angles.

3. The method according to claim 1 or 2, characterized in that, Determining at least two gripping points on the target object to be transported includes: Based on the outline of the target object, generate the outer model of the target object; The circumscribed model is divided into N1 equal parts horizontally with the center of the circle as the center, resulting in N2 equal dividing lines, where N1 is an integer greater than or equal to 2 and N2 is an integer greater than or equal to 1. For each of the bisectors, the intersection point of the bisector and the contour of the target object is determined, and the clamping point is determined based on the intersection point.

4. The method according to claim 3, characterized in that, The method further includes: After determining the gripping point, if the shape of the gripping point position on the target object and / or the gripper stroke of the robotic arm of the mobile device do not meet the gripping conditions, a new gripping point is determined within a preset distance range of the gripping point, and the new gripping point is used to replace the gripping point. Alternatively, the external model is re-divided into M equal parts in the vertical direction to redetermine M gripping points, where M is an integer greater than or equal to 2, and M is equal to or not equal to N1.

5. The method according to any one of claims 1 to 4, characterized in that, Synchronously controlling the robotic arm of each of at least two mobile devices to grasp one of the gripping points includes: For each mobile device, determine the gripping point corresponding to the mobile device, and control the mobile device to move to a preset spatial position in front of its corresponding gripping point; After reaching the preset spatial position, the robotic arm of the mobile device is controlled to face the center of the target object and adjusted to a preset gripping angle to grasp the gripping point corresponding to the mobile device.

6. The method according to claim 5, characterized in that, Determining the grip point corresponding to the mobile device includes: The gripping point corresponding to the mobile device is determined with the goal of minimizing the travel distance or travel time to the preset spatial position of the gripping point.

7. The method according to any one of claims 1-6, characterized in that, The step of lifting the target object vertically includes: After the robotic arm of the mobile device grasps the corresponding gripping point, the robotic arm of each mobile device is controlled to lift the target object in the vertical direction. During the lifting process, the first speed at which each of the robotic arms moves upward is controlled to meet a first preset condition, so that the difference in the vertical movement distance of the grippers of each robotic arm is less than a first preset height. The first preset condition includes that the first speed at which each of the robotic arms moves upward is the same or the speed difference is less than the first preset speed.

8. The method according to claim 7, characterized in that, The first speed at which each of the robotic arms moves upward satisfies a first preset condition, including: The robotic arm pose information of the mobile devices is transmitted through communication between the various mobile devices. Based on the pose information of the robotic arm of each of the mobile devices, the difference in the vertical movement distance of the gripper of each robotic arm is determined. The speed of the robotic arm is controlled by negative feedback based on the difference in the moving distance, so that the first speed at which each robotic arm moves upward meets the first preset condition.

9. The method according to claim 7 or 8, characterized in that, The method further includes: While the robotic arm lifts the target object vertically, the chassis of the mobile device is controlled to move toward the center of the target object to compensate for the coordinate changes of the gripper in the horizontal direction.

10. The method according to any one of claims 1 to 9, characterized in that, Determining the method by which the target object is lifted into position includes: When at least two robotic arms of the mobile device smoothly lift the target object in the vertical direction, and each gripping point of the target object is lifted by the same preset distance, the target object is determined to be lifted into position.

11. The method according to any one of claims 1 to 10, characterized in that, After the target object is lifted into position, the step of simultaneously controlling the chassis of each mobile device to move towards the target position to transport the target object to the target position includes: After the target object is lifted into position, the chassis of each mobile device is controlled to move to a transport posture. The system synchronously controls each chassis to move toward the target position, and during the movement, controls the second speed of each chassis to move to meet a second preset condition. The second preset condition includes that the second speeds of each chassis are the same or the speed difference is less than the second preset speed.

12. The method according to claim 11, characterized in that, The method of controlling the second speed of movement of each of the chassis to satisfy the second preset condition includes: The chassis pose information of the mobile devices is transmitted through communication between the various mobile devices. During the movement, negative feedback control of the chassis speed is performed based on the chassis pose information of each of the mobile devices, so that the second speed of each chassis movement meets the second preset condition.

13. The method according to any one of claims 1 to 12, characterized in that, The method further includes: During the transport of the target object, if it is necessary to pass through a narrow space, the chassis of each mobile device is controlled to move toward the center of the target object, and the robotic arms of each mobile device are controlled to move upward synchronously to compensate for the coordinate changes of the grippers of the robotic arms in the horizontal direction.

14. A control device for a mobile device, characterized in that, The device includes: The gripping point determination module is used to determine at least two gripping points on the target object to be transported. The control module is used for: The robotic arm of each of at least two mobile devices is synchronously controlled to grasp one of the gripping points and lift the target object in the vertical direction. After the target object is lifted into position, the chassis of each mobile device is simultaneously controlled to move toward the target position in order to transport the target object to the target position.

15. A mobile device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that run on the processor, the program or instructions being executed by the processor to implement the steps of the control method for the mobile device as described in any one of claims 1 to 13.

16. A readable storage medium having a program or instructions stored thereon, characterized in that, When the program or instructions are executed by the processor, they implement the steps of the control method for the mobile device as described in any one of claims 1 to 13.

17. A computer program product, characterized in that, The computer program product includes computer program instructions stored in a computer-readable storage medium and adapted to be invoked and executed by a processor to cause a computer device having the processor to perform the steps of the control method for a mobile device as claimed in any one of claims 1 to 13.

Images