Motion control method, device, robot and storage medium

Abstract

The present disclosure provides a motion control method, device, robot and storage medium. The method comprises: determining the target posture of the object based on the initial posture of the object and the posture change information collected by the remote control component; determining the target posture of the robot arm according to the target posture of the object and the relative posture between the robot arm and the object; and controlling the robot to move the robot arm to the target posture so that the object realizes the posture change represented by the posture change information.

Description

Technical Field

[0001] The present disclosure relates to the field of robot control technology, and in particular to a motion control method, device, robot and storage medium. Background Art

[0002] The anthropomorphic appearance of humanoid robots makes them very suitable for assisting people or replacing people in related complex and flexible tasks in manual operation environments. At present, the motion control of robots can be based on external data such as vision through traditional methods for path planning, or the robot can be dragged and calibrated by manual dragging, or the robot can be implemented by deploying end-to-end methods based on learning control methods. Among them, data collection based on manual dragging and calibration or end-to-end methods usually requires remote control to control the robot's motion, and the remote control process requires the design of corresponding mapping methods to achieve intuitive control of the operation object.

[0003] In the related art, multiple operating arms or motion capture sensors are often required to collect control data and map these control data to multiple robotic arms of the robot. However, this method is costly and can only realize direct mapping between the operating arms or motion capture sensors and the robotic arms installed on the robot. It is not intuitive for operators and the user experience is poor. Summary of the invention

[0004] To overcome the problems existing in the related art, the present disclosure provides a motion control method, device, robot and storage medium.

[0005] A first aspect of the present disclosure provides a motion control method, the method comprising:

[0006] Determining a target posture of the object based on an initial posture of the object and posture change information collected by the remote control component;

[0007] Determining a target posture of the robot arm according to the target posture of the object and the relative posture between the robot arm and the object;

[0008] The robot is controlled to move the mechanical arm to the target posture, so that the object realizes the posture change represented by the posture change information.

[0009] Optionally, the method further includes:

[0010] The relative position between the robotic arm and the object is determined according to the bias vector of the robotic arm and the expected posture of the robotic arm when operating the object, wherein the bias vector is used to characterize the position difference between the robotic arm and the object.

[0011] Optionally, the method further includes:

[0012] The target posture of the robotic arm is corrected according to the desired torque of the robotic arm and the current torque applied to the robotic arm.

[0013] Optionally, the correcting the target posture of the robotic arm according to the expected torque of the robotic arm and the torque currently applied to the robotic arm includes:

[0014] The position indicated by the target posture of the manipulator is corrected according to the desired torque of the manipulator, the current torque of the manipulator, and at least one preset admittance parameter, wherein the at least one admittance parameter includes at least one of the following:

[0015] Inertia coefficient, damping coefficient, stiffness coefficient.

[0016] Optionally, controlling the robot to move the mechanical arm to the target posture includes:

[0017] Maintaining the balance of the robot during movement is a high-priority control action, and controlling the robot arm to move to the target position is a low-priority control action. The torque of the joints of the robot is optimized, and the corresponding joints are controlled based on the torque optimization results of the joints to move the robot arm to the target position.

[0018] Optionally, the optimizing the torque of the joints of the robot includes:

[0019] According to each current control action, the priority of the control action and the Jacobian matrix corrected based on the priority task, a solution equation for the control vector is established, and the control vector is solved based on the solution method of the linear equation, wherein the control vector represents the torque of the joint of the robot.

[0020] Optionally, the remote control component includes an operating arm, and the relative position change of the operating arm is used to indicate the relative position change of the object.

[0021] Optionally, the method further includes:

[0022] Based on the current torque applied to the robotic arm, the environmental interaction force applied to the object is determined, and the environmental interaction force is fed back to the operating arm according to a preset force feedback ratio, wherein the environmental interaction force includes the resultant force of the external force applied to the object that does not originate from the robotic arm of the robot.

[0023] Optionally, the method further includes:

[0024] When the environmental interaction force of the object satisfies a preset event condition, a preset control logic corresponding to the event to which the event condition belongs is executed, wherein the event condition is used to determine whether the corresponding event occurs, and the event includes at least one of the following:

[0025] The object is placed, the object is blocked by an obstacle, or the object is in an abnormal state.

[0026] Optionally, the method further includes:

[0027] In the case where there is a deviation between the coordinate system of the remote control component and the coordinate system of the robot, the posture change information is corrected based on the rotation matrix corresponding to the deviation to obtain the posture change information corresponding to the coordinate system of the robot.

[0028] A second aspect of the present disclosure provides a motion control device, the device comprising:

[0029] An object posture determination module, used to determine a target posture of the object based on the initial posture of the object and the posture change information collected by the remote control component;

[0030] A robot arm posture determination module, used to determine the target posture of the robot arm according to the target posture of the object and the relative posture between the robot arm and the object;

[0031] A control module is used to control the robot to move the mechanical arm to the target posture, so that the object can achieve the posture change represented by the posture change information.

[0032] Optionally, the device further comprises:

[0033] A relative posture determination module is used to determine the relative posture between the robotic arm and the object based on the bias vector of the robotic arm and the expected posture of the robotic arm when operating the object, wherein the bias vector is used to characterize the position difference between the robotic arm and the object.

[0034] Optionally, the device further comprises:

[0035] The target posture correction module is used to correct the target posture of the robotic arm according to the expected torque of the robotic arm and the current torque applied to the robotic arm.

[0036] Optionally, the target posture correction module is used to correct the target posture of the robotic arm according to the expected torque of the robotic arm and the torque currently applied to the robotic arm, and is used to:

[0037] The position indicated by the target posture of the robotic arm is corrected according to the desired torque of the robotic arm, the current torque of the robotic arm and at least one preset admittance parameter, wherein the at least one admittance parameter includes at least one of the following: inertia coefficient, damping coefficient, stiffness coefficient.

[0038] Optionally, when the control module is used to control the robot to move the mechanical arm to the target posture, it is specifically used to:

[0039] Maintaining the balance of the robot during movement is a high-priority control action, and controlling the robot arm to move to the target position is a low-priority control action. The torque of the joints of the robot is optimized, and the corresponding joints are controlled based on the torque optimization results of the joints to move the robot arm to the target position.

[0040] Optionally, when the control module is used to optimize the torque of the joints of the robot, it is specifically used to:

[0041] According to each current control action, the priority of the control action and the Jacobian matrix corrected based on the priority task, a solution equation for the control vector is established, and the control vector is solved based on the solution method of the linear equation, wherein the control vector represents the torque of the joint of the robot.

[0042] Optionally, the remote control component includes an operating arm, and the relative position change of the operating arm is used to indicate the relative position change of the object.

[0043] Optionally, the device further comprises:

[0044] A force feedback module is used to determine the environmental interaction force on the object based on the current torque on the robotic arm, and to feed back the environmental interaction force to the operating arm according to a preset force feedback ratio, wherein the environmental interaction force includes the resultant force of the external force on the object that does not come from the robotic arm of the robot.

[0045] Optionally, the device further comprises:

[0046] An event determination module is used to execute a preset control logic corresponding to an event to which the event condition belongs when the environmental interaction force of the object satisfies a preset event condition, wherein the event condition is used to determine whether a corresponding event occurs, and the event includes at least one of the following: the object is placed, the object is blocked by an obstacle, and the object is in an abnormal state.

[0047] Optionally, the device further comprises:

[0048] The coordinate system correction module corrects the posture change information based on the rotation matrix corresponding to the deviation when there is a deviation between the coordinate system of the remote control component and the coordinate system of the robot, so as to obtain the posture change information corresponding to the coordinate system of the robot.

[0049] A third aspect of the present disclosure provides a computer program product, including a computer program / instruction, which implements the method described in the first aspect when executed by a processor.

[0050] A fourth aspect of the present disclosure provides a robot, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0051] A fifth aspect of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in the first aspect.

[0052] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0053] In the disclosed embodiment, firstly, the relative change posture of the remote control component within a preset time length is obtained, and this relative change posture can correspond to the relative posture of the object according to a preset mapping ratio; next, the target posture of the object can be determined according to the initial posture of the object and the above relative change posture, and finally the target posture of the mechanical arm is determined in combination with the relative posture between each mechanical arm of the robot used to pick up the object and the object, and finally the robot is controlled to move the mechanical arm so that the object realizes the posture change represented by the posture change information. Based on the above implementation, the posture mapping between the remote control component and the object can be realized, that is, the movement of the remote control component represents the movement of the object, and one or more mechanical arms of the robot can automatically adapt to the posture motion instructions issued by the operator based on a remote control component and directly for the object. Therefore, the above implementation provides a new control logic for robot teleoperation, which is not only more in line with human intuition, making the operator's operation process of the object more intuitive and simple, but also no longer requires the deployment of multiple remote control components or motion capture sensors to collect control data to complete the teleoperation process, while improving the user experience, and greatly reducing costs.

[0054] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0056] Figure 1 is a flow chart of a control method shown in some exemplary embodiments.

[0057] Figure 2 It is a scene schematic diagram of a control method shown by some exemplary embodiments.

[0058] Figure 3 It is a coordinate system diagram of a control method shown in some exemplary embodiments.

[0059] Figure 4 is a flow chart of another control method shown in some exemplary embodiments.

[0060] Figure 5 is a block diagram of a motion control device showing some exemplary embodiments.

[0061] Figure 6 is a structural diagram of a robot showing some exemplary embodiments. DETAILED DESCRIPTION

[0062] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0063] As described in the background technology, the related technology often requires multiple manipulators or motion capture sensors to collect control data and map these control data to multiple mechanical arms of the robot. However, this method is costly and can only achieve direct mapping between the manipulator or motion capture sensor and the mechanical arm installed on the robot, which is not intuitive for operators and has a poor user experience. In view of this, the present disclosure provides a motion control method, device, robot and storage medium.

[0064] Next, the embodiments of the present disclosure are described in detail.

[0065] The first aspect of the present disclosure provides a motion control method. Figure 1 , which may include steps S101 to S103.

[0066] Step S101, determining a target posture of the object based on the initial posture of the object and posture change information collected by the remote control component.

[0067] This step and its subsequent steps can be performed periodically at preset time intervals to complete continuous robot control operations (of course, it can also be performed only when the remote control component collects posture changes, and not performed if the remote control component does not collect posture change information). The remote control component is a component for receiving user instructions, which can be a control arm (also commonly referred to as a master hand or an operating arm), a computer (on which a software product for controlling a robot can be deployed and connected to input devices such as a mouse and keyboard), a remote control handle, etc.

[0068] See also Figure 2 , which shows an exemplary application scenario of the method provided by the present disclosure, wherein the remote control component is an operating arm, the base of the operating arm can generally be fixed, and the end of the operating arm is connected to the base of the operating arm through one or more joints; the user can indicate the movement of the object by moving the position of the end of the operating arm (that is, the remote control component includes the operating arm, and the relative posture change of the operating arm is used to indicate the relative posture change of the object; of course, this relative posture change can be proportional, for example, at a displacement ratio of 1:10, the user controls the end of the operating arm to move 1 cm upward, which can instruct the robot to move the object up 10 cm; and the change in posture can usually be mapped based on a ratio similar to 1:1).

[0069] The method provided in the present disclosure can be executed by the controller, processor and other components inside the main hand (after execution, the results are directly output to the robot to complete the robot motion control), or can be executed by the controller, processor and other components inside the robot (that is, the robot completes its own control based on the data collected by the main hand), or by a computer device deployed outside the robot and operating components. Their specific data flow process and instruction flow process can be set based on actual conditions, such as through preset interfaces and connecting cables (or wireless networks) to achieve connection between different devices, and the present disclosure does not limit this.

[0070] A robot may have one or more mechanical arms, such as Figure 2 In the present disclosure, the robot has two robotic arms (or two robotic arms for manipulating objects; in addition, the robot may also have robotic arms for performing other tasks, such as robotic arms that are only used to assist in overcoming obstacles, etc., but this is not mentioned in the present disclosure). The robot can generally manipulate objects by clamping, lifting, etc., moving them from one position to another, or picking up objects and performing trajectory motion, etc.; the specific method of manipulating objects and the specific posture corresponding to each robotic arm in this method (that is, the expected posture of the robotic arm when manipulating the object, for each robotic arm used to pick up objects, different picking methods may correspond to different expected postures) can be freely set, and the present disclosure does not limit this.

[0071] The posture change information collected by the remote control component can be composed of position change information and attitude change information. For example, the attitude change information of the remote control component can be obtained based on methods such as DH (Denavit-Hartenberg) or related interfaces provided by the remote control component, and combined with the position change information of the remote control component in this execution cycle of the method, the posture change information collected by the remote control component is determined. Exemplarily, the above steps can be implemented in the following manner.

[0072] First, obtain the relative position change of the operating component (the main hand is used as an example below) and map it to the relative position change of the object in proportion (the conversion step is optional and is also mentioned above). For example, , and They represent the relative position changes (i.e., relative motion) of the end of the main hand on the x-axis, y-axis, and z-axis, respectively. , and They represent the relative position changes of the object to be transported on the x-axis, y-axis, and z-axis respectively. Based on formula eq1, , and get , and :

[0073] eq1. ;

[0074] In the formula, in order to avoid the main hand moving range being too large and causing the system arms to exceed the motion workspace, the relative displacement of the main hand and the movement amount of the transported object are set to .

[0075] Next, you can get the initial posture of the main hand R 0 and ending gesture R 01 , and based on this, determine the posture changes of the master hand during the movement. It should be understood that in some cases, the coordinate system of the master hand may be different from the robot coordinate system. For example, some manufacturers may preset the coordinate system for the master hand during the production process. In addition, depending on the overall installation position and installation posture of the master hand, the coordinate system may also rotate during the deployment of the application. For example, see Figure 3 , which shows a relationship diagram between the master hand coordinate system and the coordinate systems of the robot and the object, where: x b , y b ,z b They represent the x-axis, y-axis, and z-axis of the object coordinate system respectively. x t , y t , z t They represent the x-axis, y-axis, and z-axis of the manipulator coordinate system, respectively. For intuitive reasons, the initial coordinate systems of the robot and the object can generally be made to conform to the world coordinate system. However, the coordinate system of the master hand may have deviations relative to the robot, the object, and the world coordinate system (at least one of them) (the coordinate system to be aligned and the specific deviation relationship can be determined as needed). For example, the coordinate system to be aligned can be rotated 90° around the z-axis and then 90° around the y-axis to obtain the current coordinate system of the master hand. Then "rotating 90° around the z-axis and 90° around the y-axis" can be regarded as this deviation.

[0076] In the case where there is a deviation between the coordinate system of the remote control component and the coordinate system of the robot, the posture change information can be corrected based on the rotation matrix corresponding to the deviation to obtain the posture change information corresponding to the coordinate system of the robot:

[0077] eq2. ;

[0078] eq3. ;

[0079] In the above two equations eq2 and eq3 To go around z The rotation matrix corresponding to the axis rotation of -90°, To go around y The rotation matrix corresponding to the -90° rotation of the axis; these two quantities are the rotation matrices corresponding to the deviations of the above-mentioned "90° rotation around the z-axis and 90° rotation around the y-axis". Based on this, we can complete R 0 and R 01 The correction is made to make the process of completing the operation of the object based on the main hand more in line with the operator's intuition, avoid the problem of confusion in the control direction caused by confusion in the coordinate system, and improve the user experience.

[0080] eq4. ;

[0081] eq5. ;

[0082] eq6. ;

[0083] Next, the posture change information indicated by the remote control component can be determined according to formulas eq4 to eq6 , the posture change information represents the relative posture change that the object needs to move based on the operation input by the user to the remote control component. In the above formula, R x is the relative posture change of the main hand, P x is a matrix that represents the relative position change of the master hand.

[0084] eq7. ;

[0085] Finally, the target pose of the object can be determined based on eq7 T objlast , T objini is the initial pose of the object.

[0086] Step S102, determining the target posture of the robot arm according to the target posture of the object and the relative posture between the robot arm and the object.

[0087] In other words, after determining the target posture of the object, it is also necessary to determine the target posture of the robot's mechanical arm, and then complete the operation process of the robot. As described above, the mechanical arm can have an expected posture when operating an object. For each mechanical arm used to pick up an object, different picking methods may correspond to different expected postures; for example, if the robot's mechanical arm is a relatively simple forklift-type fork arm mainly used to lift objects upward, then the expected posture can be to hold the mechanical arm horizontally and keep it still; and if the robot's mechanical arm is a slightly more complex gripper-type mechanical arm or a humanoid mechanical arm with more joints and fingers, then the expected posture can undergo a large number of feasible changes and can be freely adjusted according to actual conditions.

[0088] Next, optionally, the relative posture between the robotic arm and the object can be determined based on the bias vector of the robotic arm and the expected posture of the robotic arm when operating the object, wherein the bias vector is used to characterize the position difference between the robotic arm and the object.

[0089] by Figure 2 Taking the robot shown in as an example, the target position of its left hand , the target posture of the right hand is (The following text mainly describes the calculation process of the left hand, and the right hand is the same), among which, is the transformation relationship (i.e., relative posture) of the end of the robot's left hand relative to a reference point in the object coordinate system (the step of calculating the target posture of the object in the previous article can also be understood as the step of determining the target posture of this reference point), is the transformation relationship between the end of the robot's right hand and a reference point inside the object coordinate system; where, It can be expressed as:

[0090] eq8. ;

[0091] In the formula, is the desired posture of the left hand, which can be adjusted according to the actual situation so that the left hand can grasp the object. is the bias vector relative to the object (i.e. relative to the above reference point) obtained according to the actual grasping position of the left hand. Similarly, we can get , based on the postures of the left and right hands, the reference values ​​of each joint related to the left and right hands can be further solved.

[0092] It should be understood that in addition to the above-mentioned implementation methods that can more accurately determine the target posture of different robotic arms and more accurately realize robot control, the target posture of the robotic arm can also be determined based on the target posture of the object in other ways. For example, if the robotic arm of the robot is the fork arm mentioned above, the posture of the fork arm is kept unchanged and the position is set to a position at a preset height below the position indicated by the target posture of the object, so that the target posture of the fork arm can be obtained.

[0093] Step S103: Control the robot to move the mechanical arm to the target posture, so that the object achieves the posture change represented by the posture change information.

[0094] As mentioned above, the execution of the method provided by the present disclosure can be performed by a robot, a master hand, or an independent computer device (such as the cloud). When the method is executed by a robot, the master hand is responsible for collecting user instructions, and steps S101 to S103 can all be executed by the robot; when the method is executed by the master hand, the master hand can send the determined target posture to the robot after executing steps S101 to S102, so that the robot moves the robotic arm to the target posture; and when the method is executed by an independent computer device, the computer device can obtain posture change information from the master hand, and after calculating the target posture of the robotic arm, send the instruction carrying the target posture to the robot, so that the robot moves the robotic arm to the target posture. Of course, the above steps can also be disassembled and handed over to different execution entities for execution, but the overall process remains unchanged.

[0095] Next, the robot's arms can be controlled synchronously or asynchronously to move to the target position to complete the operation of the object.

[0096] In order to keep the robot stable during the movement control process of the robot, a priority-based motion control method can be used to control the coordination of the robot during the movement. For example, controlling the robot to move the mechanical arm to the target posture may include:

[0097] Maintaining the balance of the robot during movement is a high-priority control action, and controlling the robot arm to move to the target position is a low-priority control action. The torque of the joints of the robot is optimized, and the corresponding joints are controlled based on the torque optimization results of the joints to move the robot arm to the target position.

[0098] Exemplarily, multiple priorities can be pre-set, for example, joint limit (i.e., the range of motion of each joint of the robot is limited to prevent the joint from exceeding its physical limit, avoiding damage or loss of control), center of mass (center of gravity) position (e.g., the projection position of the robot's center of mass on the horizontal plane to ensure that the robot maintains balance during movement and prevents tipping), hand position (referring to the specific position of the robot's hand in space, used to accurately control the movement and operation of the hand), symmetrical posture (referring to the robot maintaining a symmetrical posture to maximize the range of motion of the joint and improve the flexibility and efficiency of movement) four priorities are set in order from high to low. In the subsequent process of controlling the movement of the robot, the completion of the task can be ensured in order from high to low priority to which the task belongs. For example, the priority of maintaining the stability of the robot's center of gravity so that the robot does not tip over is higher than the priority of accurately controlling the hand to reach a certain posture. Of course, the above four priorities are only exemplary, and they can be reduced or increased.

[0099] Taking the execution subject of steps S101 to S102 as the master hand as an example, the master hand can send the determined target position of each robotic arm to the robot. Next, the robot can use "control each robotic arm to move to the target position corresponding to the robotic arm" as a control action, while keeping "adjusting the center of gravity of the robot to achieve overall stability" as a higher priority control action and "controlling each robotic arm to move to the target position corresponding to the robotic arm" as a lower priority control action. In addition, other priority tasks can also be superimposed according to actual conditions, such as meeting certain interactive force requirements, moving the robot's leg components or tracks, etc.

[0100] Furthermore, the above process may include: establishing a solution equation for the control vector according to each current control action, the priority of the control action and the Jacobian matrix corrected based on the priority task, and solving the control vector based on a linear equation solution method, wherein the control vector represents the torque of the joint of the robot. For example, it can be implemented based on the following method (the robot adopts force control):

[0101]

[0102] Among them, τ represents a vector used to characterize the torque of one or more joints of the robot (of course, these joints are not necessarily all the joints of the robot), and the process of solving τ can be understood as the process of solving linear equations. express k control actions (e.g., force vectors corresponding to control actions), which can be expressed based on centroid force, Cartesian space force, joint torque, etc. For example, the process of controlling the center of gravity of a robot can also be understood as the process of controlling the torque of the center of gravity of the robot. The expression method can be exemplarily referred to the following formula (where x k That is, displacement, k pk , k dk Can be preset related parameters):

[0103]

[0104] and is the Jacobian matrix modified according to the priority task, which can be expressed as:

[0105]

[0106] In the above formula, is the null space of the control action (task) with higher priority. The above tasks can be simplified to solve linear equations or linear inequalities, as follows:

[0107]

[0108] In the above formula, , , , Representing tasks and constraints, we can further deduce:

[0109]

[0110] In the above formula, To solve The slack variables.

[0111] Based on this, the actual control environment of the robot can be better adapted, and the robot's body stability control can be better achieved. It should be understood that the algorithm used to perform the above-mentioned solution and correction process can be carried by the robot; in addition, the algorithm used to perform the above-mentioned solution and correction process can also be deployed in the main hand or in the computer device (in this case, the robot may need to send parameters back to the main hand and the computer device), and the present disclosure does not limit this.

[0112] Based on the above implementation, the posture mapping between the remote control component and the object can be realized, that is, the movement of the remote control component represents the movement of the object, and one or more mechanical arms of the robot can automatically adapt to the posture motion instructions issued by the operator based on a remote control component and directly aimed at the object. Therefore, the above implementation provides a new control logic for robot teleoperation, which is not only more in line with human intuition, making the operator's operation process of the object more intuitive and simple, but also no longer requires the deployment of multiple remote control components or motion capture sensors to collect control data to complete the teleoperation process, while greatly reducing costs while improving user experience.

[0113] In some embodiments, the end of the robotic arm may be equipped with a force sensor (e.g., a six-dimensional force sensor, the so-called end position may be a position for interacting with an object, therefore, the target position of the robotic arm, the desired posture of the robotic arm, and other physical quantities referred to in this disclosure may also be understood as physical quantities for the end of the robotic arm). After obtaining the desired position of any robotic arm, the method may further include:

[0114] The target posture of the robotic arm is corrected according to the desired torque of the robotic arm and the current torque applied to the robotic arm.

[0115] Exemplarily, the correcting the target posture of the robotic arm according to the desired torque of the robotic arm and the current torque applied to the robotic arm may include:

[0116] The position indicated by the target posture of the robotic arm is corrected according to the desired torque of the robotic arm, the current torque of the robotic arm and at least one preset admittance parameter, wherein the at least one admittance parameter includes at least one of the following: inertia coefficient, damping coefficient, stiffness coefficient.

[0117] In the actual motion process, the end of the robot's arms may have certain internal forces (such as objects being squeezed or the end of the robot arm being torn) due to control errors, mechanical vibrations and other factors. In order to prevent the structural damage of the robot arm or the object being transported caused by excessive internal forces, an admittance control method based on force sensor data is proposed. The admittance control method can be implemented based on the following admittance control formula:

[0118]

[0119] in, They are the inertia coefficient, damping coefficient and stiffness coefficient respectively. is the adjusted expected position (i.e. the corrected position information), is the theoretical position (i.e., the position before correction, which can be the position information indicated by the target posture after the target posture of the robot arm is obtained), is the expected force (i.e., the expected force on the robotic arm, which can also be understood as the expected reading of the force sensor. This value can be comprehensively determined based on the properties of the object and the properties of the robotic arm, and this disclosure does not limit this), is the actual force measured by the force sensor. In the above formula, y The concept of can be a coordinate in one direction (such as the direction of a coordinate axis) or a coordinate in multiple dimensions (such as a three-dimensional coordinate, that is, the complete position information indicated by the target posture); and the dimension of F can be the same as y Keep it consistent, e.g. y is a matrix used to represent three-dimensional positions, so F can also be used to represent three-dimensional forces, that is, forces in the directions of each coordinate axis.

[0120] Based on the above process, it can be further ensured that the object and the robotic arm will not be damaged during the operation of the object, making the operation process more rational and intelligent, and ensuring the safety of the operation process.

[0121] In some embodiments, in order to further improve the operator's operating experience and the amount of information obtained, the method may further include:

[0122] Based on the current torque applied to the robotic arm, the environmental interaction force applied to the object is determined, and the environmental interaction force is fed back to the operating arm according to a preset force feedback ratio, wherein the environmental interaction force includes the resultant force of the external force applied to the object that does not originate from the robotic arm of the robot.

[0123] In other words, when operating an object based on teleoperation, the operator may not be able to get a specific sense of the operation process, which may lead to poor operating experience or even excessive operation. Then, a master hand with force feedback function can be used, and the force sensed by the force sensor can be converted and fed back to the master hand based on the force feedback mechanism, so that the user can feel the force feedback effect. Exemplarily, the force sensor data can be mapped to the force feedback data fed back by the master hand based on the following formula:

[0124] eq9. ;

[0125] in, is the mapping scale factor, is the feedback force of the master hand (i.e., the force used to feed back to the master hand for the user to perceive). Assume that the interaction force between the object and the external environment is , according to the force balance equation:

[0126] eq10. ;

[0127] The interaction force between the object and the environment can be calculated (This is because when an object remains stationary or moves at a constant speed in a straight line, the resultant force of the external forces acting on it is 0; if the object is accelerating or decelerating, the resultant force of the external forces acting on the object can be obtained according to other physical equations, and the feedback force of the master hand end can be determined based on the resultant force of the external forces. The principle is the same as eq10). as well as The interaction forces measured by the force sensors (e.g., six-dimensional force sensors) of the left and right hands respectively, as well as is the force arm of the contact point (i.e., the contact point between the robot arm and the object) corresponding to the distance between the force sensors of the left and right hands.

[0128] In a popular way of understanding, the process of picking up an object is mainly the process of the combined force given to the object by the robot arm and the gravity (and other external forces) exerted on the object itself, and the movement of the end of the main hand corresponds to the movement of the object. That is, for the user, the end of the main hand is actually equivalent to the object, and the force exerted by the user's arm on the end of the main hand can be understood as the force exerted by the robot's robot arm on the object, and the force exerted by the object on the robot arm is the environmental interaction force of the object. Then the force feedback given to the user by the end of the main hand should also be the environmental interaction force exerted on the object (which can be scaled according to the force feedback ratio). Based on this, the force feedback mechanism can give users a force feedback effect similar to that felt when actually moving an object, which not only improves the user experience, but also allows users to promptly perceive the occurrence of events such as the completion of object placement, the obstruction of the object by obstacles, and the abnormal state of the object (for example, the object suddenly breaks), making the control process more accurate.

[0129] The above-mentioned events such as object placement completion, object being blocked by an obstacle, object abnormality, etc. can also be automatically determined based on the environmental interaction force exerted on the object. For example, the method can also include:

[0130] When the environmental interaction force of the object meets the preset event condition, the preset control logic corresponding to the event to which the event condition belongs is executed, wherein the event condition is used to determine whether the corresponding event occurs, and the event includes at least one of the following: the object is placed, the object is blocked by an obstacle, and the object is in an abnormal state.

[0131] Among them, the completion of object placement can refer to the completion of a staged task in the process of the robot operating the object. For example, when the robot is currently performing a carrying task, when the environmental interaction force of the object detected by the robot decreases or even approaches 0 (that is, the resultant force of the force on the object itself except the external force from the robot arm approaches 0), it can be considered that the object has been placed on a table, and then it can be considered that the object has been placed and the release action can be performed. In addition to judging based on the magnitude of the force, the judgment process can also be judged based on the direction of the force. For example, although the environmental interaction force of the object is reduced, it has an obvious horizontal offset, then it can be considered that the object is not actually placed well, but only less than half of the bottom surface touches the table; the above judgment logic (that is, the event conditions corresponding to the event) can be comprehensively determined by combining the specific tasks performed by the robot, the specific type of the object, the specific needs of the scene and other factors. The event of an object being blocked by an obstacle is similar to the above-mentioned event of the completion of object placement. For example, when the environmental interaction force of an object suddenly increases in a certain direction, it can be considered that the object encounters an obstacle in the opposite direction of that direction. Events of abnormal object status may include objects slipping from the robot's operating arm, objects breaking, objects disintegrating (for example, stacked goods scattering or a significant change in their stacking shape), etc. These events can usually be determined based on an abnormally rapid decrease in the environmental interaction force acting on the object.

[0132] The above events are only examples. The environmental interaction force of objects can also be used to determine the occurrence of more events, which will not be repeated here. Each event can correspond to a preset control logic, such as pausing the current task, reporting the current event to the user, executing a preset avoidance action (such as operating the object away from obstacles), etc. The above logic further improves the intelligence of the robot, so that when the operator does not realize the occurrence of the event in time, or misjudges the severity of the event, the robot can still automatically execute the preset control logic according to the current event, and report the event to the user, thereby further improving the user experience and the safety of the robot operation process.

[0133] The following combination Figure 4 The flowchart shown, taking the execution subject of the method as a main hand as an example, comprehensively describes at least one of the above embodiments.

[0134] First, the control chip inside the master hand can periodically obtain the posture information and position information of the end of the master hand, and when the posture of the master hand changes (that is, it is considered to have received the user's operation), the posture change of the master hand is determined according to the input signal of the master hand. Then this posture change is mapped to the posture change of the object (for example, mapped according to a certain ratio, the position and posture can have separate mapping ratios) to obtain the movement trajectory of the object (that is, to obtain the target posture of the object, and the specific movement trajectory can be achieved by other algorithms or other control logic built into the robot; of course, the execution cycle of the method can also be minimized to minimize the concept of the movement trajectory of the object, and replaced by several small target posture changes).

[0135] Next, the specific target posture of each manipulator (i.e., the end trajectory of each manipulator, as in the previous paragraph) and the target data of the joints of these manipulators can be determined according to the target posture of the object and the operating state of each manipulator on the object (such as its offset vector and expected posture). After the data is determined, the target posture of each manipulator can be corrected based on the admittance control method according to the actual data and expected data of the force sensor. At this point, the master hand has obtained the final target posture of each manipulator. Finally, the instructions carrying these final target postures can be sent to the robot so that the robot can comprehensively control one or more manipulators to move to the state indicated by the target posture while ensuring the balance of the fuselage, so as to complete the robot motion control within a cycle. When a stage is completed, the sensing force of the force sensor can be used to feedback to the operator (through the master hand), or it can be used for the next stage of admittance control, so as to solve the problem of abnormal force on the robot or object in a timely manner.

[0136] Corresponding to the embodiments of the aforementioned method, the present disclosure also provides embodiments of a device and a terminal to which the device is applied.

[0137] The second aspect of the present disclosure provides a motion control device, see Figure 5 , the device comprises:

[0138] The object posture determination module 501 is used to determine the target posture of the object based on the initial posture of the object and the posture change information collected by the remote control component;

[0139] A robot arm posture determination module 502 is used to determine the target posture of the robot arm according to the target posture of the object and the relative posture between the robot arm and the object;

[0140] The control module 503 is used to control the robot to move the mechanical arm to the target posture, so that the object can achieve the posture change represented by the posture change information.

[0141] Optionally, the device further comprises:

[0142] A relative posture determination module is used to determine the relative posture between the robotic arm and the object based on the bias vector of the robotic arm and the expected posture of the robotic arm when operating the object, wherein the bias vector is used to characterize the position difference between the robotic arm and the object.

[0143] Optionally, the device further comprises:

[0144] The target posture correction module is used to correct the target posture of the robotic arm according to the expected torque of the robotic arm and the current torque applied to the robotic arm.

[0145] Optionally, the target posture correction module is used to correct the target posture of the robotic arm according to the expected torque of the robotic arm and the torque currently applied to the robotic arm, and is used to:

[0146] The position indicated by the target posture of the robotic arm is corrected according to the desired torque of the robotic arm, the current torque of the robotic arm and at least one preset admittance parameter, wherein the at least one admittance parameter includes at least one of the following: inertia coefficient, damping coefficient, stiffness coefficient.

[0147] Optionally, when the control module is used to control the robot to move the mechanical arm to the target posture, it is specifically used to:

[0148] Maintaining the balance of the robot during movement is a high-priority control action, and controlling the robot arm to move to the target position is a low-priority control action. The torque of the joints of the robot is optimized, and the corresponding joints are controlled based on the torque optimization results of the joints to move the robot arm to the target position.

[0149] Optionally, when the control module is used to optimize the torque of the joints of the robot, it is specifically used to:

[0150] According to each current control action, the priority of the control action and the Jacobian matrix corrected based on the priority task, a solution equation for the control vector is established, and the control vector is solved based on the solution method of the linear equation, wherein the control vector represents the torque of the joint of the robot.

[0151] Optionally, the remote control component includes an operating arm, and the relative position change of the operating arm is used to indicate the relative position change of the object.

[0152] Optionally, the device further comprises:

[0153] A force feedback module is used to determine the environmental interaction force on the object based on the current torque on the robotic arm, and to feed back the environmental interaction force to the operating arm according to a preset force feedback ratio, wherein the environmental interaction force includes the resultant force of the external force on the object that does not come from the robotic arm of the robot.

[0154] Optionally, the device further comprises:

[0155] An event determination module is used to execute a preset control logic corresponding to an event to which the event condition belongs when the environmental interaction force of the object satisfies a preset event condition, wherein the event condition is used to determine whether a corresponding event occurs, and the event includes at least one of the following: the object is placed, the object is blocked by an obstacle, and the object is in an abnormal state.

[0156] Optionally, the device further comprises:

[0157] The coordinate system correction module corrects the posture change information based on the rotation matrix corresponding to the deviation when there is a deviation between the coordinate system of the remote control component and the coordinate system of the robot, so as to obtain the posture change information corresponding to the coordinate system of the robot.

[0158] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0159] A third aspect of the present disclosure provides a computer program product, including a computer program / instruction, which implements the method described in the first aspect when executed by a processor.

[0160] For the device embodiments and computer program product embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. In addition, the device embodiments described above are merely schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. Those of ordinary skill in the art can understand and implement it without paying creative labor.

[0161] In a fourth aspect, some embodiments of the present disclosure provide a robot, see Figure 6, which shows the structure of the robot, the robot includes a memory and a processor, the memory is used to store computer instructions that can be run on the processor, and the processor is used to control based on the method described in the first aspect when executing the computer instructions. Of course, the above robot can also be transformed into other electronic devices such as operating arms, computer devices, etc. according to different execution subjects of the method.

[0162] In a fifth aspect, the present disclosure, in an exemplary embodiment, further provides a non-transitory computer-readable storage medium including instructions, such as a memory including instructions, and the instructions can be executed by a processor of a robot or a computer device to complete the control method of the robot. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0163] The above describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0164] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the inventions claimed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not claimed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0165] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0166] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A motion control method, characterized in that: The method comprises: Determining a target posture of the object based on an initial posture of the object and posture change information collected by the remote control component; Determining a target posture of the robot arm according to the target posture of the object and the relative posture between the robot arm and the object; Controlling the robot to move the mechanical arm to the target posture, so that the object realizes the posture change represented by the posture change information; Wherein, the remote control component includes an operating arm, and the relative posture change of the operating arm is used to indicate the relative posture change of the object; The method further comprises: Based on the current torque applied to the robotic arm, the environmental interaction force applied to the object is determined, and the environmental interaction force is fed back to the operating arm according to a preset force feedback ratio, wherein the environmental interaction force includes the resultant force of the external force applied to the object that does not originate from the robotic arm of the robot.

2. The motion control method according to claim 1, characterized in that: The method further comprises: The relative position between the robotic arm and the object is determined according to the bias vector of the robotic arm and the expected posture of the robotic arm when operating the object, wherein the bias vector is used to characterize the position difference between the robotic arm and the object.

3. The motion control method according to claim 1, characterized in that: The method further comprises: The target posture of the robotic arm is corrected according to the desired torque of the robotic arm and the current torque applied to the robotic arm.

4. The motion control method according to claim 3, characterized in that: The step of correcting the target posture of the robotic arm according to the desired torque of the robotic arm and the torque currently applied to the robotic arm comprises: The position indicated by the target posture of the robotic arm is corrected according to the desired torque of the robotic arm, the current torque of the robotic arm and at least one preset admittance parameter, wherein the at least one admittance parameter includes at least one of the following: inertia coefficient, damping coefficient, stiffness coefficient.

5. The motion control method according to claim 1, characterized in that: The controlling the robot to move the mechanical arm to the target posture comprises: Maintaining the balance of the robot during movement is a high-priority control action, and controlling the robot arm to move to the target position is a low-priority control action. The torque of the joints of the robot is optimized, and the corresponding joints are controlled based on the torque optimization results of the joints to move the robot arm to the target position.

6. The motion control method according to claim 5, characterized in that: The step of optimizing the torque of the joints of the robot comprises: According to each current control action, the priority of the control action and the Jacobian matrix corrected based on the priority task, a solution equation for the control vector is established, and the control vector is solved based on the solution method of the linear equation, wherein the control vector represents the torque of the joint of the robot.

7. The motion control method according to claim 1, characterized in that: The method further comprises: When the environmental interaction force of the object meets the preset event condition, the preset control logic corresponding to the event to which the event condition belongs is executed, wherein the event condition is used to determine whether the corresponding event occurs, and the event includes at least one of the following: the object is placed, the object is blocked by an obstacle, and the object is in an abnormal state.

8. The motion control method according to claim 1, characterized in that: The method further comprises: In the case where there is a deviation between the coordinate system of the remote control component and the coordinate system of the robot, the posture change information is corrected based on the rotation matrix corresponding to the deviation to obtain the posture change information corresponding to the coordinate system of the robot.

9. A motion control device, characterized in that: The device comprises: An object posture determination module, used to determine a target posture of the object based on the initial posture of the object and the posture change information collected by the remote control component; A robot arm posture determination module, used to determine the target posture of the robot arm according to the target posture of the object and the relative posture between the robot arm and the object; A control module, used for controlling the robot to move the mechanical arm to the target posture, so that the object realizes the posture change represented by the posture change information; Wherein, the remote control component includes an operating arm, and the relative posture change of the operating arm is used to indicate the relative posture change of the object; The device also includes: A force feedback module is used to determine the environmental interaction force on the object based on the current torque on the robotic arm, and to feed back the environmental interaction force to the operating arm according to a preset force feedback ratio, wherein the environmental interaction force includes the resultant force of the external force on the object that does not come from the robotic arm of the robot.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

11. A robot, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images