Robot pointing motion control method, device, electronic device, and storage medium

Abstract

The present invention provides a method, an apparatus, an electronic device, and a storage medium for controlling a pointing motion of a robot. The method includes the steps of: when a target object is beyond a reachable range of a tip of a robot arm, selecting one of two arms of the robot closer to the target object based on target coordinates of the target object in a robot coordinate system, acquiring a position and posture in which the tip of the selected arm points to the target object within a reachable range based on mapping of the target coordinates, the coordinates of a shoulder joint of the selected arm in the robot coordinate system, and a predetermined pointing motion rule, determining a trajectory of the selected arm from a current position and posture of each joint of the selected arm to a target position and posture of each joint based on the acquired position and posture of the arm tip, and controlling the selected arm to point to the target object based on the trajectory.

Description

[Technical field]

[0001] The present invention relates to the technical field of human-like movement of a robot arm, and in particular to a method, apparatus, electronic device and storage medium for controlling the pointing movement of a robot. [Background technology]

[0002] With the development of science and technology, the application of robots has penetrated into all aspects of people's lives. For example, many shopping malls, restaurants, exhibition halls, etc. have introduced service robots. However, in the current robot-human interaction technology, the voice and vision technology is relatively mature, and the control of body movements is relatively lacking, so the interaction between robots and humans is relatively awkward. If body language is added to this, the robot will be more human-like when interacting with people.

[0003] In the related art, when a robot arm performs an operation, if a target object is outside the reachable range of the arm, the robot does nothing with respect to the target object. In this way, the application scenarios of the robot arm are limited. Summary of the Invention

[0004] The present invention provides a method, apparatus, electronic device and storage medium for controlling a robot's pointing action to realize application to scenes where the robot points to a target object and improve the versatility of the robot.

[0005] The present invention provides a method for controlling a pointing motion of a robot, the method comprising: When the target object is beyond a reachable range of the arm tip of the robot, selecting one of the two arms of the robot that is closer to the target object based on target coordinates of the target object in a robot coordinate system; acquiring a position and posture in which the tip of the selected arm points to the target object within a reachable range based on mapping of the target coordinates, coordinates of a shoulder joint of the selected arm in the robot coordinate system, and a predetermined pointing motion rule; determining a trajectory of the selected arm from a current position and posture of each joint of the selected arm to a target position and posture of each joint of the selected arm based on the acquired position and posture of the tip of the arm; and controlling the selected arm to point to a target object based on the trajectory.

[0006] Further, the step of acquiring a position and posture in which the tip of the selected arm points to the target object within a reachable range based on the mapping of the target coordinates, the coordinates of the shoulder joint of the selected arm in the robot coordinate system, and a predetermined pointing motion rule, includes: mapping the target coordinates to coordinates within a reachable range of the tip of the selected arm; determining a pointing direction of the tip of the selected arm based on the predetermined pointing motion rule; generating a pose of the tip of the selected arm based on the mapped coordinates, the coordinates of a shoulder joint of the selected arm in the robot coordinate system, and the pointing direction.

[0007] Furthermore, the step of mapping the target coordinates to coordinates within a reachable range of the tip of the selected arm includes: acquiring an X-coordinate and a Y-coordinate of a tip of the selected arm based on an angle between a vector from a shoulder joint of the selected arm toward the target object in the robot coordinate system and an X-axis of the robot coordinate system, and a length of the selected arm; obtaining a Z coordinate of a tip of the selected arm based on a relationship between a Z direction coordinate of the target object in the robot coordinate system, a coordinate of a shoulder joint of the selected arm, and a length of the arm; and / or generating a pose of the tip of the selected arm based on the mapped coordinates, a coordinate of a shoulder joint of the selected arm in a robot coordinate system, and the pointing direction, determining unit vectors of a vector from the target object to an arm tip as three Z direction components for constituting a Z direction vector of an attitude matrix of the selected arm tip, the attitude matrix of the selected arm tip being a matrix of 3 rows and 3 columns; The tip of the selected arm is pointed to a target object, and the posture matrix Y component of the Y direction vector, oz is positive and the selected arm is the right arm, the Y-direction vector of the posture matrix of Y-direction component oy is determined to be positive Then, three Y-direction components for constituting the Y-direction vector of the posture matrix are determined. Steps and The tip of the selected arm is pointed to a target object, and the Y component of the Y direction vector, oz is positive and the selected arm is the left arm, the Y-direction vector of the orientation matrix of Y-direction component oy Determined to be negative Then, three Y-direction components for constituting the Y-direction vector of the posture matrix are determined. Steps and and determining three X-direction components for constituting an X-direction vector of a posture matrix based on the three Z-direction components and three Y-direction components of the posture matrix of the tip of the selected arm in accordance with the right-hand rule.

[0008] Further, after controlling the selected arm to point to a target object based on the trajectory, the method includes: The method further includes measuring the acquired arm tip position and orientation using a measurement device and determining an error in the operation of pointing to a target object at the tip of the selected arm.

[0009] Further, the step of measuring the acquired position and orientation of the arm tip using a measurement device and determining an error in the operation of pointing to the target object of the tip of the selected arm includes: determining a vector from the target object to the tip of the selected arm; and determining, as the error, an angle between the vector and a Z-direction vector of a posture matrix of the tip of the selected arm.

[0010] Furthermore, the step of determining a trajectory of the selected arm from a current position and posture of each joint of the selected arm to a target position and posture of each joint of the selected arm based on the acquired position and posture of the arm tip includes: acquiring a target posture of each joint of the selected arm by solving inverse kinematics based on the acquired position and posture of the arm tip; and obtaining a trajectory of the selected arm from a current position and posture to a target position and posture of each of the joints by trajectory interpolation.

[0011] Furthermore, the step of acquiring a target posture of each joint of the selected arm by solving inverse kinematics based on the acquired position and posture of the arm tip includes: A step of solving inverse kinematics based on the acquired position and orientation of the arm tip to acquire a target configuration of the selected arm, the target configuration including a target angle of each of the joints; and performing trajectory interpolation based on the current configuration of the robot and the goal configuration, and controlling the selected arm to move from the current configuration of the robot to the goal configuration of the robot pointing to the target object, wherein the current configuration of the robot includes the current angles of each of the joints.

[0012] The present invention provides a robot pointing motion control device, the device comprising: an arm selection module for selecting one of the two arms of the robot that is closer to the target object based on a target coordinate of the target object in a robot coordinate system when the target object is beyond a reachable range of an arm tip of the robot; an arm-tip position and orientation determination module for acquiring a position and orientation in which the tip of the selected arm points to the target object within a reachable range based on mapping of the target coordinates, a coordinate of a shoulder joint of the selected arm in the robot coordinate system, and a predetermined pointing motion rule; a trajectory determination module for determining a trajectory of the selected arm from a current position and posture of each joint of the selected arm to a target position and posture of each joint, based on the acquired position and posture of the tip of the arm; and a motion control module for controlling the selected arm to point to a target object based on the trajectory.

[0013] The present invention provides an electronic device, the electronic device including one or more processors and a memory for storing one or more programs, the one or more programs being executed by the one or more processors to cause the one or more processors to implement a pointing motion control method for the robot.

[0014] The present invention provides a computer-readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the robot pointing motion control method.

[0015] In some embodiments, the method for controlling a robot pointing motion of the present invention completes the motion of the selected arm pointing to the target object by acquiring a position and posture in which the tip of the selected arm points to the target object within the reachable range based on mapping of the target coordinates, the coordinates of the shoulder joint of the selected arm in the robot coordinate system, and a predetermined pointing motion rule, when the target object is beyond the reachable range of the tip of the robot arm. In this way, the method can be applied to scenes in which the target object is pointed to, and the versatility of the robot can be improved. [Brief description of the drawings]

[0016] [Figure 1]FIG. 2 is a schematic diagram showing a flow of a robot pointing motion control method provided in an embodiment of the present invention. [Diagram 2] FIG. 2 is a schematic diagram showing a specific flow of a robot pointing motion control method provided in an embodiment of the present invention. [Diagram 3] FIG. 2 is a schematic diagram showing a simplified configuration of a robot coordinate system and a target object in the robot pointing motion control method provided in an embodiment of the present invention. [Figure 4a] FIG. 2 is a schematic diagram showing the current configuration of a robot in the robot pointing motion control method provided in an embodiment of the present invention. [Figure 4b] FIG. 2 is a schematic diagram showing a robot goal configuration of the robot pointing motion control method provided in an embodiment of the present invention. [Diagram 5] FIG. 1 is a schematic diagram showing a flow of measuring the movement accuracy of a robot pointing movement control method provided in an embodiment of the present invention. [Figure 6] FIG. 2 is a schematic diagram showing modules of a robot pointing motion control device provided in an embodiment of the present invention. [Figure 7] FIG. 1 is a block diagram showing an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Exemplary embodiments are now described in detail, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings refer to the same or similar elements unless otherwise specified. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of the present invention. Instead, they are merely examples of apparatus and methods consistent with some aspects of one or more embodiments of the present invention, as detailed in the appended claims.

[0018] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in the present invention. In some other embodiments, the method may include more or fewer steps than those described in the present invention. Also, a single step described in the present invention may be decomposed into multiple steps in other embodiments, and multiple steps described in the present invention may be combined into a single step in other embodiments.

[0019] In order to solve the technical problem that the application scenarios of robot arms are relatively limited, an embodiment of the present invention provides a method for controlling the pointing motion of a robot, which, when a target object is beyond the reachable range of the tip of the robot's arm, selects one of the two arms of the robot that is closer to the target object based on target coordinates of the target object in the robot coordinate system, obtains a position and posture for the tip of the selected arm to point to the target object within the reachable range based on mapping of the target coordinates, the coordinates of the shoulder joint of the selected arm in the robot coordinate system, and a predetermined pointing motion rule, determines a trajectory of the selected arm from the current position and posture of each joint of the selected arm to the target position and posture of each joint based on the obtained position and posture of the arm tip, and controls the selected arm to point to the target object based on the trajectory.

[0020] In an embodiment of the present invention, when the target object is beyond the reachable range of the tip of the robot's arm, the position and posture of the tip of the selected arm pointing to the target object within the reachable range is acquired based on the mapping of the target coordinates, the coordinates of the shoulder joint of the selected arm in the robot coordinate system, and a predetermined pointing motion rule, thereby completing the motion of pointing to the target object of the selected arm. In this way, application to scenes in which the target object is pointed to can be realized, and the versatility of the robot can be improved.

[0021] The method for controlling a robot's pointing action according to an embodiment of the present invention is applied to a scene in which the robot points to a target object. The application scene in which the robot points to a target object may include, but is not limited to, a scene in which the robot gives an explanation at an exhibition hall or a scene in which the robot explains the contents of a presentation. The contents of the presentation may include, but are not limited to, a video, a lecture manuscript, and the contents of a lesson. The target may include, but is not limited to, the specific contents of the presentation contents.

[0022] A specific procedure for implementing the method for controlling the pointing motion of a robot will be described in detail below.

[0023] FIG. 1 is a schematic diagram showing a flow of a method for controlling a pointing motion of a robot provided in an embodiment of the present invention.

[0024] As shown in FIG. 1, the method for controlling a pointing movement of a robot includes the following steps 110 to 140.

[0025] In step 110, if the target object is beyond the reachable range of the robot's arm tip, the arm closer to the target object is selected from the two arms of the robot based on the target coordinates of the target object in the robot coordinate system.

[0026] In this way, the fact that the target object is beyond the reachable range of the end of the robot's arm means that the target object is outside the reachable range of the end of the robot's arm. Also, by determining the target coordinates of the target object in the robot coordinate system, the target object and the two arms of the robot are in the same robot coordinate system, so that the selected arm that is closer to the target object can be easily and quickly obtained.

[0027] The above step 110 completes the pointing action using the arm closest to the target object, which can improve the convenience of pointing, meet the needs of users, and is favorable for implementing the operation.

[0028] There are various embodiments of step 110. In one embodiment, step 110 includes a step of determining, based on target coordinates of the target object in the robot coordinate system, a region in which the target object is located from two pre-divided regions corresponding to the two arms of the robot, and determining the arm corresponding to the region in which the target object is located as the arm closer to the target object out of the two arms of the robot.

[0029] In another embodiment, the above step 110 includes determining a mapping distance between vectors between the target object and each of the two shoulder joints of the arm on the horizontal coordinate plane of the robot coordinate system, and determining the arm corresponding to the shorter mapping distance as the arm of the robot's two arms that is closer to the target object.

[0030] In this embodiment, the coordinates of the left shoulder joint in the robot coordinate system are calculated based on the dimensions of each shoulder joint of the robot, for example, the height h and width w of the shoulder joint.

number

number

[0031] In step 120, a position and orientation is obtained in which the tip of the selected arm points to the target object within the reachable range based on the mapping of the target coordinates, the coordinates of the shoulder joint of the selected arm in the robot coordinate system, and a predetermined pointing motion rule.

[0032] Here, the predetermined indicated action rule can provide a reliable guarantee for the execution of the subsequent predetermined action as a constraint of the indicated action.

[0033] In the above step 120, a position point where the tip of the selected arm points to the target object within the reachable range may be obtained by mapping transformation of the target coordinates, so that the subsequent selected arm can be controlled to point to the target object.

[0034] The position and orientation in which the tip of the selected arm points to a target object within the reachable range includes the position and orientation of the tip of the selected arm of the robot in the robot coordinate system.

[0035] There are various embodiments for determining the position of the tip of the selected arm in the robot coordinate system in the above step 120. In some embodiments, a position point whose distance from the shoulder of the selected arm in the direction in which the tip of the selected arm points to the target object is smaller than the length of the selected arm may be determined as a position where the target coordinates are mapped within the reachable range of the tip of the selected arm.

[0036] In another embodiment, in the direction in which the tip of the selected arm points to the target object, the length of the selected arm may be multiplied by a reduction factor to obtain a reduced position point within the reachable range as a position where the target coordinates are mapped within the reachable range of the tip of the selected arm, where the reduction factor is greater than 0 and less than 1, thus ensuring that the tip of the arm is at a position within the reachable range. See below for a detailed description.

[0037] In yet another embodiment, the length of the selected arm may be shortened by a predetermined length in the direction in which the tip of the selected arm points to the target object, and a reduced position point within the reachable range may be obtained as a position whose target coordinates are mapped within the reachable range of the tip of the selected arm, where the predetermined length is greater than 0 and less than the arm length. In order to ensure that the selected arm is extended as much as possible, increasing the pointing range of the arm tip, and that the arm tip is within the reachable range, the predetermined length is as small as possible compared to the arm length. See below for a detailed description.

[0038] Of course, other ways by which the position of the tip of the selected arm in the robot coordinate system can be determined belong within the scope of protection of embodiments of the present invention and will not be given as examples here.

[0039] In step 130, a trajectory of the selected arm from the current position and posture of each joint of the selected arm to the target position and posture of each joint is determined based on the acquired position and posture of the arm tip. This trajectory can be the basis for controlling the selected arm to point to the target object.

[0040] In step 140, the selected arm is controlled to point to the target object based on the trajectory, and the operation of pointing to the target object at the tip of the selected arm is completed.

[0041] FIG. 2 is a schematic diagram showing a specific flow of a method for controlling a pointing motion of a robot provided in an embodiment of the present invention.

[0042] As shown in FIG. 2, before the above step 110, the method may further include, but is not limited to, the following steps 101 to 104 to determine that the target object is beyond the reachable range of the robot's arm tip.

[0043] In step 101, a robot world coordinate system, a robot coordinate system, and local coordinate systems for each joint of the two arms are created.

[0044] Here, the robot coordinate system is the base coordinate system of the robot.

[0045] In step 102, the angle at which the robot's base should rotate in order for the target object to be located in the forward range of the robot is obtained based on the angle between the vector extending from the origin in the robot coordinate system to the coordinates of the target object in the robot coordinate system and the X-axis of the robot coordinate system.

[0046] Specifically, the above step 102 may be realized by, but is not limited to, the following three steps.

[0047] (1) The coordinates of the robot in the robot's world coordinate system.

number

number

number

number

number

number

number

number

[0048] (2) Using the following equations (2) and (3), the relationship between the angle α and the direction angle θ of the front of the robot in the world coordinate system is calculated to determine the angle β at which the robot's base should rotate.

number

number

number

number

number

number

[0049] (3), the orientation angle γ after the robot’s base in the world coordinate system is rotated is obtained using the following equation (4).

number

[0050] In step 103, the coordinates of the robot in the world coordinate system

number

number

number

number

number

number

number

[0051] In step 104, it is determined that the target object is beyond the reachable range of the robot's arm end. Here, in an embodiment of the above step 104, if the target object is outside the maximum reachable range of the robot's arm end, it is determined that the target object is beyond the reachable range of the robot's arm end. In another embodiment of the above step 104, it is determined that the target object is beyond the reachable range of the robot's arm end if the distance between the target object and the two shoulder joints of the robot is greater than the length of the robot's arm.

[0052] FIG. 3 is a schematic diagram showing a simplified configuration of a robot coordinate system and a target object in the robot pointing motion control method provided in an embodiment of the present invention.

[0053] As shown in Fig. 3, the XY plane coordinate system is a coordinate system in which the robot's base coordinate system is projected onto the XY plane. Here, the XY plane coordinate system includes an origin O of the XY plane coordinate system corresponding to the robot's head position 21, a left shoulder joint 22 of the left arm, a right shoulder joint 23 of the right arm, a target object 24, and a position point 25 within the reachable range of the tip of the selected arm. A method for determining three-dimensional coordinates, which are the X coordinate, Y coordinate, and Z coordinate of the position point 25 in the robot coordinate system, will be described in detail below.

[0054] Description will be made with reference to FIG. 3. The above step 120 may include, but is not limited to, the following steps 121 to 123. In step 121, the target coordinates are mapped to coordinates within the reachable range of the tip of the selected arm.

[0055] In some embodiments of the above step 121, in the first step, based on the angle between the vector from the shoulder joint of the selected arm in the robot coordinate system to the target object and the X-axis of the robot coordinate system, and the length of the selected arm, the X coordinate and Y coordinate of the tip of the selected arm are obtained.

[0056] Exemplarily, the shoulder joint of the selected arm in the robot coordinate system

Number

Number

Number

Number

Number

Number

Number

[0057] In the second step, the Z coordinate of the tip of the selected arm is determined based on the relationship between the Z direction coordinate of the target object in the robot coordinate system, the coordinate of the shoulder joint of the selected arm, and the length of the arm.

number

number

[0058] In the above formula (8-1), the position of the target object is lower than the shoulder joint in the Z direction, so the tip of the selected arm is lower than the shoulder joint and the

number

[0059] The above formula (8-2) indicates that in the Z direction, the position of the target object is higher than the shoulder joint and lower than the sum of the heights of the shoulder joint and the arm. Therefore, the position of the target object is higher than the shoulder joint and within the reachable range of the arm.

number

[0060] The above formula (8-3) is that in the Z direction, the position of the target object is higher than the sum of the heights of the shoulder joint and the arm, so

number

[0061] In step 122, the pointing direction of the tip of the selected arm is determined based on predetermined pointing rules.

[0062] In step 123, a pose of the tip of the selected arm is generated based on the mapped coordinates, the coordinates of the shoulder joint of the selected arm in the robot coordinate system, and the pointing direction.

[0063] In some embodiments of step 123, in the first step, the unit vector of the vector from the target object to the arm tip is determined as three Z direction components for constituting the Z direction vector of the attitude matrix of the tip of the selected arm. The attitude matrix of the tip of the selected arm is a matrix with 3 rows and 3 columns. Exemplarily, the attitude matrix R of the tip of the selected arm is expressed by the following formula (9).

number

number

number

number

number

number

number

[0064] The first step may be calculated according to the following formulas (10) to (11):

number

number

number

number

number

number

[0065] In the second step, the tip of the selected arm is pointed to the target object, and the pose matrix Y component of the Y direction vector, oz is positive and the selected arm is the right arm, the Y-direction vector of the orientation matrix of Y-direction component oy is determined to be positive Then, determine the three Y-direction components that make up the Y-direction vector of the orientation matrix. .

[0066] In the third step, the tip of the selected arm is pointed to the target object, and the pose matrix Y component of the Y direction vector, ozis positive, and when the selected arm is the left arm, the Y-direction vector of the pose matrix of Y-direction component oy is determined to be negative Then, determine the three Y-direction components that make up the Y-direction vector of the orientation matrix. .

[0067] FIG. 4a is a schematic diagram showing the current configuration of a robot of an operation control method indicated by a finger of a robot provided in an embodiment of the present invention. FIG. 4b is a schematic diagram showing the target configuration of a robot of an operation control method indicated by a finger of a robot provided in an embodiment of the present invention.

[0068] As shown in FIGS. 4a and 4b, the base center position of the base coordinate system of the robot is Or. The two arms include a left arm (not shown) and a right arm 30. Hereinafter, the right arm 30 is taken as an example as the selected arm. The right arm 30 may include, but is not limited to, a right shoulder joint 31, a right upper arm 32, a right forearm 33, and a right hand 34. The same applies to the left arm, and the description thereof is omitted here.

[0069] This will be described with reference to FIGS. 4a and 4b. The above second step and third step may be realized by the following steps.

[0070] Based on the rule that when the arm points to the target object, the hand is facing obliquely upward, when the arm points to the target object, the Y-direction vector [Number] Y-direction component of oz is positive, and when the right arm points to the target object, the Y-direction vector [Number] Y-direction component oy is positive be It can be seen that this represents. On the other hand, when the left arm points to the target object, the Y-direction vector

number

number

[0071] The value of oz affects the width of the palm upwards and may be selected based on the pointing effect. In this implementation, oz=0.4 is selected, but a value close to this value may be selected based on the actual calculation results. The following equation (13) is obtained.

number

[0072]

number

number

[0073] The left arm is similar and will not be repeated here.

[0074] In the fourth step, the three X-direction components for constituting the X-direction vector of the posture matrix are determined based on the three Z-direction components and three Y-direction components of the posture matrix of the tip of the selected arm according to the right-hand rule. In this way, the realization process of solving nonlinear equations is more convenient and does not depend on existing libraries such as the library for solving nonlinear equations in MATLAB, so that the calculation speed can be improved.

[0075] The fourth step is to calculate the second column of the arm tip posture matrix according to the right-hand rule.

number

number

number

[0076] The above step 130 may further include, but is not limited to, the following two steps: In the first step, based on the acquired position and posture of the arm tip, the inverse kinematics is solved to acquire the target posture of each joint of the selected arm; In the second step, the trajectory of the selected arm from the current position and posture to the target position and posture of each joint is obtained by trajectory interpolation.

[0077] Here, the first step is A step of solving inverse kinematics based on the acquired position and orientation of the arm tip to acquire a target configuration of the selected arm, the target configuration including a target angle of each joint; The method may further include, but is not limited to, a step of performing trajectory interpolation based on the current configuration and the goal configuration of the robot, and controlling the selected arm to move from the current configuration of the robot to a goal configuration of the robot pointing to the target object, where the current configuration of the robot includes the current angles of each joint.

[0078] FIG. 5 is a schematic diagram showing a flow of measuring the movement accuracy of the robot pointing movement control method provided in the embodiment of the present invention.

[0079] 1 to 5, after the above step 140, the method may further include, but is not limited to, step 150 of measuring the position and orientation of the arm tip acquired by using a measuring device and determining an error in the operation of pointing to a target object of the tip of the selected arm. In this way, after controlling the arm to point to the target object, the deviation of the arm pointing to the target object is determined, the accuracy of the pointing operation is measured, and subsequent adjustment and optimization of the control can be facilitated.

[0080] In some embodiments, the step of measuring the position and orientation of the arm tip acquired using a measurement device and determining an error in the operation of pointing to a target object at the tip of the selected arm may include a first step and a second step. In the first step, a vector from the target object to the tip of the selected arm is determined. Specifically, the position and orientation of the arm tip may be measured using a measurement device, and a vector from the target object to the arm tip may be calculated based on the position of the target object and the measured position of the arm tip. In the second step, the angle between the vector and the Z direction vector of the orientation matrix of the tip of the selected arm is determined as the error.

[0081] Furthermore, the above step 150 may further include the following steps 1) to 3), but is not limited thereto.

[0082] 1) A laser tracker is used to measure the position and orientation of the arm tip. The coordinate system used for the measurement is the same as the robot coordinate system, and the measurement result is the actual position and orientation of the arm tip in the robot coordinate system.

number

[0083] 2) The measurement result of step 1) above and the coordinates of the target object in the robot coordinate system

number

number

[0084] 3), posture matrix

number

number

number

number

[0085] FIG. 6 is a schematic diagram showing modules of a robot pointing motion control device provided in an embodiment of the present invention.

[0086] As shown in FIG. 6, the robot's pointing motion control device includes: an arm selection module 41 for selecting one of the two arms of the robot that is closer to the target object based on target coordinates of the target object in the robot coordinate system when the target object is beyond the reachable range of the end of the arm of the robot; an arm tip position and orientation determination module 42 for acquiring a position and orientation in which the tip of the selected arm points to a target object within a reachable range based on mapping of target coordinates, coordinates of a shoulder joint of the selected arm in the robot coordinate system, and a predetermined pointing motion rule; a trajectory determination module 43 for determining a trajectory of the selected arm from a current position and posture of each joint of the selected arm to a target position and posture of each joint based on the acquired position and posture of the tip of the arm; An operation control module 44 for controlling the selected arm to point to a target object based on a trajectory is included.

[0087] In some embodiments, the position and orientation determination module 42 of the arm tip includes a coordinate mapping sub-module for mapping the target coordinates to coordinates within the reachable range of the tip of the selected arm, a pointing direction determination sub-module for determining the pointing direction of the tip of the selected arm based on the predetermined pointing operation rule, and an arm tip attitude generation sub-module for generating the attitude of the tip of the selected arm based on the mapped coordinates, the coordinates of the shoulder joint of the selected arm in the robot coordinate system, and the pointing direction.

[0088] In some embodiments, the coordinate mapping sub-module specifically obtains the X and Y coordinates of the tip of the selected arm based on the angle between the vector from the shoulder joint of the selected arm to the target object in the robot coordinate system and the X-axis of the robot coordinate system, and the length of the selected arm, and is used to obtain the Z coordinate of the tip of the selected arm based on the relationship between the Z-direction coordinate of the target object in the robot coordinate system, the coordinates of the shoulder joint of the selected arm, and the length of the arm. And / or Specifically, the arm tip attitude generation sub-module determines the unit vector of the vector from the target object to the arm tip as three Z-direction components for constructing the Z-direction vector of the attitude matrix of the tip of the selected arm. The attitude matrix of the tip of the selected arm is a 3x3 matrix. The tip of the selected arm points to the target object, and when Y component of the Y direction vector, oz is positive and the selected arm is a right arm, the Y-direction vector of the attitude matrix of Y-direction component oy is determined to be positive, determining three Y-direction components for constituting a Y-direction vector of the orientation matrix; The tip of the selected arm is pointed to a target object, and the posture matrix Y component of the Y direction vector, oz is positive and the selected arm is the left arm, the Y-direction vector of the orientation matrix of Y-direction component oy is determined to be negative, determining three Y-direction components for constituting a Y-direction vector of the orientation matrix; According to the right-hand rule, based on the three Z direction components and three Y direction components of the posture matrix of the tip of the selected arm, three X direction components are used to determine for constituting an X direction vector of the posture matrix.

[0089] In some embodiments, the device further comprises: The system further includes a pointing error determination module for measuring the acquired arm tip position and orientation using a measurement device and determining a pointing error of the selected arm tip to a target object.

[0090] In some embodiments, the pointing motion error determination module specifically comprises: determining a vector from the target object to the tip of the selected arm; The angle between said vector and the Z direction vector of the orientation matrix of the tip of the selected arm is used to determine the error.

[0091] In some embodiments, the trajectory determination module 43 comprises: a target posture determination sub-module for solving inverse kinematics based on the acquired position and posture of the arm tip to acquire a target posture of each joint of the selected arm; and a trajectory determination sub-module for obtaining a trajectory of the selected arm from a current position and orientation to a target position and orientation of each of the joints by trajectory interpolation.

[0092] In some embodiments, the end-of-arm position determination module 42 specifically: solving inverse kinematics based on the acquired position and orientation of the arm tip to acquire a target configuration of the selected arm; Based on the current configuration of the robot and the goal configuration, a trajectory interpolation is performed and used to control the selected arm to move from the current configuration of the robot to a goal configuration of the robot pointing to the target object, the goal configuration including a target angle for each of the joints, and the current configuration of the robot including a current angle for each of the joints.

[0093] The process of implementing the functions and roles of each module in the above device may be referred to as the process of implementing the corresponding steps in the above method, and the description thereof will be omitted here.

[0094] The present invention further provides a computer-readable storage medium, which stores a computer program, and when the program is executed by a processor, performs the above-mentioned robot pointing motion control method.

[0095] The present invention further provides an electronic device. Figure 7 is a block diagram showing an electronic device provided in an embodiment of the present invention.

[0096] As shown in FIG. 7, an electronic device 50 includes one or more processors 51 for implementing the robot pointing motion control method described above.

[0097] In some embodiments, electronic device 50 may include memory 59. Memory 59 may store programs that may be called by processor 51 and may include a non-volatile storage medium. In some embodiments, electronic device 50 may include internal memory 58 and interface 57. In some embodiments, electronic device 50 may include other hardware depending on the actual application.

[0098] The memory 59 in the embodiment of the present invention stores a program, and when the program is executed by the processor 51, the above-mentioned robot pointing motion control method is implemented.

[0099] The present invention may take the form of a computer program product embodied in one or more memories 59 (including, but not limited to, magnetic disk memories, CD-ROMs, optical memories, etc.) that contain program code. Memory 59 may include non-volatile and volatile media, removable and non-removable media, and may implement any method or technology for storage of information. Information may be computer-readable instructions, data structures, program modules, or other data. Memory 59 may include, but is not limited to, Phase Change Memory (PRAM), Static Random-Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, magnetic cassette tape, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that may be used to store information accessible to a computing device.

[0100] The above is only the preferred embodiment of the present invention, and is not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made without departing from the spirit and principle of the present invention should be included in the protection scope of the present invention.

[0101] Additionally, the terms "comprise," "contain," or any other variation thereof, are intended to include a non-exclusive inclusion, whereby a process, method, article, or device that includes a set of elements not only includes those elements, but also includes other elements not expressly listed, or includes the inherent elements of such process, method, article, or device. Absent more limitations, an element qualified by the phrase "comprises a..." does not exclude the presence of further identical elements in a process, method, article, or device that includes said element.

Claims

1. A method for controlling a pointing motion of a robot, comprising: When the target object is beyond a reachable range of the arm tip of the robot, selecting one of the two arms of the robot that is closer to the target object based on target coordinates of the target object in a robot coordinate system; acquiring a position and posture in which the tip of the selected arm points to the target object within a reachable range based on mapping of the target coordinates, coordinates of a shoulder joint of the selected arm in the robot coordinate system, and a predetermined pointing motion rule; determining a trajectory of the selected arm from a current position and posture of each joint of the selected arm to a target position and posture of each joint of the selected arm based on the acquired position and posture of the tip of the arm; and controlling the selected arm to point to a target object based on the trajectory; The step of acquiring a position and posture in which the tip of the selected arm points to the target object within a reachable range based on the mapping of the target coordinates, the coordinates of a shoulder joint of the selected arm in the robot coordinate system, and a predetermined pointing motion rule, includes: mapping the target coordinates to coordinates within a reachable range of the tip of the selected arm; determining a pointing direction of the tip of the selected arm based on the predetermined pointing motion rule; generating a pose of the tip of the selected arm based on the mapped coordinates, a coordinate of a shoulder joint of the selected arm in the robot coordinate system, and the pointing direction; generating a pose of the tip of the selected arm based on the mapped coordinates, a coordinate of a shoulder joint of the selected arm in the robot coordinate system, and the pointing direction, determining unit vectors of a vector directed from the target object to the arm tip of the selected arm as three Z direction components for constituting a Z direction vector of an attitude matrix of the arm tip of the selected arm, the attitude matrix of the arm tip of the selected arm being a matrix R with 3 rows and 3 columns; [0010] is an X-direction vector, ax, ay, and az are three X-direction components, 【number】 is a Y-direction vector, and ox, oy, and oz are the three Y-direction components, respectively. 【number】 is a Z-direction vector, and nx, ny, and nz are the three Z-direction components, respectively. The calculation of the three Z-direction components is as follows: [0025] Using [0030] are vectors from the target to the arm tip of the selected arm, [0045] are the components corresponding to the X, Y, and Z coordinate axes of [0050] is a vector [006] , which is the norm of,step,and, a step of determining that a Y-direction component oy of the Y-direction vector of the posture matrix is ​​positive when the tip of the selected arm is pointing at a target object, the Y-direction component oz of the Y-direction vector of the posture matrix is ​​positive, and the selected arm is a right arm, and determining three Y-direction components for constituting the Y-direction vector of the posture matrix, The calculation of the three Y-direction components is as follows: [0070] Using p is a preset value based on the width of the palm of the selected arm facing upward when the selected arm is pointing at the target object, A=nx 2 +ny 2 , B=2*oz*nz*ny, D=B 2 -4*A*C, C=oz 2 +nz 2 -nx 2 +nx 2 *oz 2 , oz>0, oy>0, ox 2 +oy 2 +oz 2 =1, ox*nx+oy*ny+oz*nz=0; a step of determining that a Y-direction component oy of the Y-direction vector of the posture matrix is ​​negative when the tip of the selected arm is pointing at a target object, the Y-direction component oz of the Y-direction vector of the posture matrix is ​​positive, and the selected arm is the left arm, and determining three Y-direction components for constituting the Y-direction vector of the posture matrix, The calculation of the three Y-direction components is as follows: [0080] p is a preset value based on the width of the palm of the selected arm facing upward when the selected arm is pointing at the target object, A=nx 2 +ny 2 , B=2*oz*nz*ny, D=B 2 -4*A*C, C=oz 2 +nz 2 -nx 2 +nx 2 *oz 2 , oz>0, oy<0, ox 2 +oy 2 +oz 2 =1, ox*nx+oy*ny+oz*nz=0; determining three X-direction components for constituting an X-direction vector of a posture matrix based on the three Z-direction components and three Y-direction components of the posture matrix of the tip of the selected arm according to the right-hand rule, The calculation of the three X-direction components is as follows: [0097] and A method for controlling a robot's pointing motion, comprising:

2. Mapping the target coordinates to coordinates within a reachable range of the tip of the selected arm includes: acquiring an X-coordinate and a Y-coordinate of a tip of the selected arm based on an angle between a vector from a shoulder joint of the selected arm toward the target object in the robot coordinate system and an X-axis of the robot coordinate system, and a length of the selected arm; obtaining a Z coordinate of a tip of the selected arm based on a relationship between a Z direction coordinate of the target object in the robot coordinate system, a coordinate of a shoulder joint of the selected arm, and a length of the arm; 2. The method for controlling a robot's pointing motion according to claim 1.

3. After the step of controlling the selected arm to point to a target object based on the trajectory, the method further comprises: measuring the acquired position and orientation of the tip of the arm using a measurement device; determining a vector from the target object to the tip of the selected arm based on the position of the target object and the measured position of the tip of the selected arm; determining an angle between the vector and a Z-direction vector of a measured attitude matrix of the tip of the selected arm as an error of the movement of the tip of the selected arm pointing to the target object, 2. The method for controlling a robot's pointing motion according to claim 1.

4. The step of determining a trajectory of the selected arm from a current position and posture of each joint of the selected arm to a target position and posture of each joint of the selected arm based on the acquired position and posture of the tip of the arm, acquiring a target angle for each joint of the selected arm by solving inverse kinematics based on the acquired position and orientation of the arm tip; and obtaining a trajectory of the selected arm from a current position and posture of each of the joints to a target position and posture of each of the joints by trajectory interpolation based on a current angle of each joint of the selected arm and the target angle.

2. The method for controlling a robot's pointing motion according to claim 1.

5. A robot pointing motion control device, comprising: an arm selection module for selecting one of the two arms of the robot that is closer to the target object based on a target coordinate of the target object in a robot coordinate system when the target object is beyond a reachable range of an arm tip of the robot; an arm-tip position and orientation determination module for acquiring a position and orientation in which the tip of the selected arm points to the target object within a reachable range based on mapping of the target coordinates, a coordinate of a shoulder joint of the selected arm in the robot coordinate system, and a predetermined pointing motion rule; a trajectory determination module for determining a trajectory of the selected arm from a current position and posture of each joint of the selected arm to a target position and posture of each joint, based on the acquired position and posture of the tip of the arm; and a motion control module for controlling the selected arm to point to a target object based on the trajectory, The arm tip position and orientation determination module a coordinate mapping submodule for mapping the target coordinates to coordinates within a reachable range of the tip of the selected arm; a pointing direction determination sub-module for determining a pointing direction of the tip of the selected arm based on the predetermined pointing motion rule; an arm end pose generation sub-module for generating a pose of the end of the selected arm based on the mapped coordinates, a coordinate of a shoulder joint of the selected arm in a robot coordinate system, and the pointing direction; The arm tip posture generation sub-module: A unit vector of a vector from the target object to the arm tip of the selected arm is determined as three Z direction components for constituting a Z direction vector of an attitude matrix of the tip of the selected arm, the attitude matrix of the tip of the selected arm being a matrix R with 3 rows and 3 columns, where: [0089] is an X-direction vector, ax, ay, and az are the three X-direction components, 【number】 is a Y-direction vector, and ox, oy, and oz are the three Y-direction components, respectively. 【number】 is a Z-direction vector, and nx, ny, and nz are the three Z-direction components, respectively. The calculation of the three Z-direction components is as follows: ##EQU00011## Using ##EQU00012## are vectors from the target to the arm tip of the selected arm, ##EQU00013## are the components corresponding to the X, Y, and Z coordinate axes of ##EQU14## is a vector ##EQU00015## is the norm of When the tip of the selected arm is pointing at a target object, a Y-direction component oz of the Y-direction vector of the posture matrix is ​​positive, and the selected arm is a right arm, it is determined that a Y-direction component oy of the Y-direction vector of the posture matrix is ​​positive, and three Y-direction components for constituting the Y-direction vector of the posture matrix are determined, where the calculation of the three Y-direction components is ##EQU00016## Using p is a preset value based on the width of the palm of the selected arm facing upward when the selected arm is pointing at the target object, A=nx 2 +ny 2 , B=2*oz*nz*ny, D=B 2 -4*A*C, C=oz 2 +nz 2 -nx 2 +nx 2 *oz 2 , oz>0, oy>0, ox 2 +oy 2 +oz 2 =1, ox*nx+oy*ny+oz*nz=0; When the tip of the selected arm is pointing at a target object, a Y-direction component oz of the Y-direction vector of the posture matrix is ​​positive, and the selected arm is a left arm, it is determined that a Y-direction component oy of the Y-direction vector of the posture matrix is ​​negative, and three Y-direction components for constituting the Y-direction vector of the posture matrix are determined, where the calculation of the three Y-direction components is ##EQU00017## p is a preset value based on the width of the palm of the selected arm facing upward when the selected arm is pointing at the target object, A=nx 2 +ny 2 , B=2*oz*nz*ny, D=B 2 -4*A*C, C=oz 2 +nz 2 -nx 2 +nx 2 *oz 2 , oz>0, oy<0, ox 2 +oy 2 +oz 2 =1, ox*nx+oy*ny+oz*nz=0; According to the right-hand rule, three X-direction components for constituting an X-direction vector of the posture matrix are determined based on the three Z-direction components and three Y-direction components of the posture matrix of the tip of the selected arm, where the calculation of the three X-direction components is performed as follows: [0018] to be carried out using, to be used for, A robot pointing motion control device comprising:

6. An electronic device including one or more processors and a memory for storing one or more programs, the one or more programs being executed by the one or more processors to cause the one or more processors to implement the robot pointing motion control method according to any one of claims 1 to 4.

1. An electronic device comprising:

7. A computer-readable storage medium storing computer instructions, which, when executed by a processor, performs the robot pointing motion control method according to any one of claims 1 to 4. A computer-readable storage medium comprising:

Images