Visual servo control method and device based on heterogeneous joint weighted motion decomposition

Abstract

This application relates to the field of robotics, and particularly to a visual servo control method and apparatus based on heterogeneous joint weighted motion decomposition. The method includes: weighting the pose deviation of the target pose measurement data based on the target pose measurement data of the agile arm; planning the expected joint angle data for the next cycle of the agile arm, and weighting the expected joint angle data to obtain the joint motion data of the agile arm; converting the joint motion data into motor motion data based on the target kinematic transformation relationship, and weighting the motor motion data according to the motor response characteristics to obtain weighted motor motion data, thereby controlling the agile arm to perform corresponding visual servo control actions. This application can weight the pose information of the target in each direction, the planned joint angle data, and the corresponding motor angle data, ensuring that the response characteristics of each motor are basically consistent, thus enabling the visual servo method to be well adapted to the motion characteristics of the tethered agile arm.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to a visual servo control method and device based on heterogeneous joint weighted motion decomposition. Background Technology

[0002] Among related technologies, the rope-driven agile arm, as a novel type of rope-driven multi-degree-of-freedom robotic arm, has advantages such as flexible movement, low inertia, and rear-mounted drive, and has great application potential in space operations, complex environment operations, and other occasions. Compared with traditional rigid robotic arms, the rope-driven agile arm has a longer transmission link between the drive end and the working end, including motor drive, drive rope transmission, joint movement, and end effector movement, in order to achieve multi-degree-of-freedom movement and small-size envelope.

[0003] However, the joint structures of rope-driven agile arms vary greatly in related technologies. The complex kinematic models and multi-layered transmission links increase the difficulty of accurately adjusting the closed-loop motion parameters of the robotic arm joints and motors. Since the response characteristics of each motor are not the same, the actual response of the joint may lag behind the command from the motor drive end. It is difficult to ensure that the target is always in the camera's field of view during the movement of the robotic arm. Visual servoing methods are difficult to adapt to the motion characteristics of rope-driven agile arms, which urgently needs to be solved. Summary of the Invention

[0004] This application provides a visual servo control method and device based on heterogeneous joint weighted motion decomposition to solve the problems in related technologies, such as the large differences in the joint structures of rope-driven agile arms, the complex kinematic models and multi-layer transmission links that increase the difficulty of accurately adjusting the closed-loop motion parameters of the robot arm joints and motors, the fact that the response characteristics of each motor are not the same, which may cause the actual response of the joint to lag behind the command of the motor drive end, the difficulty in ensuring that the target is always in the camera's field of view during the movement of the robot arm, and the difficulty of adapting the visual servo method to the motion characteristics of rope-driven agile arms.

[0005] The first aspect of this application provides a visual servo control method based on heterogeneous joint weighted motion decomposition, comprising the following steps: based on target pose measurement data of an agile arm, weighting the pose deviation of the target pose measurement data to obtain the weighted pose deviation of the agile arm; based on the weighted pose deviation, planning the expected joint angle data for the next cycle of the agile arm, and weighting the expected joint angle data to obtain the joint motion data of the agile arm; based on the target kinematic transformation relationship, converting the joint motion data into motor motion data, and weighting the motor motion data according to the response characteristics of the motor to obtain weighted motor motion data, so as to control the agile arm to perform corresponding visual servo control actions using the weighted motor motion data.

[0006] Optionally, in one embodiment of this application, the weighted processing of the pose deviation of the target pose measurement data includes: during the initial movement of visual servoing, weighting the pose deviation with visual servoing time as the independent variable to determine the weighted pose deviation; when the pose deviation between the end-effector pose of the agile arm and the target pose is less than a preset distance, weighting the pose deviation as the independent variable to determine the weighted pose deviation; and weighting the target pose deviation based on the actual needs of the agile arm to determine the weighted pose deviation.

[0007] Optionally, in one embodiment of this application, the step of planning the expected joint angle data of the agile arm in the next cycle based on the weighted pose deviation, and weighting the expected joint angle data to obtain the joint motion data of the agile arm, includes: planning the expected joint angle, expected joint angular velocity, and expected joint angular acceleration of the agile arm in the next cycle according to the pose deviation; and weighting the expected joint angle, the joint angular velocity, and the joint angular acceleration based on the joint response characteristics to obtain the joint motion data of the agile arm.

[0008] Optionally, in one embodiment of this application, the step of planning the expected joint angle, expected joint angular velocity, and expected joint angular acceleration of the agile arm in the next cycle based on the pose deviation includes: calculating the joint angle increment of the agile arm in the end-effector coordinate system based on the pose deviation, and correcting the joint angle increment based on an angular velocity threshold; and calculating the expected joint angle, the expected joint angular velocity, and the expected joint angular acceleration based on the current joint angle of the agile arm and the joint angle increment.

[0009] Optionally, in one embodiment of this application, the step of converting the joint motion data into motor motion data based on the target kinematic transformation relationship, and weighting the motor motion data according to the motor's response characteristics to obtain weighted motor motion data includes: calculating the motor angular velocity and motor angular acceleration in the motor motion data based on the joint motion data according to the target kinematic transformation relationship; and weighting the motor angular velocity and motor angular acceleration according to the motor's response characteristics to obtain weighted motor motion data.

[0010] A second aspect of this application provides a visual servo control device based on heterogeneous joint weighted motion decomposition, comprising: a first processing module, configured to perform weighted processing on the pose deviation of the target pose measurement data based on the target pose measurement data of the agile arm, to obtain the weighted pose deviation of the agile arm; a second processing module, configured to plan the expected joint angle data of the agile arm in the next cycle based on the weighted pose deviation, and perform weighted processing on the expected joint angle data to obtain the joint motion data of the agile arm; and a control module, configured to convert the joint motion data into motor motion data based on the target kinematic transformation relationship, and perform weighted processing on the motor motion data according to the response characteristics of the motor, to obtain weighted motor motion data, so as to control the agile arm to perform corresponding visual servo control actions using the weighted motor motion data.

[0011] Optionally, in one embodiment of this application, the first processing module includes: a first processing unit, configured to perform weighted processing on the pose deviation with visual servoing time as the independent variable during the initial movement of visual servoing, to determine the weighted pose deviation; a second processing unit, configured to perform weighted processing on the pose deviation as the independent variable when the pose deviation between the end-effector pose of the agile arm and the target pose is less than a preset distance, to determine the weighted pose deviation; and a third processing unit, configured to perform weighted processing on the target pose deviation based on the actual needs of the agile arm, to determine the weighted pose deviation.

[0012] Optionally, in one embodiment of this application, the second processing module includes: a planning unit, configured to plan the expected joint angle, expected joint angular velocity, and expected joint angular acceleration of the agile arm in the next cycle based on the pose deviation; and a fourth processing unit, configured to perform weighted processing on the expected joint angle, the joint angular velocity, and the joint angular acceleration based on the joint response characteristics to obtain the joint motion data of the agile arm.

[0013] Optionally, in one embodiment of this application, the planning unit includes: a first calculation subunit, configured to calculate the joint angle increment of the agile arm in the end-effector coordinate system based on the pose deviation, and correct the joint angle increment based on an angular velocity threshold; and a second calculation subunit, configured to calculate the desired joint angle, the desired joint angular velocity, and the desired joint angular acceleration based on the current joint angle of the agile arm and the joint angle increment.

[0014] Optionally, in one embodiment of this application, the control module includes: a calculation unit, configured to calculate the motor angular velocity and motor angular acceleration in the motor motion data based on the target kinematic transformation relationship and the joint motion data; and a fifth processing unit, configured to perform weighted processing on the motor angular velocity and the motor angular acceleration according to the response characteristics of the motor to obtain weighted motor motion data.

[0015] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the program to implement the visual servo control method based on heterogeneous joint weighted motion decomposition as described in the above embodiments.

[0016] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described visual servo control method based on heterogeneous joint weighted motion decomposition.

[0017] A fifth aspect of this application provides a computer program product, including a computer program that, when executed, is used to implement the above-described visual servo control method based on heterogeneous joint weighted motion decomposition.

[0018] This application's embodiments can weight the pose information of the target in each direction, enabling the target to move quickly to the center of the camera's field of view, ensuring the target is within the camera's field of view. Weighting of planned joint angles, joint angular velocities, and joint angular accelerations, and increasing or decreasing the increment of each planned data, ensures the synchronization of joint movements. Weighting of the output motor angular velocities and motor angular accelerations ensures that the corresponding characteristics of each motor are basically consistent. This allows the visual servoing method to be well adapted to the motion characteristics of the rope-driven agile arm. Therefore, it solves the problems in related technologies, such as the large differences in the joint structures of rope-driven agile arms, the complex kinematic models and multi-layered transmission links increasing the difficulty of accurately adjusting the closed-loop motion parameters of the robotic arm joints and motors, the inconsistent response characteristics between motors potentially causing the actual joint response to lag behind the motor drive commands, the difficulty in ensuring the target remains within the camera's field of view during robotic arm movement, and the difficulty in adapting the visual servoing method to the motion characteristics of rope-driven agile arms.

[0019] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0020] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0021] Figure 1 This is a schematic diagram of a rope-driven agile arm according to an embodiment of this application;

[0022] Figure 2 This is a flowchart of a visual servo control method based on heterogeneous joint weighted motion decomposition provided in an embodiment of this application;

[0023] Figure 3 This is a flowchart of a visual servo control method based on heterogeneous joint weighted motion decomposition according to an embodiment of this application;

[0024] Figure 4 This is a schematic diagram of the structure of a visual servo control device based on heterogeneous joint weighted motion decomposition according to an embodiment of this application;

[0025] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application.

[0026] Figure label:

[0027] 10-Visual servo control device based on heterogeneous joint weighted motion decomposition; 100-First processing module, 200-Second processing module and 300-Control module; 501-Memory, 502-Processor and 503-Communication interface. Detailed Implementation

[0028] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0029] The following describes a visual servo control method and apparatus based on heterogeneous joint weighted motion decomposition according to embodiments of this application, with reference to the accompanying drawings. Addressing the significant differences in joint structures among the joints of the cable-driven agile arm mentioned in the background art, the complex kinematic models and multi-layered transmission links increase the difficulty of precisely adjusting the closed-loop motion parameters of the robotic arm joints and motors. Due to the varying response characteristics of each motor, the actual response of the joint may lag behind the commands from the motor drive end, making it difficult to ensure the target remains within the camera's field of view during robotic arm movement. This makes it difficult for visual servo methods to adapt to the motion characteristics of cable-driven agile arms. This application provides a visual servo control method based on heterogeneous joint weighted motion decomposition. In this method, the pose information of the target in each direction is weighted, enabling the target to quickly move to the center of the camera's field of view, ensuring the target remains within the camera's field of view. The planned joint angles, joint angular velocities, and joint angular accelerations are weighted, and the synchronization of joint movements is ensured by increasing or decreasing the increment of each planned data. The output motor angular velocities and motor angular accelerations are weighted to ensure that the corresponding characteristics of each motor are basically consistent. This enables the visual servo method to adapt well to the motion characteristics of cable-driven agile arms. This solves the problems in related technologies, such as the large differences in the joint structures of rope-driven agile arms, the complex kinematic models and multi-layer transmission links that increase the difficulty of accurately adjusting the closed-loop motion parameters of the robotic arm joints and motors, the fact that the response characteristics of each motor are not the same, which may cause the actual response of the joint to lag behind the command of the motor drive end, the difficulty in ensuring that the target is always in the camera's field of view during the movement of the robotic arm, and the difficulty of adapting visual servoing methods to the motion characteristics of rope-driven agile arms.

[0030] The following is an explanation of the schematic diagram of the rope-driven agile arm involved in this application.

[0031] like Figure 1 As shown, the controlled object in this embodiment is a rope-driven agile arm. The robotic arm has seven joints, including two shoulder joints. These two joints are driven by motors connected via reducers. These two joints are also connected to elbow and wrist joint drive boxes, which contain five drive motors for the elbow and wrist joints. The motors rotate to pull the connected drive ropes, causing each joint to rotate. The robotic arm includes two elbow joints and three wrist joints. Wrist joints one and two are universal joints, driven by two motors. Wrist joint three is a roll joint. Each joint has a joint encoder to measure the joint's rotation angle, and a resolver at the motor end to measure the motor's rotation angle. The robotic arm's closed-loop control method is a joint-level position closed loop + a motor-level position closed loop.

[0032] When the end-effector camera is far from the target, it can see a larger range. The closer it gets to the target, the smaller the observable range becomes, which can easily cause the target to be out of the field of view. Therefore, it is necessary to move the target to the center of the field of view as soon as possible to ensure that the target remains in the camera's field of view during the movement of the robotic arm.

[0033] Specifically, Figure 2 This is a flowchart illustrating a visual servo control method based on heterogeneous joint weighted motion decomposition, provided in an embodiment of this application.

[0034] like Figure 2 As shown, the visual servoing control method based on heterogeneous joint weighted motion decomposition includes the following steps:

[0035] In step S201, based on the target pose measurement data of the agile arm, the pose deviation of the target pose measurement data is weighted to obtain the pose deviation of the agile arm after weighting.

[0036] It is understandable that "agile arm" here refers to a rope-driven agile arm. As a new type of rope-driven multi-degree-of-freedom robotic arm, it has advantages such as flexible movement, low inertia, and rear-mounted drive, and has great application potential in space operations, complex environment operations, and other occasions. Visual servoing is an important means for robotic arms to operate autonomously online. The end effector of the robotic arm is equipped with a hand-eye camera, which can use the camera to observe the surrounding environment and calculate the target pose in real time. Based on the target pose information, the robotic arm plans and controls its movement to the desired position.

[0037] Target pose measurement data can be understood here as the specific position and orientation that the robotic arm's end effector needs to achieve in three-dimensional space. This pose can be described by position and orientation: position can be understood as the specific coordinates of the robotic arm's end effector in space, usually represented by {x, y, z} in a three-dimensional coordinate system. These coordinate values ​​define the precise position of the end effector relative to a reference coordinate system (such as the world coordinate system or the robotic arm's base coordinate system); orientation refers to the orientation or direction of the robotic arm's end effector in space, describing the rotational state of the end effector relative to a reference coordinate system. Orientation can be described in various ways, such as rotation matrices, Euler angles (i.e., rotation angles about the x-axis, y-axis, and z-axis), etc.

[0038] In some embodiments, when controlling the rope-driven agile arm, target pose measurement data can be input first, and then the input target pose measurement data can be weighted with pose deviations in each direction step by step, so as to use the data to control the agile arm.

[0039] The embodiments of this application can weight the pose deviation of the agile arm, so that the target moves quickly to the center of the field of view, ensuring that the target is in the field of view. Based on the adjustment capability of the agile arm end effector in each direction and the range of target operation, the robotic arm can quickly reach the working range.

[0040] The following section will further elaborate on the process of weighting the pose deviation of the target pose measurement data.

[0041] Optionally, in one embodiment of this application, the pose deviation of the target pose measurement data is weighted, including: during the initial movement of visual servoing, the pose deviation is weighted with the visual servoing time as the independent variable to determine the weighted pose deviation; a second processing unit is used to perform weighted processing with the pose deviation as the independent variable when the pose deviation between the end-effector pose of the agile arm and the target pose is less than a preset distance, to determine the weighted pose deviation; and a third processing unit is used to perform weighted processing on the target pose deviation based on the actual needs of the agile arm to determine the weighted pose deviation.

[0042] In actual execution, the input target pose measurement data can be understood as the pose deviation of the target relative to the end of the agile arm.

[0043] When weighting the pose deviation of the target pose measurement data, the magnitudes of the position deviation vector and the attitude deviation vector relative to the end effector of the agile arm can be calculated first. For example, the input data is the pose deviation of the target relative to the end effector of the agile arm [P]. x P y P z R x R y R z ] Calculate the magnitude ||P|| of the position deviation vector and the magnitude ||R|| of the attitude deviation vector of the target relative to the end of the agile arm, respectively. The calculation formula can be expressed as follows:

[0044]

[0045] Next, the ratio of pose deviation to the end-effector step size threshold for a single cycle of the agile arm can be calculated to reduce the pose deviation to within the threshold range. Specifically, the end-effector position step size can be set as P0, and the end-effector attitude step size as R0. The position step size ratio can be expressed as: k A =||P|| / P0, the attitude step size ratio can be expressed as: k R =||R|| / R0. When k A When the value is greater than 1, the pose deviation can be expressed as:

[0046] [P x P y Pz ] = [P x P y P z ] / k A ,

[0047] [R x R y R z ] = [R x R y R z ] / k R .

[0048] Furthermore, in some cases where actual needs differ, the pose deviations in each direction can be amplified or reduced according to the requirements: for example, when it is necessary to quickly move the target to the center of the end-camera's field of view, the position deviations in the x and y directions can be appropriately amplified, while the position deviation in the z direction can be appropriately reduced. In this case:

[0049] [P x P y P z ] = [k x P x k y P y k z P z ](k x k y >1,k z <1),

[0050] Where, k x k y k z These represent the position deviation weighting coefficients in the x, y, and z directions, respectively.

[0051] For example, when the tolerance of the end effector of the robotic arm is large in the x-axis direction, the attitude deviation around the y and z axes can be appropriately increased, while the attitude deviation around the x-axis can be reduced.

[0052] When the agile arm begins to move, the pose deviation can be incremented by the input time of the target pose measurement data. This allows the agile arm to gradually accelerate to a constant speed during the initial movement phase, reducing potential errors during the movement and maintaining a stable state. For example, the pose deviation can be controlled using a PID (Proportional-Integral-Derivative) controller.

[0053] With pose deviation [P x P y P z R x, R y , R z the P in x is taken as an example to explain the process of PID control for pose deviation. After PID control, P x is:

[0054] P x = k P P x + k I OP x + k D dP x

[0055] dP x = P x - P x ′

[0056] OP x = OP x ′ + P x

[0057] Among them, k P , k I , k D are the proportional, integral, and differential control parameters for PID control of P x respectively. OP x is the accumulated amount of the position deviation in the x direction, and dP x is the difference between the position deviation in the x direction in this cycle and the position deviation in the x direction in the previous cycle. P x ′ is the position deviation in the x direction in the previous cycle, and OP x ′ is the accumulated amount of the position deviation in the x direction up to the previous cycle.

[0058] When the running time t < t0, that is, in a period of time at the beginning of visual servo, the robotic arm needs to accelerate slowly to prevent the robotic arm from shaking due to excessive acceleration. The parameters of the proportional control coefficient k P and the differential control coefficient k D can be expressed as:

[0059]

[0060] Among them, t is the current running time, and t0 is the time threshold. are the preset proportional control coefficient and differential control coefficient respectively.

[0061] When the running time t > t0, the P and D parameters can be set to and At this time, the robotic arm enters a relatively stable motion stage, and the robotic arm is relatively close to uniform motion.

[0062] When the target's distance from the end effector of the robotic arm is small, to avoid misoperation, the pose deviation can be used as the independent variable. This allows the robotic arm to move at a lower speed as it gets closer to the target, avoiding uncertainties caused by excessive speed and facilitating precise control of the robotic arm. Specifically, the pose deviation can be calculated using PID control. When ||P|| < P0 and ||R|| < R0, it indicates that the end effector of the robotic arm is very close to the target and needs to decelerate to approach it. At this point, k can be compared... A =||P|| / P0 and k R = The size of ||R|| / R0:

[0063] When k R <k A When this occurs, it indicates that the position deviation is larger than the attitude deviation, and the proportional control coefficient k in the PID control is... P and differential control coefficient k D It can be represented as:

[0064]

[0065] When k R >k P When this occurs, it indicates that the attitude deviation is larger than the position deviation, and the proportional control coefficient k in the PID control is... P and differential control coefficient k D It can be represented as:

[0066]

[0067] Wherein, P0 and R0 are the position deviation threshold and attitude deviation threshold, respectively.

[0068] Taking the x-direction position deviation as an example again, the output x-direction position deviation can be expressed as:

[0069] P x =k P P x +k I OP x +k D dP x

[0070] dP x =P x -P x ′

[0071] OP x =OP x ′+P x

[0072] Among them, P x ' represents the positional deviation in the x-direction of the previous cycle, OP x′ represents the cumulative position deviation in the x-direction up to the previous period.

[0073] In the embodiments of this application, the step length can be weighted at the beginning of the visual servoing stage to make the robotic arm accelerate slowly, so as to prevent the robotic arm joints and motors from oscillating due to excessive acceleration. When the positional deviation between the robotic arm end effector and the target is within a certain range, the step length can be weighted to reduce the movement speed of the robotic arm, so as to prevent the robotic arm end effector from colliding with the target prematurely and causing equipment damage.

[0074] Step S202: Based on the weighted pose deviation, plan the expected joint angle data for the next cycle of the agile arm, and perform weighted processing on the expected joint angle data to obtain the joint motion data of the agile arm.

[0075] In other embodiments, after weighting the pose deviation of the target pose measurement data, the joint angle related data of the agile arm in the next cycle can be planned based on the weighted pose deviation. This includes, but is not limited to, expected joint angles, expected joint angular velocities, and expected joint angular accelerations. Based on the planned joint angle data and the response characteristics of each joint, the planned expected joint angles, expected joint angular velocities, and expected joint angular accelerations are weighted to obtain the joint motion data of the agile arm in the next cycle, which can then be further utilized.

[0076] The embodiments of this application can weight the joint angles, joint angular velocities, and joint angular accelerations of the visual servo planning, so that each joint can move to the planned joint angle within a specified time period, or ensure that the movement of each joint is synchronized, avoiding large deviations between the movement of some joints and the planned joint angles, which would result in a large difference between the overall arm movement effect and the desired arm shape, thus affecting the visual servo effect.

[0077] The process will now be explained in more detail.

[0078] Optionally, in one embodiment of this application, based on the weighted pose deviation, the expected joint angle data for the next cycle of the agile arm is planned, and the expected joint angle data is weighted to obtain the joint motion data of the agile arm, including: planning the expected joint angle, expected joint angular velocity and expected joint angular acceleration for the next cycle of the agile arm according to the pose deviation; and weighting the expected joint angle, joint angular velocity and joint angular acceleration based on the joint response characteristics to obtain the joint motion data of the agile arm.

[0079] Based on the descriptions of other embodiments, it is understood that after weighting the pose deviation, the expected joint angle data for the next cycle of the agile arm can be planned based on the weighted pose deviation, thereby obtaining the subsequent joint motion data of the agile arm.

[0080] For example, during the calculation process, the expected joint angles, expected joint angular velocities, and expected joint angular accelerations for the agile arm in the next cycle can be planned based on the pose deviation. Then, the joint angle increments between the current pose and the planned pose of the agile arm can be calculated, and by weighting each joint angle increment, the joint motion data required by the agile arm for the next cycle can be obtained.

[0081] For example, for joints with slow response speeds, the joint angle increment can be amplified. For joint PID closed-loop control based on kinematic feedforward, the joint angle output by the control loop can be expressed as:

[0082] q = q0 + dq + k Pq dq+k Iq fdq+k Dq ddq

[0083] Where q0 is the current joint angle, dq is the joint angle increment calculated using pose deviation, q0+dq is the planned joint angle, and k Pq dq is the proportional term of the joint closed-loop control, k Pq k is the joint angle proportional control coefficient. Iq fdq is the integral term of the joint closed-loop control, k Iq Here, fdq is the integral control coefficient for the joint angle, and k is the cumulative amount of the joint angle increment. Dq ddq is the differential term of the joint closed-loop control, k Dq Here, is the differential control coefficient for the joint angle, and ddq is the difference between the joint angle increment in the current cycle and the joint angle increment in the previous cycle. The larger the joint angle output by the control loop, the larger the motor angle increment translated to the motor end through the kinematic model. The increased increment will improve the motor's movement speed and output torque, and accelerate its movement to the desired position.

[0084] For the proportional term k in joint closed-loop control Pq dq, the proportional term of the closed-loop control for each joint angle is k Pqi dq i (i = 1, 2, ..., 7), where k Pqi dq are the weighting coefficients. i For the increments of each joint angle, then:

[0085] For the shoulder joint, since its rotation is directly driven by a motor without being transmitted through cables, the joint response characteristics are relatively fast. Therefore, the weighting coefficient k for shoulder 1 and shoulder 2 joints is... Pq1 k Pq2 It can be smaller, for example, 1.

[0086] Elbow 1 and Elbow 2 joints are driven by a motor via a drive cable. The transmission of motion via the cable causes hysteresis. The weighting coefficient k... Pq3 k Pq4 It can be increased appropriately, for example, by taking 1.5.

[0087] Wrist joints 1, 2, and 3 are driven by a motor via a drive cable. The cable's transmission causes motion lag because the drive cable needs to pass through the shoulder and elbow to reach the wrist; its transmission chain is relatively long, making the cable lag more pronounced. Therefore, the weighting coefficient k... Pq5 k Pq6 k Pq7 It needs to be increased further, for example, by taking 2.

[0088] This allows us to obtain the joint motion data for the next step of the agile arm, and then process the joint motion data to control the agile arm to reach the target pose.

[0089] The following is a further explanation of the expected joint angle, expected joint angular velocity, and expected joint angular acceleration of the agile arm in the next cycle based on the pose deviation in the embodiments of this application.

[0090] Optionally, in one embodiment of this application, planning the expected joint angle, expected joint angular velocity, and expected joint angular acceleration of the agile arm in the next cycle based on the pose deviation includes: calculating the joint angle increment of the agile arm in the end-effector coordinate system based on the pose deviation, and correcting the joint angle increment based on the angular velocity threshold; calculating the expected joint angle, expected joint angular velocity, and expected joint angular acceleration based on the current joint angle and the joint angle increment of the agile arm.

[0091] In some embodiments, when calculating the joint angle data for the agile arm in the next cycle based on the pose deviation, the joint angle increment between the target pose and the current pose can be calculated first. Specifically, the calculation process can be represented as follows:

[0092] (1) Calculate the Jacobian matrix between the joint angles and the end-effector pose of the agile arm in the end-effector coordinate system. n , can be represented as follows:

[0093]

[0094] in, n R0 is the rotation transformation matrix of the robot arm's base coordinate system relative to the end effector coordinate system, and J0 is the Jacobian matrix between the robot arm's joint angles and the end effector pose in the base coordinate system.

[0095] (2) Calculate the end-effector pose increment:

[0096] The weighted pose deviation [P] x P y P z R x R y R z Transform into a homogeneous transformation matrix n Tt Let the current end pose be n T n This can be represented as follows:

[0097]

[0098] The end-effector pose increment dA can be expressed as:

[0099]

[0100] dr = 0.5(n0×n + o0×o + a0×a)

[0101] Where, n0 = [n x0 n y0 n z0 ] T , o0 = [o x0 o y0 n z0 ] T , a0 = [a x0 a y0 a z0 ] T , are the cosines of the angles between the x, y, and z axes of the target coordinate system and the axes of the robotic arm's end effector coordinate system, respectively, n = [n x n y n z ] T , o = [o x o y n z ] T , a = [a x a y a z ] T , , are the cosines of the angles between the x, y, and z axes of the end-effector coordinate system and the axes of the end-effector coordinate system of the robotic arm, respectively.

[0102] (3) Calculate the inverse of the Jacobian matrix:

[0103] This application embodiment can use the Jacobian pseudo-inverse method to calculate the Jacobian matrix J between the joint angles and the end effector pose of the robotic arm. n The inverse matrix of the Jacobian matrix is ​​obtained, and the singularity avoidance method of DLS (Damped Least Squares) is used to prevent matrix singularity. The formula for solving the inverse matrix of the Jacobian matrix can be expressed as follows:

[0104] J n + =(J n T J n +λ 2 I)-1 J n T

[0105] Among them, J n + J is the inverse of the Jacobian matrix. n T Let I be the transpose of the Jacobian matrix, I be the identity matrix, and λ be the DLS singularity parameter.

[0106] (4) Calculate the joint angle increment of the agile arm. The formula can be expressed as follows:

[0107] dq = J n + dA;

[0108] Where dq is the joint angle increment of the agile arm, and dA is the end-effector pose deviation (increment).

[0109] (5) Correct the joint angle increment based on the joint angular velocity threshold:

[0110] If some joint angle increments are greater than the joint angular velocity threshold, the ratio of each joint angle increment to its respective velocity threshold can be calculated, and the largest ratio can be selected. The overall joint angle increments can then be reduced by that factor, thereby correcting unsuitable joint angle increments.

[0111] (6) Calculate the desired joint angle based on the current joint angle and angle increment, determine whether the desired joint angle exceeds the joint angle mechanical limit, and if it exceeds the limit, assign the limit angle and recalculate the joint angle increment.

[0112] (7) Calculate the desired joint angular velocity and joint angular acceleration, and weight each joint angular velocity and angular acceleration. The joint angular velocity is obtained by dividing the joint angular increment by the period. The joint angular acceleration is obtained by subtracting the joint angular velocity from the joint angular velocity of the previous period. For some control systems, when using joint angular velocity or joint angular acceleration as control input, weighting can also be performed according to the response characteristics of each joint.

[0113] Step S203: Based on the target kinematic transformation relationship, the joint motion data is converted into motor motion data, and the motor motion data is weighted according to the motor response characteristics to obtain weighted motor motion data, so as to control the agile arm to perform corresponding visual servo control actions using the weighted motor motion data.

[0114] It is understandable that the target kinematic transformation relationship here can be understood as the kinematic transformation relationship between the joints of the agile arm and the motor. Taking the rope-driven agile arm as an example, the working principle is to drive the rope movement through the motor to realize the various movements of the robotic arm. Its joint movement is realized by the drive mechanism composed of ropes and pulleys, while the position and posture of the robotic arm are controlled by the length and angle of the rope.

[0115] Therefore, in this embodiment, joint motion data can be converted into motor motion data through the target kinematic transformation relationship, and then the motor motion data is weighted. Finally, control commands are sent to the agile arm based on the weighted motor motion data to control the agile arm to perform corresponding actions.

[0116] The motor motion data includes, but is not limited to, the desired motor angle, desired motor angular velocity, and desired motor angular acceleration. Furthermore, when weighting the motor motion data, the response characteristics of each motor must also be taken into account.

[0117] The following section will further explain the process of data transformation and weighted processing of motor motion data.

[0118] Optionally, in one embodiment of this application, joint motion data is converted into motor motion data based on the target kinematic transformation relationship, and the motor motion data is weighted according to the motor's response characteristics to obtain weighted motor motion data. This includes: calculating the motor angular velocity and motor angular acceleration in the motor motion data based on the joint motion data according to the target kinematic transformation relationship; and weighting the motor angular velocity and motor angular acceleration according to the motor's response characteristics to obtain weighted motor motion data.

[0119] In actual implementation, the planned joint motion data can be converted into motor motion data based on the kinematic transformation relationship between the agile arm joints and the motor. For example, the motor angular velocity and angular acceleration can be calculated based on the current joint angle and the desired joint angle. It should be noted that the conversion and calculation process, as well as the tools used, can be selected by those skilled in the art according to the actual situation. This is only an illustrative example and no specific limitations are imposed.

[0120] Next, the angular velocity and angular acceleration of each motor are weighted according to their response speed. For example, for some motors, when the desired motion increment is small, the motor may struggle to overcome its starting torque due to its small step size. For rope-driven agile arms, if the proportional control coefficient in the motor's PID control parameters is too large, it can easily lead to arm oscillation. Therefore, the motor's response and tracking characteristics cannot be improved solely by adjusting the proportional control coefficient. Specifically, the motor angle output from the motor control terminal can be expressed as:

[0121] Q = Q0 + dQ + kPQ dQ+kI Q fdQ+k DQ ddQ

[0122] Where Q0 is the current motor angle, dQ is the motor angular velocity calculated using the kinematic model based on the joint angle increment, and it is calculated as the difference between the motor angle converted from the desired joint angle output in step C and the motor angle converted from the current joint angle, k PQ dQ is the proportional term of the motor closed-loop control, k PQ k is the proportional control coefficient for the motor. IQ fdQ is the integral term of the motor closed-loop control, k IQ Here, fdQ is the integral control coefficient of the motor, and k is the cumulative change in motor speed. DQ ddQ is the differential term of the motor closed-loop control, k DQ ddQ is the differential control coefficient of the motor, and ddQ is the difference between the motor angular velocity in the current cycle and the motor angular velocity in the previous cycle.

[0123] For the proportional term k in motor closed-loop control PQ dQ, the proportional term of the closed-loop control for each motor is k pQi dQ i (i = 1, 2, 3, ... 7), where k pQi dQ represents the proportional control coefficient for each motor. i Here are the angular velocities of each motor.

[0124] Different proportional control coefficients can be set for different motors and different motor angle increments. For motors with slower response speeds, the proportional control coefficient can be increased to improve their tracking ability. When the motor angle increment is small, the proportional control coefficient can be increased to improve its response ability.

[0125] The embodiments of this application can weight the output motor angular velocity and motor angular acceleration, so that each motor can smoothly move to the desired motor angle within a specified time period, avoiding the influence of different motor response speeds on the motion effect of each motor.

[0126] The following detailed description of an embodiment of this application is based on a specific example.

[0127] like Figure 3 The diagram shown is a flowchart of a visual servo control method based on heterogeneous joint weighted motion decomposition for a rope-driven agile arm according to an embodiment of this application.

[0128] Step S301, Begin.

[0129] Step S302: Input the current joint angle and the pose of the target relative to the end of the agile arm.

[0130] Step S303, input pose measurement data preprocessing, including: calculating the magnitude of the target position vector and the magnitude of the target pose vector; calculating the ratio of the magnitude to the end-effector step length threshold to reduce the pose deviation to the threshold range; weighting the pose deviation; and correcting the pose deviation based on the running time and pose deviation.

[0131] Step S304, joint angle planning, includes: calculating the Jacobian matrix between the robot arm end-effector pose and the joint angle; calculating the pose difference dA between the target pose and the robot arm end-effector; then calculating the inverse of the Jacobian matrix and using the Jacobian pseudo-inverse method to calculate the inverse of the Jacobian matrix to calculate the joint angle increment; and correcting the joint angle increment according to the joint angular velocity threshold.

[0132] Step S305, joint angle planning data post-processing, includes: weighted processing of joint angle increments; calculation of desired joint angles and correction of joint angles according to joint limits; calculation of joint angular velocity and angular acceleration, and weighted processing.

[0133] Step S306, post-processing of motor angle control data, includes: converting the planned joint angles into motor angles based on the kinematic model; calculating the motor angular velocity and angular acceleration, and performing weighted processing.

[0134] Step S307: Determine whether the target has reached the capture area. If the target has not reached the capture area, restart.

[0135] In step S308, after the target arrives at the capture area, the robotic arm can reach the capture position.

[0136] Step S309, End.

[0137] The visual servoing control method based on heterogeneous joint weighted motion decomposition proposed in this application can weight the pose information of the target in each direction, enabling the target to move quickly to the center of the camera's field of view, ensuring that the target is within the camera's field of view. It weights the planned joint angles, joint angular velocities, and joint angular accelerations, ensuring the synchronization of joint movements by increasing or decreasing the increment of each planned data point. It also weights the output motor angular velocities and motor angular accelerations, ensuring that the corresponding characteristics of each motor are basically consistent. This allows the visual servoing method to be well adapted to the motion characteristics of a rope-driven agile arm. This solves the problems in related technologies, such as the large differences in the joint structures of rope-driven agile arms, the complex kinematic models and multi-layered transmission links increasing the difficulty of accurately adjusting the closed-loop motion parameters of the robotic arm joints and motors, the inconsistent response characteristics between motors potentially causing the actual joint response to lag behind the motor drive commands, the difficulty in ensuring the target remains within the camera's field of view during robotic arm movement, and the difficulty in adapting the visual servoing method to the motion characteristics of rope-driven agile arms.

[0138] Next, referring to the accompanying drawings, a visual servo control device based on heterogeneous joint weighted motion decomposition proposed according to an embodiment of this application is described.

[0139] Figure 4 This is a schematic diagram of the structure of the visual servo control device based on heterogeneous joint weighted motion decomposition according to an embodiment of this application.

[0140] like Figure 4 As shown, the visual servo control device 10 based on heterogeneous joint weighted motion decomposition includes: a first processing module 100, a second processing module 200, and a control module 300.

[0141] The first processing module 100 is used to perform weighted processing on the pose deviation of the target pose measurement data based on the target pose measurement data of the agile arm, so as to obtain the pose deviation after weighted processing by the agile arm.

[0142] The second processing module 200 is used to plan the expected joint angle data of the agile arm in the next cycle based on the weighted pose deviation, and to perform weighted processing on the expected joint angle data to obtain the joint motion data of the agile arm.

[0143] The control module 300 is used to convert joint motion data into motor motion data based on the target kinematic transformation relationship, and to perform weighted processing on the motor motion data according to the response characteristics of the motor to obtain weighted motor motion data, so as to control the agile arm to perform corresponding visual servo control actions using the weighted motor motion data.

[0144] Optionally, in one embodiment of this application, the first processing module 100 includes: a first processing unit, a second processing unit, and a third processing unit.

[0145] The first processing unit is used to perform weighted processing on the pose deviation with the visual servoing time as the independent variable during the initial motion period of visual servoing, so as to determine the pose deviation after weighted processing.

[0146] The second processing unit is used to perform weighted processing with the pose deviation as the independent variable when the pose deviation between the end pose of the agile arm and the target pose is less than a preset distance, so as to determine the pose deviation after weighted processing.

[0147] The third processing unit is used to perform weighted processing on the target pose deviation based on the actual needs of the agile arm, so as to determine the pose deviation after weighted processing.

[0148] Optionally, in one embodiment of this application, the second processing module 200 includes a planning unit and a fourth processing unit.

[0149] The planning unit is used to plan the desired joint angle, desired joint angular velocity, and desired joint angular acceleration for the agile arm in the next cycle based on the pose deviation.

[0150] The fourth processing unit is used to perform weighted processing on the desired joint angle, joint angular velocity and joint angular acceleration based on the joint response characteristics to obtain the joint motion data of the agile arm.

[0151] Optionally, in one embodiment of this application, the planning unit includes: a first calculation subunit and a second calculation subunit.

[0152] The first calculation subunit is used to calculate the joint angle increment of the agile arm in the end-effector coordinate system based on the pose deviation, and to correct the joint angle increment based on the angular velocity threshold.

[0153] The second calculation subunit is used to calculate the desired joint angle, desired joint angular velocity, and desired joint angular acceleration based on the agile arm's current joint angle and joint angle increment.

[0154] Optionally, in one embodiment of this application, the control module 300 includes a computing unit and a fifth processing unit.

[0155] The calculation unit is used to calculate the motor angular velocity and motor angular acceleration in the motor motion data based on the target kinematic transformation relationship and the joint motion data.

[0156] The fifth processing unit is used to perform weighted processing on the motor angular velocity and motor angular acceleration according to the motor's response characteristics to obtain the weighted motor motion data.

[0157] It should be noted that the foregoing explanation of the visual servo control method based on heterogeneous joint weighted motion decomposition also applies to the visual servo control device based on heterogeneous joint weighted motion decomposition in this embodiment, and will not be repeated here.

[0158] The visual servo control device based on heterogeneous joint weighted motion decomposition proposed in this application can weight the pose information of the target in each direction, enabling the target to move quickly to the center of the camera's field of view, ensuring that the target is within the camera's field of view. It weights the planned joint angles, joint angular velocities, and joint angular accelerations, ensuring the synchronization of joint movements by increasing or decreasing the increment of each planned data point. It also weights the output motor angular velocities and motor angular accelerations, ensuring that the corresponding characteristics of each motor are basically consistent. This allows the visual servo method to be well adapted to the motion characteristics of a rope-driven agile arm. This solves the problems in related technologies, such as the large differences in the joint structures of rope-driven agile arms, the complex kinematic models and multi-layered transmission links increasing the difficulty of accurately adjusting the closed-loop motion parameters of the robotic arm joints and motors, the inconsistent response characteristics between motors potentially causing the actual joint response to lag behind the motor drive commands, the difficulty in ensuring the target remains within the camera's field of view during robotic arm movement, and the difficulty in adapting the visual servo method to the motion characteristics of rope-driven agile arms.

[0159] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0160] The memory 501, the processor 502, and the computer program stored on the memory 501 and capable of running on the processor 502.

[0161] When the processor 502 executes the program, it implements the visual servo control method based on heterogeneous joint weighted motion decomposition provided in the above embodiments.

[0162] Furthermore, electronic devices also include:

[0163] Communication interface 503 is used for communication between memory 501 and processor 502.

[0164] The memory 501 is used to store computer programs that can run on the processor 502.

[0165] The memory 501 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0166] If the memory 501, processor 502, and communication interface 503 are implemented independently, then the communication interface 503, memory 501, and processor 502 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0167] Optionally, in a specific implementation, if the memory 501, processor 502, and communication interface 503 are integrated on a single chip, then the memory 501, processor 502, and communication interface 503 can communicate with each other through an internal interface.

[0168] Processor 502 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0169] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described visual servo control method based on heterogeneous joint weighted motion decomposition.

[0170] This application also provides a computer program product, including a computer program that can run computer instructions. When the computer instructions are executed by a processor, they implement the visual servo control method based on heterogeneous joint weighted motion decomposition provided in this application.

[0171] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0172] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0173] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0174] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0175] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or more of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0176] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0177] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0178] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A visual servoing control method based on heterogeneous joint weighted motion decomposition, characterized in that, Includes the following steps: Based on the target pose measurement data of the agile arm, the pose deviation of the target pose measurement data is weighted to obtain the pose deviation of the agile arm after weighting. Based on the weighted pose deviation, the expected joint angle data for the next cycle of the agile arm is planned, and the expected joint angle data is weighted to obtain the joint motion data of the agile arm. Based on the target kinematic transformation relationship, the joint motion data is converted into motor motion data, and the motor motion data is weighted according to the motor response characteristics to obtain weighted motor motion data, which is then used to control the agile arm to perform corresponding visual servo control actions.

2. The method according to claim 1, characterized in that, The weighted processing of the pose deviation of the target pose measurement data includes: During the initial motion phase of visual servoing, the pose deviation is weighted with the visual servoing time as the independent variable to determine the weighted pose deviation. When the pose deviation between the end-effector pose and the target pose is less than a preset distance, the pose deviation is used as an independent variable for weighted processing to determine the weighted pose deviation. The target pose deviation is weighted based on the actual needs of the agile arm to determine the weighted pose deviation.

3. The method according to claim 1, characterized in that, The step of planning the expected joint angle data for the next cycle of the agile arm based on the weighted pose deviation, and then weighting the expected joint angle data to obtain the joint motion data of the agile arm, includes: Based on the posture deviation, the desired joint angle, desired joint angular velocity, and desired joint angular acceleration of the agile arm in the next cycle are planned. Based on the joint response characteristics, the desired joint angle, the desired joint angular velocity, and the desired joint angular acceleration are weighted to obtain the joint motion data of the agile arm.

4. The method according to claim 3, characterized in that, The step of planning the desired joint angle, desired joint angular velocity, and desired joint angular acceleration of the agile arm in the next cycle based on the pose deviation includes: The joint angle increment of the agile arm in the end-effector coordinate system is calculated based on the pose deviation, and the joint angle increment is corrected based on the angular velocity threshold. The desired joint angle, the desired joint angular velocity, and the desired joint angular acceleration are calculated based on the current joint angle of the agile arm and the joint angle increment.

5. The method according to claim 1, characterized in that, The process of converting joint motion data into motor motion data based on the target kinematic transformation relationship, and then weighting the motor motion data according to the motor's response characteristics to obtain weighted motor motion data, includes: Based on the target kinematic transformation relationship, the motor angular velocity and motor angular acceleration in the motor motion data are calculated according to the joint motion data; The motor angular velocity and motor angular acceleration are weighted according to the motor's response characteristics to obtain weighted motor motion data.

6. A visual servo control device based on heterogeneous joint weighted motion decomposition, characterized in that, include: The first processing module is used to perform weighted processing on the pose deviation of the target pose measurement data based on the target pose measurement data of the agile arm, so as to obtain the weighted pose deviation of the agile arm. The second processing module is used to plan the expected joint angle data of the agile arm in the next cycle based on the weighted pose deviation, and to perform weighted processing on the expected joint angle data to obtain the joint motion data of the agile arm. The control module is used to convert the joint motion data into motor motion data based on the target kinematic transformation relationship, and to perform weighted processing on the motor motion data according to the response characteristics of the motor to obtain weighted motor motion data, so as to control the agile arm to perform corresponding visual servo control actions using the weighted motor motion data.

7. The apparatus according to claim 6, characterized in that, The first processing module includes: The first processing unit is used to perform weighted processing on the pose deviation with the visual servoing time as the independent variable during the initial motion period of visual servoing, so as to determine the pose deviation after weighted processing. The second processing unit is used to perform weighted processing with the pose deviation as the independent variable when the pose deviation between the end pose of the agile arm and the target pose is less than a preset distance, so as to determine the pose deviation after weighted processing. The third processing unit is used to perform weighted processing on the target pose deviation based on the actual needs of the agile arm, so as to determine the pose deviation after weighted processing.

8. An electronic device, characterized in that, include: The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the visual servo control method based on heterogeneous joint weighted motion decomposition as described in any one of claims 1-5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the visual servo control method based on heterogeneous joint weighted motion decomposition as described in any one of claims 1-5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed, it is used to implement the visual servo control method based on heterogeneous joint weighted motion decomposition as described in any one of claims 1-5.

Images