Visual servo control method and device based on heterogeneous joint segmented motion decomposition

Abstract

This application relates to the field of robotics, and particularly to a visual servo control method and device based on heterogeneous joint segmented motion decomposition. The method includes: processing target pose measurement data stepwise to obtain target pose deviation data; planning the desired joint angle motion endpoint of the agile arm in subsequent target cycles, and planning the joint motion data of the agile arm in subsequent target cycles during the online phase in an offline phase; converting the joint motion data into motor motion data, and segmenting the motor motion data within subsequent target cycles to control the agile arm to perform corresponding visual servo control actions using the segmented motor motion data. This application can subdivide and interpolate the planned motion data according to the control cycle of the underlying motor motion controller, achieving efficient and accurate visual servo control. Furthermore, it allows for the development of multi-threading for visual servo planning calculations, with offline planning performed in the main thread, offering high flexibility and practicality.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to a visual servo control method and device based on heterogeneous joint segmented motion decomposition. Background Technology

[0002] Among related technologies, the rope-driven agile arm, as an innovative rope-driven multi-degree-of-freedom robotic arm design, has shown great application potential in fields such as space exploration and complex environment operations due to its superior flexibility, low inertia characteristics, and the advantage of a rear-mounted drive system. Compared to traditional rigid robotic arms, the rope-driven agile arm has a more complex and varied structural design. Its transmission link from the drive end to the working end is significantly extended, encompassing multiple aspects such as motor drive, rope transmission, joint rotation, and the precise operation of the end effector. To achieve high-precision multi-degree-of-freedom motion and a compact overall structure, the design of each joint in the rope-driven agile arm varies significantly, making the kinematic transformation relationship between the motor, rope, and joints extremely complex and nonlinear.

[0003] However, the joint structures of rope-driven agile arms vary greatly in related technologies. The complex kinematic models and multi-layer transmission links increase the difficulty of accurately adjusting the closed-loop motion parameters of the robotic arm joints and motors. Since the response characteristics of each motor are not the same, the actual response of the joint may lag behind the command from the motor drive end. When faced with the unique motion characteristics of rope-driven agile arms, it is often difficult to achieve efficient and accurate visual servo control, which urgently needs to be solved. Summary of the Invention

[0004] This application provides a visual servo control method, device, electronic device, and storage medium based on heterogeneous joint segmented motion decomposition to solve the problems in related technologies, such as the large differences in the joint structures of rope-driven agile arms, the complex kinematic models and multi-layer transmission links that increase the difficulty of accurately adjusting the closed-loop motion parameters of the robot arm joints and motors, and the fact that the response characteristics of each motor are not the same, which may cause the actual response of the joint to lag behind the command of the motor drive end. When facing the unique motion characteristics of rope-driven agile arms, it is often difficult to achieve efficient and accurate visual servo control.

[0005] The first aspect of this application provides a visual servo control method based on heterogeneous joint segmented motion decomposition, comprising the following steps: based on the target pose measurement data of an agile arm, the target pose measurement data is processed stepwise to obtain target pose deviation data; based on the target pose deviation data, the expected joint angle motion endpoint of the agile arm in a subsequent target cycle is planned, and the joint motion data of the agile arm in the online phase is planned according to the expected joint angle motion endpoint in the offline phase; the joint motion data is converted into motor motion data, and the motor motion data is segmented in the subsequent target cycle to obtain segmented motor motion data, so as to use the segmented motor motion data to control the agile arm to perform corresponding visual servo control actions.

[0006] Optionally, in one embodiment of this application, the step-by-step processing of the target pose measurement data based on the agile arm to obtain target pose deviation data includes: calculating the end effector step length threshold of the agile arm based on the target pose measurement data; and performing step-by-step processing of the target pose measurement data based on the end effector step length threshold to obtain the target pose deviation data.

[0007] Optionally, in one embodiment of this application, planning the expected joint angle motion endpoint of the agile arm in a subsequent target cycle includes: acquiring the end-effector pose and current joint angle of the target pose deviation data; calculating the joint angle increment based on the end-effector pose and the current joint angle; planning the expected joint angle motion endpoint of the agile arm in a subsequent target cycle in a visual servo speed mode based on the joint angle increment and the current joint angle, or planning the expected joint angle motion endpoint of the agile arm in a subsequent target cycle in a visual servo position mode based on the end-effector pose and the current joint angle.

[0008] Optionally, in one embodiment of this application, the step of planning the expected joint angle motion endpoint of the agile arm in a subsequent target cycle based on the target pose deviation data, and planning the joint motion data of the agile arm in the subsequent target cycle in the online stage based on the expected joint angle motion endpoint in the offline stage, further includes: planning the joint motion data of the agile arm in the subsequent target cycle based on the joint response characteristics, joint motion characteristics and the joint angle increment of the agile arm.

[0009] Optionally, in one embodiment of this application, the step of planning the expected joint angle motion endpoint of the agile arm in a subsequent target cycle based on the target pose deviation data, and planning the joint motion data of the agile arm in the subsequent target cycle in the online phase based on the expected joint angle motion endpoint in the offline phase, includes: determining the actual needs of the agile arm based on the target pose deviation data; determining the planning method of the expected joint angle motion endpoint and the joint motion data in the subsequent target cycle based on the actual needs, so as to plan the expected joint angle motion endpoint and the joint motion data, wherein the planning method includes visual servo planning and offline planning.

[0010] A second aspect of this application provides a visual servo control device based on heterogeneous joint segmented motion decomposition, comprising: a processing module, configured to process the target pose measurement data of an agile arm in steps to obtain target pose deviation data based on the target pose measurement data of the agile arm; a planning module, configured to plan the expected joint angle motion endpoint of the agile arm in a subsequent target cycle based on the target pose deviation data, and to plan the joint motion data of the agile arm in the subsequent target cycle in the online phase according to the expected joint angle motion endpoint in the offline phase; and a control module, configured to convert the joint motion data into motor motion data, and to segment the motor motion data in the subsequent target cycle to obtain segmented motor motion data, so as to control the agile arm to perform corresponding visual servo control actions using the segmented motor motion data.

[0011] Optionally, in one embodiment of this application, the processing module includes: a first calculation unit, configured to calculate the end effector step length threshold of the agile arm based on the target pose measurement data; and a step-by-step unit, configured to perform step-by-step processing on the target pose measurement data based on the end effector step length threshold to obtain the target pose deviation data.

[0012] Optionally, in one embodiment of this application, the planning module includes: an acquisition unit, configured to acquire the end pose and current joint angle of the target pose deviation data; a second calculation unit, configured to calculate the joint angle increment based on the end pose and the current joint angle; and a first planning unit, configured to plan the expected joint angle motion endpoint of the agile arm in a subsequent target cycle in a visual servo speed mode based on the joint angle increment and the current joint angle, or to plan the expected joint angle motion endpoint of the agile arm in a subsequent target cycle in a visual servo position mode based on the end pose and the current joint angle.

[0013] Optionally, in one embodiment of this application, the planning module further includes: a second planning unit, configured to plan the joint motion data of the agile arm in a subsequent target cycle based on the joint response characteristics, joint motion characteristics and the joint angle increment of the agile arm.

[0014] Optionally, in one embodiment of this application, the planning module includes: a determining unit, configured to determine the actual requirements of the agile arm based on the target pose deviation data; and a third planning unit, configured to determine the planning method of the expected joint angle motion endpoint and the joint motion data in a subsequent target period based on the actual requirements, so as to plan the expected joint angle motion endpoint and the joint motion data, wherein the planning method includes visual servo planning and offline planning.

[0015] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the program to implement the visual servo control method based on heterogeneous joint segmented motion decomposition as described in the above embodiments.

[0016] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described visual servo control method based on heterogeneous joint segmented motion decomposition.

[0017] A fifth aspect of this application provides a computer program product, including a computer program that, when executed, is used to implement the above-described visual servo control method based on heterogeneous joint segmented motion decomposition.

[0018] This application combines visual servo planning with offline planning, segmenting the joint motion process of the agile arm and then subdividing and interpolating according to the control cycle of the underlying motor motion controller, thereby controlling the agile arm's movement through the motor. This achieves the correction of the robotic arm's motion through visual servo planning, ensuring the end effector moves towards the target. Offline planning of several cycles of the robotic arm's motion reduces the deviation from the desired configuration. Subdividing and interpolating the planned motor angles according to the control cycle of the underlying motor motion controller achieves efficient and precise visual servo control. Furthermore, for processors with limited processing power and long visual servo time, multi-threading can be developed for visual servo planning calculations, with offline planning performed in the main thread, offering high flexibility and practicality. This solves the problems in related technologies where the joint structures of rope-driven agile arms vary greatly, complex kinematic models and multi-layered transmission links increase the difficulty of precisely adjusting the closed-loop motion parameters of the robotic arm joints and motors. Due to the different response characteristics of each motor, the actual response of the joints may lag behind the commands from the motor drive end, making it difficult to achieve efficient and precise visual servo control when facing the unique motion characteristics of rope-driven agile arms.

[0019] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0020] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0021] Figure 1 This is a schematic diagram of a rope-driven agile arm according to an embodiment of this application;

[0022] Figure 2 This is a flowchart of a visual servo control method based on heterogeneous joint segmented motion decomposition provided in an embodiment of this application;

[0023] Figure 3 This is a flowchart of a visual servo control method based on heterogeneous joint segmented motion decomposition according to an embodiment of this application;

[0024] Figure 4 This is a schematic diagram of the structure of a visual servo control device based on heterogeneous joint segmented motion decomposition according to an embodiment of this application;

[0025] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application.

[0026] Figure label:

[0027] 10-Visual servo control device based on heterogeneous joint segmented motion decomposition: 100-processing module, 200-planning module and 300-control module; 501-memory, 502-processor and 503-communication interface. Detailed Implementation

[0028] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0029] The following describes a visual servo control method and apparatus based on heterogeneous joint segmented motion decomposition according to embodiments of this application, with reference to the accompanying drawings. In the related technologies mentioned in the background section, the joint structures of rope-driven agile arms vary greatly. Complex kinematic models and multi-layered transmission links increase the difficulty of accurately adjusting the closed-loop motion parameters of the robotic arm joints and motors. Due to the different response characteristics of each motor, the actual response of the joint may lag behind the command from the motor drive end. When facing the unique motion characteristics of rope-driven agile arms, it is often difficult to achieve efficient and accurate visual servo control. This application provides a visual servo control method based on heterogeneous joint segmented motion decomposition. In this method, visual servo planning can be combined with offline planning to segment the joint motion process of the agile arm, and then subdivided and interpolated according to the control cycle of the underlying motor motion controller, thereby controlling the agile arm's movement through the motor. This technology enables the correction of robotic arm motion through visual servo planning, guiding the end effector towards the target. By planning the robotic arm's motion over several cycles offline, the desired configuration deviation is minimized. The planned motor angles are subdivided and interpolated according to the control cycle of the underlying motor motion controller, achieving efficient and precise visual servo control. Furthermore, for processors with limited processing power and long visual servo processing times, multi-threading can be developed for visual servo planning calculations, with offline planning performed on the main thread, offering high flexibility and practicality. This addresses the challenges of highly variable joint structures in rope-driven agile arms, complex kinematic models, and multi-layered transmission links that increase the difficulty of precisely adjusting the closed-loop motion parameters of the joints and motors. The varying response characteristics of different motors can cause the actual joint response to lag behind the motor drive commands, making efficient and precise visual servo control difficult when dealing with the unique motion characteristics of rope-driven agile arms.

[0030] Before explaining the visual servo control method based on heterogeneous joint segmented motion decomposition provided in the embodiments of this application, the structure of the rope-driven agile arm involved in the embodiments of this application will be described first.

[0031] Figure 1 This is a schematic diagram of a rope-driven agile arm according to one embodiment of this application. Figure 1 As shown, the controlled object in this embodiment is a rope-driven agile arm. The robotic arm has seven joints, including two shoulder joints. These two joints are driven by motors connected via reducers. These two joints are also connected to elbow and wrist joint drive boxes, which contain five drive motors for the elbow and wrist joints. The motors rotate to pull the connected drive ropes, causing each joint to rotate. The robotic arm includes two elbow joints and three wrist joints. Wrist joints one and two are universal joints, driven by two motors. Wrist joint three is a roll joint. Each joint has a joint encoder to measure the joint's rotation angle, and a resolver at the motor end to measure the motor's rotation angle. The robotic arm's closed-loop control method is a joint-level position closed loop + a motor-level position closed loop.

[0032] In a typical vision servo control framework, the motion increments of each joint angle of the robotic arm can be defined as: [dQ1, dQ2, dQ3, dQ4, dQ5, dQ6, dQ7], corresponding to the angle adjustments of the seven joints. However, due to the motion hysteresis effect of the joint motors and transmission system, as well as the different response characteristics between the joints, the joints do not actually achieve the expected increments after one control cycle. Specifically, the actual joint angle increment dQ′ deviates from the planned dQ, such as dQ′ = [0.9dQ1, 0.9dQ2, 0.5dQ3, 0.5dQ4, 0.3dQ5, 0.2dQ6, 0.2dQ7], showing varying degrees of reduction.

[0033] In subsequent control cycles, the system recalculates and plans new joint angle increments based on the target pose measurement values ​​obtained through the hand-eye camera (note that these measurements may lag behind the actual target position, as the camera measurement cycle is often longer than the motion planning cycle) and the current joint angle state of the robotic arm. However, due to the consistently lower actual motion of some joints compared to the planned values, and the existence of motion lag and mismatch issues, the overall motion speed of the robotic arm becomes sluggish, and there is a significant deviation between its actual motion trajectory and the pre-planned path. This accumulated deviation may cause the end effector of the robotic arm to fail to accurately align with or maintain the target object in the expected position within the camera's field of view, thereby affecting the accuracy and stability of the vision servo system. In severe cases, it may cause the target to completely escape the camera's field of view, increasing the complexity and uncertainty of the operation.

[0034] Specifically, Figure 2 This is a flowchart of a visual servo control method based on heterogeneous joint segmented motion decomposition provided in an embodiment of this application.

[0035] like Figure 2 As shown, the visual servo control method based on heterogeneous joint segmented motion decomposition includes the following steps:

[0036] In step S201, based on the target pose measurement data of the agile arm, the target pose measurement data is processed step by step to obtain the target pose deviation data.

[0037] It is understandable that "agile arm" here can be interpreted as a rope-driven agile arm. As a new type of rope-driven multi-degree-of-freedom robotic arm, the rope-driven agile arm has advantages such as flexible movement, low inertia, and rear-mounted actuators, making it a promising candidate for applications in space operations and complex environments. Visual servoing is a crucial method for online autonomous operation of robotic arms. The end effector of the robotic arm is equipped with a hand-eye camera, which can observe the surrounding environment and calculate the target pose in real time. Based on the target pose information, the robotic arm plans and controls its movement to the desired position.

[0038] Target pose measurement data can be understood here as the specific position and orientation that the robotic arm's end effector needs to achieve in three-dimensional space. This pose can be described by position and orientation: position can be understood as the specific coordinates of the robotic arm's end effector in space, usually represented by {x, y, z} in a three-dimensional coordinate system. These coordinate values ​​define the precise position of the end effector relative to a reference coordinate system (such as the world coordinate system or the robotic arm's base coordinate system); orientation refers to the orientation or direction of the robotic arm's end effector in space, describing the rotational state of the end effector relative to a reference coordinate system. Orientation can be described in various ways, such as rotation matrices, Euler angles (i.e., rotation angles about the x-axis, y-axis, and z-axis), etc.

[0039] In some embodiments, when controlling the rope-driven agile arm, target pose measurement data can be input first, and then the input target pose measurement data can be processed step by step to obtain the target pose deviation data of the agile arm.

[0040] For example, based on the input target pose measurement data, the complex adjustment process during the motion control of the agile arm is broken down into a series of orderly and manageable steps, thereby obtaining the deviation data between the agile arm and the target pose measurement data in each step. This process may involve the precise measurement and calculation of the position and orientation of the robotic arm's end effector in three-dimensional space to ensure that the robotic arm can accurately reach and maintain the specified position and orientation, i.e., the target pose.

[0041] The embodiments of this application can divide the pose deviation of the agile arm into steps, and divide the process of the agile arm reaching the target pose into a series of steps, which helps to improve the operational accuracy of the visual servo control of the agile arm, and facilitates real-time adjustments based on the actual situation during the control process.

[0042] The following section will further elaborate on the step-by-step processing of the pose deviation in the target pose measurement data.

[0043] Optionally, in one embodiment of this application, the target pose measurement data of the agile arm is processed stepwise to obtain target pose deviation data, including: calculating the end effector step length threshold of the agile arm based on the target pose measurement data; and processing the target pose measurement data stepwise based on the end effector step length threshold to obtain target pose deviation data.

[0044] In actual execution, when processing the target pose measurement data in steps, we can first determine that the agile arm is within the visual servoing planning cycle, and then determine the offline planning cycle based on the input target pose relative to the end effector of the robotic arm. For example, when the target is far from the end effector, the offline planning cycle can be increased, allowing the robotic arm (agile arm) to quickly approach the target; when the target is close to the end effector, the offline planning cycle can be reduced to minimize errors, so that the robotic arm can frequently correct its movement based on visual measurement information.

[0045] Next, the end-effector step size threshold can be calculated based on the set number of offline planning cycles. If the algorithm pre-stores data for the end-effector position step size and attitude step size as one cycle, the end-effector step size threshold used by the algorithm later can be the product of the pre-stored data and the number of offline planning cycles.

[0046] Finally, the magnitudes of the position deviation vector and the attitude deviation vector of the target relative to the end effector of the agile arm are calculated respectively. Then, based on the ratio of the calculated attitude deviation to the threshold of the movement step length of the agile arm end effector, the attitude deviation is reduced to the threshold range to obtain the required target attitude deviation data.

[0047] For example, the input data is the pose deviation of the target relative to the end effector of the agile arm [P]. x P y P z R x R y R z ] Calculate the magnitude ||P|| of the position deviation vector and the magnitude ||R|| of the attitude deviation vector of the target relative to the end of the agile arm, respectively. The calculation formula can be expressed as follows:

[0048]

[0049] Next, the ratio of pose deviation to the end-effector step size threshold for a single cycle of the agile arm can be calculated to reduce the pose deviation to within the threshold range. Specifically, the end-effector position step size can be set as P0, and the end-effector attitude step size as R0. The position step size ratio can be expressed as: kA =||P|| / P0, the attitude step size ratio can be expressed as: k R =||R|| / R0. When k A When the value is greater than 1, the pose deviation can be expressed as:

[0050] [P x P y P z ] = [P x P y P z ] / k A ,

[0051] [R x R y R z ] = [R x R y R z ] / k R .

[0052] This application embodiment can ensure that the robotic arm adopts a more refined motion control strategy when approaching the target pose by calculating the end-effector step length threshold, reducing pose deviation caused by excessive step length. Step-by-step measurement of target pose deviation data helps to promptly detect and correct minor errors in the motion process, and allows for real-time adjustment and optimization of the robotic arm's motion planning, providing improvement directions for subsequent motion planning, thereby improving overall motion accuracy.

[0053] Step S202: Based on the target pose deviation data, plan the expected joint angle motion endpoint of the agile arm in the subsequent target cycle, and plan the joint motion data of the agile arm in the subsequent target cycle in the online phase according to the expected joint angle motion endpoint in the offline phase.

[0054] As one possible approach, after obtaining the processed target pose deviation data, this embodiment of the application can initiate visual servo planning based on the pose deviation data to plan the expected joint angle motion endpoints of the agile arm for the next few cycles.

[0055] Furthermore, based on the planned expected joint angle motion endpoint, this embodiment of the application may, but is not limited to, use interpolation to plan the expected joint angle, joint angular velocity, and joint angular acceleration for each cycle during the motion process in the offline stage, and use these data as joint motion data for subsequent control of the agile arm.

[0056] The embodiments of this application can separate the planning of the expected joint angle motion endpoint and the planning of joint motion data of the agile arm, which is equivalent to performing a segmented processing. This can effectively increase the accuracy of the data planned in each process, facilitate real-time modification, and improve planning and correction efficiency.

[0057] The process will now be explained in further detail.

[0058] Optionally, in one embodiment of this application, based on the target pose deviation data, the expected joint angle motion endpoint of the agile arm in the subsequent target cycle is planned, and the joint motion data of the agile arm in the subsequent target cycle in the online stage is planned according to the expected joint angle motion endpoint in the offline stage, including: determining the actual needs of the agile arm based on the target pose deviation data; determining the planning method of the expected joint angle motion endpoint and joint motion data in the subsequent target cycle according to the actual needs, so as to plan the expected joint angle motion endpoint and joint motion data, wherein the planning method includes visual servo planning and offline planning.

[0059] Based on the descriptions of other embodiments, it is understood that when processing the target pose measurement data in steps, it is necessary to first determine whether the current movement of the agile arm is within the visual servo planning cycle. If it is within the visual servo planning cycle, then the length of the offline planning cycle is determined.

[0060] In some embodiments, the visual servoing planning period and the offline planning period of this application can be configured according to actual needs. Actual needs here can be understood as the motion requirements of the agile arm determined based on the target pose measurement data, or other requirements. Furthermore, the planning method for the expected joint angle motion endpoint and joint motion data within subsequent target periods can be determined based on these actual needs, such as visual servoing planning and offline planning.

[0061] Visual servoing planning, also known as online planning, can plan the desired joint angles for the next cycle or several cycles in real time based on the target pose and the current joint angle state. Offline planning, on the other hand, can be understood as planning the total time and interpolating the time given the initial and final joint angles, and then interpolating the joint angles according to the planning cycle.

[0062] For example, in processes where high-speed movement is possible, the number of offline planning cycles can be increased, as the movement speed is fast; in processes where deceleration or low-speed movement is required, the number of offline planning cycles can be reduced, so that visual measurement information can be used more frequently to correct the planning values.

[0063] Additionally, embodiments of this application may also allocate the number of offline planning cycles based on the distance between the robotic arm's end effector and the target. That is, when the target is far away, the number of offline planning cycles is greater, and more cycles of motion are planned using visual servoing; when the target is close, the number of offline planning cycles is less, and visual servoing is frequently used to correct the joint angle planning values.

[0064] For processors with poor processing power, visual servoing takes a long time and may exceed the control cycle. Multi-threading can be developed for visual servoing planning calculations, with offline planning performed on the main thread.

[0065] This application embodiment can comprehensively employ visual servo planning and offline planning when planning the expected joint angle endpoints and joint motion data in subsequent cycles of the agile arm. By combining visual servo planning with offline planning, the expected joint angle motion endpoints for the next few motion cycles are planned using the online visual servo planning method, and the joint angles planned by the visual servo planning are interpolated using the offline planning method. This approach combines the advantages of online visual servo planning, which can correct the motion trajectory in real time, with the smooth motion and fast planning speed of offline planning, which helps to make real-time adjustments based on the actual situation during the movement of the agile arm.

[0066] Optionally, in one embodiment of this application, planning the desired joint angle motion endpoint of the agile arm in a subsequent target cycle includes: acquiring the end-effector pose and current joint angle of the target pose deviation data; calculating the joint angle increment based on the end-effector pose and the current joint angle; planning the desired joint angle motion endpoint of the agile arm in a subsequent target cycle in a visual servo speed mode based on the joint angle increment and the current joint angle, or planning the desired joint angle motion endpoint of the agile arm in a subsequent target cycle in a visual servo position mode based on the end-effector pose and the current joint angle.

[0067] In some embodiments, when planning the expected joint angle motion endpoint of the agile arm in the subsequent target cycle based on the target pose deviation data, it is necessary to first obtain the end pose and current joint angle of the target pose deviation data, calculate the joint angle increment of the agile arm, and then select different visual servo planning methods to plan the expected joint angle motion endpoint of the agile arm in the subsequent target cycle according to actual needs.

[0068] The visual servoing planning in this application includes, but is not limited to, two methods: visual servoing speed planning and visual servoing position planning. Visual servoing speed planning calculates the joint angle increment based on the inverse of the Jacobian matrix between the end-effector pose and the joint angle in the robot arm's end-effector coordinate system, and then superimposes it on the current joint angle to obtain the desired joint angle; visual servoing position planning calculates the desired joint angle based on the robot arm's target end-effector pose and the current joint angle.

[0069] Taking the visual servoing speed mode as an example to plan the expected joint angle motion endpoint of the agile arm in the subsequent target cycle, the planning process can be represented as follows:

[0070] (1) Calculate the Jacobian matrix J between the joint angles and the end-effector pose of the agile arm in the end-effector coordinate system. n , can be represented as follows:

[0071]

[0072] in, n R0 is the rotation transformation matrix of the robot arm's base coordinate system relative to the end effector coordinate system, and J0 is the Jacobian matrix between the robot arm's joint angles and the end effector pose in the base coordinate system.

[0073] (2) Calculate the end-effector pose increment:

[0074] The weighted pose deviation [P] x P y P z R x R y R z Transform into a homogeneous transformation matrix n T t Let the current end pose be n T n This can be represented as follows:

[0075]

[0076] The end-effector pose increment dA can be expressed as:

[0077]

[0078] dr = 0.5(n0×n + o0×o + a0×a)

[0079] Where, n0 = [n x0 n y0 n z0 ] T , o0 = [o x0 o y0 n z0 ] T , a0 = [a x0 a y0 a z0 ] T , are the cosines of the angles between the x, y, and z axes of the target coordinate system and the axes of the robotic arm's end effector coordinate system, respectively, n = [n x n y n z ] T , o = [o x o y n z ] T , a = [a x a y a z ] T , , are the cosines of the angles between the x, y, and z axes of the end-effector coordinate system and the axes of the end-effector coordinate system of the robotic arm, respectively.

[0080] (3) Calculate the inverse of the Jacobian matrix:

[0081] This application embodiment can use the Jacobian pseudo-inverse method to calculate the Jacobian matrix J between the joint angles and the end effector pose of the robotic arm. n The inverse matrix of the Jacobian matrix is ​​obtained, and the singularity avoidance method of DLS (Damped Least Squares) is used to prevent matrix singularity. The formula for solving the inverse matrix of the Jacobian matrix can be expressed as follows:

[0082] J n + =(J n T J n +λ 2 I) -1 J n T

[0083] Among them, J n + J is the inverse of the Jacobian matrix. n T Let I be the transpose of the Jacobian matrix, I be the identity matrix, and λ be the DLS singularity parameter.

[0084] (4) Calculate the joint angle increment of the agile arm. The formula can be expressed as follows:

[0085] dq = J n + dA;

[0086] Where dq is the joint angle increment of the agile arm, and dA is the end-effector pose deviation (increment).

[0087] (5) Correct the joint angle increment based on the joint angular velocity threshold:

[0088] If some joint angle increments are greater than the joint angular velocity threshold, the ratio of each joint angle increment to its respective velocity threshold can be calculated. The largest ratio is selected, and the overall joint angle increments are reduced by that factor, thereby correcting the unsuitable joint angle increments.

[0089] Taking the planning of the agile arm's expected joint angle motion endpoint in a subsequent target cycle using visual servoing position mode as an example, the planning process can be represented as follows:

[0090] (1) Calculate the current end-effector pose of the robotic arm based on the current joint angles, and calculate the end-effector pose using the kinematic model from the joints to the end effector. 0 T n The end-effector pose increment is transformed into a homogeneous transformation matrix. n T t Then the desired end-effector pose of the robotic arm 0 Tt = 0 T n * n T t ;

[0091] (2) Based on the current joint angles of the robotic arm and the desired end-effector pose 0 T t The desired joint angle of the robotic arm is calculated using a kinematic model of the end effector to the joint.

[0092] (3) Calculate the joint angle increment by using the current joint angle and the expected joint angle. Correct the joint angle increment according to the joint angular velocity threshold. If there is a joint angle increment greater than the joint angular velocity threshold, calculate the ratio of each joint angle increment to its respective velocity threshold, select the largest ratio, and reduce the overall joint angle increment by that factor to correct unsuitable joint angle increments.

[0093] The embodiments of this application can select different visual servo planning methods based on the actual situation and actual needs according to the joint angle increment of the agile arm to plan the expected joint angle motion endpoint of the agile arm in the subsequent target cycle, so as to realize the joint angle planning value by frequently using visual servo planning, which helps the application of this application in a variety of practical scenarios.

[0094] Optionally, in one embodiment of this application, based on the target pose deviation data, the expected joint angle motion endpoint of the agile arm in the subsequent target cycle is planned, and the joint motion data of the agile arm in the subsequent target cycle in the online stage is planned according to the expected joint angle motion endpoint in the offline stage. The method further includes: planning the joint motion data of the agile arm in the subsequent target cycle according to the joint response characteristics, joint motion characteristics and joint angle increment of the agile arm.

[0095] Based on the descriptions of other embodiments, it is understood that after planning the expected joint angle endpoints for the agile arm's subsequent target cycles, the joint motion data for the agile arm's subsequent target cycles in the online phase can be planned based on the expected joint angle endpoints during the offline phase. The joint motion data includes, but is not limited to, expected joint angles, expected joint angular velocities, and expected joint angular accelerations.

[0096] Furthermore, when planning these joint motion data, the joint response characteristics, joint motion characteristics, and joint angle increments of the agile arm can be taken into account to reduce the delays and hysteresis reactions that these factors may cause during the movement of the agile arm.

[0097] For example, based on the planned expected joint angle motion endpoint, the expected joint angle, joint angular velocity, and joint angular acceleration for each cycle are interpolated. Different interpolation methods can be used for joints with different response and motion characteristics. The process can be represented as follows:

[0098] For joint PID closed-loop control based on kinematic feedforward, the joint angle output by the control loop can be expressed as:

[0099] q = q0 + dq + k Pq dq+k Iq fdq+k Dq ddq

[0100] Where q0 is the current joint angle, dq is the joint angle increment calculated using pose deviation, q0+dq is the planned joint angle, and k Pq dq is the proportional term of the joint closed-loop control, k Pq k is the joint angle proportional control coefficient. Iq fdq is the integral term of the joint closed-loop control, k Iq Here, fdq is the integral control coefficient for the joint angle, and k is the cumulative amount of the joint angle increment. Dq ddq is the differential term of the joint closed-loop control, k Dq Here, is the differential control coefficient for the joint angle, and ddq is the difference between the joint angle increment in the current cycle and the joint angle increment in the previous cycle. The larger the joint angle output by the control loop, the larger the motor angle increment translated to the motor end through the kinematic model. The increased increment will improve the motor's movement speed and output torque, and accelerate its movement to the desired position.

[0101] For the shoulder joint, the joint rotation is directly driven by the motor without being transmitted through the rope. The joint response characteristics are relatively fast. The planned shoulder joint angle increment can be linearly interpolated according to the number of execution cycles. The planned value is increased by a small step size in each cycle. Even if the joint tracking effect is average, the joint will increase its movement speed in subsequent cycles due to the increase in deviation dq, thereby improving the following performance.

[0102] For the elbow joint, which is driven by a motor via a drive cable, the transmission of the cable causes lag in the movement. It is necessary to further improve the dq during the movement process. For example, half of the planned joint angle increment is sent in the first few offline planning cycles, and the full joint angle increment is sent in the later offline planning cycles. Then, the increment of the elbow joint at the beginning of the movement phase is actually the increment that needs to be reached in several cycles. The elbow joint will quickly approach the joint angle. The same applies to the later offline planning cycles.

[0103] For the wrist joint, wrist 1, wrist 2, and wrist 3 are driven by a motor via a drive rope. The transmission of the rope causes motion lag because the drive rope needs to pass through the shoulder and elbow to reach the wrist. Its transmission chain is relatively long, and the rope lag phenomenon is more obvious. The planned wrist joint angle increment can be sent throughout the entire planning cycle. A larger dq can improve the joint's response characteristics. The joint can also be moved into position using several cycles of offline planning.

[0104] Next, the desired joint angular velocity and joint angular acceleration are calculated. The joint angular velocity is obtained by dividing the increment of the joint angle by the period. The joint angular velocity is obtained by subtracting the joint angular velocity from the joint angular velocity of the previous period. For some control systems, the joint angular velocity or joint angular acceleration is used as the control input.

[0105] Finally, it is determined whether the offline planning cycle count has reached the preset number of cycles. If it has, the visual servo planning flag is set to 1, and visual servo planning will be performed in the next cycle. Alternatively, when performing visual servo planning in the next cycle, it can be determined whether the visual servo planning flag is 1 first. If the visual servo planning flag is 1, then visual servo planning will be performed in the next cycle.

[0106] In planning joint motion data, this application comprehensively considers the joint response and various characteristics of the agile arm. By taking into account the hysteresis and nonlinearity of the joint response, pre-compensation can be performed during the planning stage, thereby reducing deviations and jitters in actual motion and improving motion accuracy. Furthermore, it helps the planning algorithm better adapt to these dynamic changes, planning motion trajectories that meet operational requirements and conform to physical laws, maintaining motion stability.

[0107] Step S203: The joint motion data is converted into motor motion data, and the motor motion data is segmented in the subsequent target cycle to obtain segmented motor motion data, so as to use the segmented motor motion data to control the agile arm to perform corresponding visual servo control actions.

[0108] It is understood that joint motion data includes, but is not limited to, desired joint angle, desired joint angular velocity, and desired joint angular acceleration; correspondingly, motor motion data includes, but is not limited to, desired motor angle, desired motor angular velocity, and desired motor angular acceleration.

[0109] In other embodiments, after obtaining joint motion data, the joint motion data can be converted into motor motion data through a certain conversion relationship. Taking the rope-driven agile arm as an example, the working principle is to drive the rope movement through the motor to realize various actions of the robotic arm. Its joint movement is realized by a drive mechanism composed of ropes and pulleys, while the position and posture of the robotic arm are controlled by the length and angle of the rope.

[0110] For example, by utilizing the kinematic transformation relationship between the agile arm joints and the motor, the desired joint angle, desired joint angular velocity, and desired joint angular acceleration can be respectively converted into the desired motor angle, desired motor angular velocity, and desired motor angular acceleration. It should be noted that the transformation and calculation processes, as well as the principles and tools used, can be selected by those skilled in the art based on the actual situation; this is merely an illustrative example and no specific limitations are imposed.

[0111] Next, in this embodiment of the application, the motor motion data can be subdivided and interpolated into shorter cycle motor motion data for use by the underlying motor motion controller to control the motor motion. For example, for a motor with a slow response speed, the subdivided incremental intervals can be sent to increase the motion step size and thus increase the response speed. For example, the final motor position to be reached can be sent in several cycles instead of incrementing the motor planning value in each cycle.

[0112] The embodiments of this application can interpolate motor motion data into data within a shorter period, thereby acquiring more motion data points in a shorter time period. This allows for more precise control of the motor's motion trajectory and speed, reducing errors caused by insufficient data points. The data is then output to the underlying motion controller, resulting in smoother motor motion and improved stability and accuracy.

[0113] The present application will be described in detail below with reference to a specific embodiment.

[0114] like Figure 3 The diagram shown is a flowchart of a visual servo control method based on heterogeneous joint segmented motion decomposition for a rope-driven agile arm according to an embodiment of this application.

[0115] Step S301, Begin.

[0116] Step S302: Input the current joint angle and the pose of the target relative to the end of the agile arm.

[0117] Step S303: Determine whether the current visual servo planning flag is 1. If it is not 1, increment the offline planning cycle count by 1.

[0118] Step S304: When the current visual servo planning flag is 1, input pose measurement data preprocessing includes: determining the number of offline planning cycles based on the target pose, calculating the end-effector step length threshold based on the number of offline planning cycles, calculating the magnitude of the target position vector and the magnitude of the target pose vector; calculating the ratio of the magnitude to the end-effector step length threshold, and reducing the pose deviation to the threshold range.

[0119] Step S305, initiate visual servo planning, including: resetting the offline planning cycle count to zero, and selecting a visual servo planning mode according to specific circumstances and actual needs. If the visual servo speed mode is selected, calculate the Jacobian matrix between the robot arm end-effector pose and joint angles, and the pose difference dA between the target pose and the robot arm end-effector. Then, calculate the inverse of the Jacobian matrix and use the pseudo-inverse method to calculate the inverse of the Jacobian matrix. Finally, calculate the joint angle increment and correct the joint angle increment according to the joint angular velocity threshold. If the visual servo position mode is selected, calculate the current end-effector pose and the desired end-effector pose based on the current joint angle, then calculate the desired joint angle based on the current joint angle and the desired end-effector pose. After calculating the joint angle increment, the joint angle increment is also corrected according to the joint angular velocity threshold.

[0120] Step S306: Increment the offline planning cycle count by 1.

[0121] Step S307 involves planning offline joint motion data, including: setting the visual servo planning flag to 0; performing offline planning for different joints using different interpolation methods; calculating the expected joint angular velocity and joint angular acceleration; determining whether the offline planning period has reached a preset value; and setting the visual servo planning flag to 1 if the offline planning period has reached the preset value. Alternatively, if the offline planning period has not reached the preset value, the process can directly jump to the step of converting the joint motion data into motor motion data.

[0122] Step S308, post-processing of motor angle control data, includes: converting joint motion data into motor motion data, that is, converting planned joint angle, joint angular velocity, and angular acceleration into motor angle, motor angular velocity, and motor angular acceleration, and then interpolating these motor motion data into more granular periodic data according to the cycle of the underlying motor controller, so that the motion process is smoother.

[0123] Step S309: Determine whether the target has reached the capture area. If it has not reached the capture area, restart.

[0124] Step S310: After the target arrives at the capture area, control the robotic arm to move to the capture position.

[0125] Step S311, End.

[0126] The visual servo control method based on heterogeneous joint segmented motion decomposition proposed in this application combines visual servo planning with offline planning. The joint motion process of the agile arm is segmented, and then subdivided and interpolated according to the control cycle of the underlying motor motion controller, thereby controlling the agile arm's movement via motors. This achieves the correction of the robotic arm's motion through visual servo planning, enabling the end effector to move towards the target. Offline planning of the robotic arm's motion over several cycles reduces the desired configuration deviation. Subdividing and interpolating the planned motor angles according to the control cycle of the underlying motor motion controller achieves efficient and precise visual servo control. Furthermore, for processors with limited processing power and long visual servo time, multi-threading can be developed for visual servo planning calculations, with offline planning performed on the main thread, offering high flexibility and practicality. This solves the problems in related technologies, such as the large differences in the joint structures of rope-driven agile arms, the complex kinematic models and multi-layer transmission links that increase the difficulty of accurately adjusting the closed-loop motion parameters of the robotic arm joints and motors, and the fact that the response characteristics of each motor are not the same, which may cause the actual response of the joint to lag behind the command of the motor drive end. When facing the unique motion characteristics of rope-driven agile arms, it is often difficult to achieve efficient and accurate visual servo control.

[0127] Next, referring to the accompanying drawings, a visual servo control device based on heterogeneous joint segmented motion decomposition proposed according to an embodiment of this application is described.

[0128] Figure 4 This is a schematic diagram of the structure of the visual servo control device based on heterogeneous joint segmented motion decomposition according to an embodiment of this application.

[0129] like Figure 4 As shown, the visual servo control device 10 based on heterogeneous joint segmented motion decomposition includes: a processing module 100, a planning module 200, and a control module 300.

[0130] The processing module 100 is used to process the target pose measurement data step by step based on the target pose measurement data of the agile arm to obtain the target pose deviation data.

[0131] The planning module 200 is used to plan the expected joint angle motion endpoint of the agile arm in the subsequent target cycle based on the target pose deviation data, and to plan the joint motion data of the agile arm in the subsequent target cycle in the online phase based on the expected joint angle motion endpoint in the offline phase.

[0132] The control module 300 is used to convert joint motion data into motor motion data, and to segment the motor motion data in subsequent target cycles to obtain segmented motor motion data, so as to use the segmented motor motion data to control the agile arm to perform corresponding visual servo control actions.

[0133] Optionally, in one embodiment of this application, the processing module 100 includes: a first calculation unit and a step-by-step unit.

[0134] The first calculation unit is used to calculate the end-effector step length threshold of the agile arm based on the target pose measurement data.

[0135] The step-by-step unit is used to process the target pose measurement data step by step based on the end-effector step length threshold to obtain the target pose deviation data.

[0136] Optionally, in one embodiment of this application, the planning module 200 includes: an acquisition unit, a second calculation unit, and a first planning unit.

[0137] The acquisition unit is used to acquire the end pose and current joint angle of the target pose deviation data.

[0138] The second calculation unit is used to calculate the joint angle increment based on the end pose and the current joint angle.

[0139] The first planning unit is used to plan the desired joint angle motion endpoint of the agile arm in a subsequent target cycle in a visual servo speed mode based on the joint angle increment and the current joint angle, or to plan the desired joint angle motion endpoint of the agile arm in a visual servo position mode based on the end pose and the current joint angle.

[0140] Optionally, in one embodiment of this application, the planning module 200 further includes a second planning unit.

[0141] The second planning unit is used to plan the joint motion data of the agile arm in subsequent target cycles based on the joint response characteristics, joint motion characteristics and joint angle increments of the agile arm.

[0142] Optionally, in one embodiment of this application, the planning module 200 includes: a determination unit and a third planning unit.

[0143] The determining unit is used to determine the actual requirements of the agile arm based on the target pose deviation data.

[0144] The third planning unit is used to determine the planning method for the expected joint angle motion endpoint and joint motion data in the subsequent target period based on actual needs, so as to plan the expected joint angle motion endpoint and joint motion data. The planning method includes visual servo planning and offline planning.

[0145] It should be noted that the foregoing explanation of the visual servo control method based on heterogeneous joint segmented motion decomposition also applies to the visual servo control device based on heterogeneous joint segmented motion decomposition in this embodiment, and will not be repeated here.

[0146] The visual servo control device based on heterogeneous joint segmented motion decomposition proposed in this application combines visual servo planning with offline planning. It segments the joint motion process of the agile arm and then performs subdivision interpolation according to the control cycle of the underlying motor motion controller, thereby controlling the agile arm's movement via motors. This achieves the correction of the robotic arm's motion through visual servo planning, enabling the end effector to move towards the target. Offline planning of the robotic arm's motion over several cycles reduces the desired configuration deviation. The planned motor angles are subdivided and interpolated according to the control cycle of the underlying motor motion controller, achieving efficient and precise visual servo control. Furthermore, for processors with limited processing power and long visual servo time, multi-threading can be developed for visual servo planning calculations, with offline planning performed in the main thread, offering high flexibility and practicality. This solves the problems in related technologies, such as the large differences in the joint structures of rope-driven agile arms, the complex kinematic models and multi-layer transmission links that increase the difficulty of accurately adjusting the closed-loop motion parameters of the robotic arm joints and motors, and the fact that the response characteristics of each motor are not the same, which may cause the actual response of the joint to lag behind the command of the motor drive end. When facing the unique motion characteristics of rope-driven agile arms, it is often difficult to achieve efficient and accurate visual servo control.

[0147] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0148] The memory 501, the processor 502, and the computer program stored on the memory 501 and capable of running on the processor 502.

[0149] When the processor 502 executes the program, it implements the visual servo control method based on heterogeneous joint segmented motion decomposition provided in the above embodiments.

[0150] Furthermore, electronic devices also include:

[0151] Communication interface 503 is used for communication between memory 501 and processor 502.

[0152] The memory 501 is used to store computer programs that can run on the processor 502.

[0153] The memory 501 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0154] If the memory 501, processor 502, and communication interface 503 are implemented independently, then the communication interface 503, memory 501, and processor 502 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0155] Optionally, in a specific implementation, if the memory 501, processor 502, and communication interface 503 are integrated on a single chip, then the memory 501, processor 502, and communication interface 503 can communicate with each other through an internal interface.

[0156] Processor 502 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0157] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described visual servo control method based on heterogeneous joint segmented motion decomposition.

[0158] This application also provides a computer program product, including a computer program that can run computer instructions. When the computer instructions are executed by a processor, they implement the visual servo control method based on heterogeneous joint segmented motion decomposition provided in this application.

[0159] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0160] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0161] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0162] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0163] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or more of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0164] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0165] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0166] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A visual servo control method based on heterogeneous joint segmented motion decomposition, characterized in that, Includes the following steps: Based on the target pose measurement data of the agile arm, the target pose measurement data is processed in steps to obtain target pose deviation data; Based on the target pose deviation data, the expected joint angle motion endpoint of the agile arm in the subsequent target cycle is planned, and the joint motion data of the agile arm in the subsequent target cycle in the online stage is planned according to the expected joint angle motion endpoint in the offline stage. The joint motion data is converted into motor motion data, and the motor motion data is segmented within the subsequent target cycle to obtain segmented motor motion data, so as to control the agile arm to perform corresponding visual servo control actions using the segmented motor motion data. Specifically, the step of planning the expected joint angle motion endpoint of the agile arm in the subsequent target cycle based on the target pose deviation data, and planning the joint motion data of the agile arm in the subsequent target cycle in the online phase based on the expected joint angle motion endpoint in the offline phase, includes: planning the joint motion data of the agile arm in the subsequent target cycle based on the joint response characteristics, joint motion characteristics and joint angle increment of the agile arm.

2. The method according to claim 1, characterized in that, The target pose measurement data based on the agile arm is processed in steps to obtain target pose deviation data, including: The end-effector step length threshold of the agile arm is calculated based on the target pose measurement data; The target pose measurement data is processed in steps based on the end-effector step length threshold to obtain the target pose deviation data.

3. The method according to claim 1, characterized in that, The planning of the agile arm's expected joint angle motion endpoint in subsequent target cycles includes: Obtain the end pose and current joint angle of the target pose deviation data; Calculate the joint angle increment based on the end-effector pose and the current joint angle; The desired joint angle motion endpoint of the agile arm in a subsequent target cycle is planned in visual servo speed mode based on the joint angle increment and the current joint angle, or the desired joint angle motion endpoint of the agile arm in a visual servo position mode is planned in visual servo position mode based on the end pose and the current joint angle.

4. The method according to claim 1, characterized in that, The step of planning the expected joint angle motion endpoint of the agile arm in subsequent target cycles based on the target pose deviation data, and planning the joint motion data of the agile arm in subsequent target cycles in the online phase based on the expected joint angle motion endpoint in the offline phase, includes: The actual requirements for the agile arm are determined based on the target pose deviation data. Based on the actual requirements, the planning method for the expected joint angle motion endpoint and the joint motion data in the subsequent target period is determined to plan the expected joint angle motion endpoint and the joint motion data. The planning method includes visual servo planning and offline planning.

5. A visual servo control device based on heterogeneous joint segmented motion decomposition, characterized in that, include: The processing module is used to process the target pose measurement data in steps based on the target pose measurement data of the agile arm to obtain target pose deviation data. The planning module is used to plan the expected joint angle motion endpoint of the agile arm in the subsequent target cycle based on the target pose deviation data, and to plan the joint motion data of the agile arm in the subsequent target cycle in the online phase according to the expected joint angle motion endpoint in the offline phase. The control module is used to convert the joint motion data into motor motion data, and to segment the motor motion data within the subsequent target cycle to obtain segmented motor motion data, so as to use the segmented motor motion data to control the agile arm to perform corresponding visual servo control actions. The planning module includes a second planning unit, used to plan the joint motion data of the agile arm in a subsequent target cycle based on the joint response characteristics, joint motion characteristics and joint angle increments of the agile arm.

6. The apparatus according to claim 5, characterized in that, The processing module includes: A calculation unit is used to calculate the end-effector step length threshold of the agile arm based on the target pose measurement data; The step-by-step unit is used to process the target pose measurement data step by step based on the end-effector step length threshold to obtain the target pose deviation data.

7. An electronic device, characterized in that, include: The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the visual servo control method based on heterogeneous joint segmented motion decomposition as described in any one of claims 1-4.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the visual servo control method based on heterogeneous joint segmented motion decomposition as described in any one of claims 1-4.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed, it is used to implement the visual servo control method based on heterogeneous joint segmented motion decomposition as described in any one of claims 1-4.

Images