Shared autonomous reconfigurable control method for multi-arm system remote operation

Abstract

A kind of shared autonomous reconfigurable control method of multi-arm system remote operation, constructs scalable single-arm control mathematical model: the stiffness index of user's arm and the expected end effector posture are sent as reference to robot controller, and the expected joint torque is calculated by variable cartesian impedance controller.Multiple robot arm control mode is constructed: coordinated control mode allows any possible relative posture between end effectors, and collaborative work between multiple robots is realized through shared database and distributed control;independent control mode simulates the movement of human hand through any number of robot end effectors, and realizes independent work and flexible control.Control strategy conversion protocol is designed: conversion protocol and working principle between control strategies are formulated, so that users can intuitively control multi-robot system.The present application realizes the coordinated and independent control of any number of robots, and improves the flexibility and adaptability of the system.

Description

Technical Field

[0001] The present invention belongs to the technical field of electric power automation, and in particular relates to a shared autonomous reconfigurable control method for remote operation of a multi-arm system. Background Art

[0002] Teleoperation holds great industrial value because it integrates human cognitive skills into systems, particularly for operating in unknown, hazardous, or dynamic environments. Within this broad research area, it is possible to find applications that demonstrate the usability of these systems, such as space exploration, research and rescue, underwater, and industrial environments. Recent technological advances in collaborative robotics and motion capture systems have reduced the traditional limitations of effective teleoperation and rekindled interest in these applications. In teleoperation systems, the motion of the slave robot can be controlled through various interfaces, such as a joystick, tactile interface, 3D mouse, or, as in our case, a motion capture system. Among these, direct motion mapping has been found to be the most intuitive and effective method for user manipulation of the robot. The success of teleoperation also relies on the system's ability to adapt to its interaction with the environment. In this context, the teleimpedance paradigm is a powerful tool because it estimates the impedance of the user's arm using surface electromyography (sEMG) and replicates it on the robot. In this way, the telemanipulator mimics human dynamic behavior, increasing safety, adaptability, and efficiency. As the complexity of robotic manipulation tasks increases, the simultaneous use and coordination of multiple robots is essential, as it also expands the workspace and the range of payloads that can be manipulated. In particular, bimanual manipulation is becoming increasingly popular, requiring dual-arm robotic systems where the manipulators coordinate to achieve a common goal. To this end, the concept of shared control, in which the robot operates with a certain degree of autonomy to reduce user effort, has been introduced into teleoperation systems where humans and robots share a common task. The aforementioned studies have all implemented different shared control teleoperation strategies, but all are limited to dual-arm teleoperation.

[0003] Compared with the existing technology, the technical differences are as follows:

[0004] The technical differences between the patent application number 202211048193.9 and the patent name "Human-machine shared autonomous teleoperation method and system based on multi-motion skill priors" are as follows:

[0005] 1. Technical Differences 1

[0006] This patent is compatible with "independent control" and "interactive control", and calculates the expected posture and actual posture of each selected robot according to the proposed control strategy.

[0007] 2. Technical Differences 2

[0008] This patent does not require "a method based on dynamic motion primitives to build a motion skill library by teaching and learning different task trajectories", but rather a pure spatial constraint, and uses sensor measurement to calibrate the force feedback data during the operation. The speed of the manipulator is also taken into consideration.

[0009] 3. Technical Differences 3

[0010] This patent can be extended to multiple robotic arms. Summary of the Invention

[0011] To address the above technical issues, the present invention proposes a shared autonomous reconfigurable control method for remote operation of a multi-arm system. The patent aims to overcome the structural constraints of two robots, when one arm controls the end effector, and to handle any number of manipulators. The patent first proposes a coordinated control mode to allow any possible relative posture between the end effectors, and secondly proposes independent and coordinated control modes to control any number of single-arm robots.

[0012] To achieve the above object, the technical solution adopted by the present invention is:

[0013] A shared autonomous reconfigurable control method for remote operation of a multi-arm system is described as follows:

[0014] 1) A scalable single-arm control mathematical model was constructed;

[0015] 2) Constructed a multi-robot control model;

[0016] 3) Designed the working principles of these units and the protocols used for mutual conversion;

[0017] 4) Build independent control according to the control needs of multiple robotic arms;

[0018] 5) Build coordinated control based on the control needs of multiple robotic arms.

[0019] As a further improvement of the present invention, the scalable single-arm control mathematical model in step 1) is specifically as follows:

[0020] Remote impedance control is used to regulate the motion and dynamic behavior of each robot. The user's arm stiffness index AAA and the desired end-effector posture are sent to the robot controller as references. The desired posture frame BB and the robot base CC are represented by DD. EE is the homogeneous transformation matrix of the frame FF at time t, which is defined as formula (1):

[0021]

[0022] in are the rotation matrix and position vector respectively. The desired stiffness matrix of the end effector sent to the robot controller is defined as follows:

[0023]

[0024] where k l (t) and k ω (t) represent the required one-dimensional linear stiffness and rotational stiffness, respectively, and the user stiffness index s a (t) is set proportionally, so the desired joint torque τ des (t) is calculated by the variable Cartesian impedance controller as formula (3):

[0025]

[0026] Where q(t) is the joint position vector. The force feedback data during the operation is calibrated by offline sensor measurement, and the force feedback model is formed according to the difference between the operation type and the object. T (q(t)) is the Jacobian matrix of the robot arm, It is an expectation attitude and actual posture The position and orientation error vector between is its time derivative, K(t) and D(t) are the variable stiffness and damping matrices, D(t) is the damping matrix of the end effector, defined by its relationship with K(t), M(q(t)) is the joint mass matrix, is the centrifugal Coriolis term, g(q(t)) represents the contribution of gravity compensation torque, is the zero-space impedance used to control the robot's redundancy, Satisfying formula (4):

[0027]

[0028] Formula (4) and are joint position error and velocity error respectively, K q and D q is the joint space stiffness matrix and damping matrix, q d To force the joints away from their position limits, , is set to 0, but this can be changed as needed to optimize the internal robot configuration rather than the task objectives or to avoid collisions with the environment. N(q(t)) is the null space projection, and N(q(t)) satisfies Equation (5):

[0029] N(q(t))=IJ T (q(t))J ΥT (q(t)) (5)

[0030] inΥ It represents the pseudo-inverse operator.

[0031] As a further improvement of the present invention, the multi-robot control mode in step 2) includes coordinated control and independent control.

[0032] As a further improvement of the present invention, the working principles of these units in step 3) and the protocols used for mutual conversion are as follows;

[0033] To enable users to intuitively control the multi-manipulator system, each proposed control strategy requires two inputs obtained through the joystick. The multi-manipulator control mode is based on the robot id and the control owner;

[0034] Robot id: This input instructs the user to select the robot to be controlled. When the user selects n ≥ 2 robots, a robot group is created. A group of robots will move together using one of the control strategies.

[0035] Control Owner: Defines the user hand that is selected to teleoperate the robot group. In the following section, Ψ hand A frame for indicating the hand, which is the controlling owner of the chosen strategy;

[0036] During teleoperation, the control strategy proposed is used to calculate the and K ee (t), then, the local controller of each robot calculates the desired joint torque through equation (3).

[0037] As a further improvement of the present invention, the step 4) multi-manipulator control needs to construct independent control as follows;

[0038] By using any number of robot end effectors to imitate the motion of a human hand, when controlled at time t = t s At the beginning, the multi-manipulator is initialized to obtain the initial relative rotation between the hand frame and the end effector frame of the selected robot and translation The distance between each robot's hand and the end effector and Respectively by and get;

[0039] The translation vector of the hand relative to the end effector is expressed in the robot frame and satisfies formula (6):

[0040]

[0041] where Ψ bi and Represent the base frame and end effector frame of the i-th robot respectively. After initialization, the control loop step 2) begins to execute, and the desired posture of each robot is calculated as formula (7):

[0042]

[0043] In (7), the current hand rotation Multiply by the initial rotation offset from step 1) Similarly, the translation offset is added to the current basis to the hand position vector The rotation and translation of the hand are directly transmitted to the end of the robot.

[0044] As a further improvement of the present invention, in step 4), the stiffness of the robot is adjusted by a mapping method to simulate the stiffness profile of the user's arm commanding the robot.

[0045] As a further improvement of the present invention, the step 5) multi-manipulator control needs to construct coordinated control as follows;

[0046] In this control strategy, the user manipulates the translation and rotation of the virtual frame located between the robot end effectors. In independent control, when the control strategy is at t = t s During initialization, the virtual frame Ψ ν Created at the midpoint between the robot end effectors and oriented as the command hand, satisfying formula (8)

[0047]

[0048] Where Ψ w ,Ψ v , Ψ hard 、 n are the world frame, virtual frame, hand frame, end-effector frame of the i-th robot, and the total number of robots controlled in this mode, and then Ψ v Linked to the technology described in the independent control claim hard :register and The virtual frame pose during the control loop is then calculated using Equation (9):

[0049]

[0050] The robot end effector is connected to the virtual frame through virtual mobile joints, given The desired pose of the i-th end effector relative to the virtual frame is calculated as (9)

[0051]

[0052] wherein satisfies formula (10):

[0053]

[0054] saturated between the lower limit l min and the upper limit l max to prevent robots from colliding with each other or reaching unreachable positions, α(t) is defined in (10) as formula (11),

[0055]

[0056] According to the joystick command, increase or decrease α(t) according to the joystick command using ε(t). ε(t) is defined as formula (12):

[0057]

[0058] In formula (12), when the joystick command is increased, take ε, when it is less, take -ε, and otherwise take 0. The pose sent to the i-th robot controller as a reference is formula (13):

[0059]

[0060] where the current base coordinate system of the i-th robot is transformed to the virtual coordinate system, multiplied by the desired transformation matrix of the robot end effector desired coordinate system relative to the virtual coordinate system .

[0061] Summarizing the advantages of the above patent "Shared autonomous reconfigurable control method for remote operation of multi-arm system", the following points can be summarized:

[0062] 1. Flexibility and scalability;

[0063] Flexible control mode: This patent proposes various control modes such as coordinated control, independent control, etc., allowing users to choose the appropriate control strategy according to task requirements, thereby improving the flexibility of the system.

[0064] Scalable single-arm control mathematical model: A scalable mathematical model is constructed, which can support remote operation of any number of mechanical arms, providing a foundation for future possible expansion.

[0065] 2. High efficiency;

[0066] Remote impedance control: Remote impedance control is used to adjust the motion and dynamic behavior of the robot, which can more accurately control the pose and stiffness of the end effector, improving work efficiency.

[0067] Real-time calculation and adjustment: By calculating the desired joint torque and pose in real time and adjusting according to the force feedback data during operation, the robot can accurately and quickly respond to control instructions.

[0068] 3. Adaptability and reconfigurability;

[0069] Adapting to changing tasks: This control method can handle any number of manipulators and adapt to different scales and complexity of task requirements.

[0070] Reconfigurable control: By constructing multi-arm control modes, flexible conversion and reconfiguration of control strategies are realized, improving the adaptability of the system to different working environments.

[0071] 4. User experience optimization;

[0072] Intuitive control interface: An intuitive control interface and protocol are designed to allow users to easily control the multi-manipulator system, reducing the difficulty of operation.

[0073] Simulate human hand movement: Through independent control mode, the movement of human hand can be simulated, improving the operation experience and work efficiency of users.

[0074] 5. Cost and reliability;

[0075] Reduced cost: By sharing the network and autonomous control method, the cost and maintenance cost of the remote operation system are reduced.

[0076] High reliability: By using the openness and interconnectivity of the network and reliable control algorithm, the reliability and stability of the system are improved.

[0077] In summary, this patent has significant advantages in flexibility, efficiency, adaptability, user experience, cost and reliability, and provides an innovative multi-arm system remote operation control method for the field of power automation technology. BRIEF DESCRIPTION OF DRAWINGS

[0078] Figure 1 is the relationship between the architecture and control mode of the present invention. DETAILED DESCRIPTION

[0079] The present invention will be described in further detail below in conjunction with the drawings and specific embodiments:

[0080] A shared autonomous reconfigurable control method for multi-arm system remote operation constructs an extensible single-arm control mathematical model.

[0081] Remote impedance control is used to regulate the motion and dynamic behavior of each robot. To this end, the user's arm stiffness index AAA and the desired end-effector pose are sent to the robot controller as references. The desired pose frame BB. The robot base frame CC is denoted by DD, and EE is the homogeneous transformation matrix of the frame FF at time t, defined as formula (1):

[0082]

[0083] in are the rotation matrix and position vector respectively. The desired stiffness matrix of the end effector sent to the robot controller is defined as formula (2):

[0084]

[0085] where k l (t) and k ω (t) represents the required one-dimensional linear stiffness and rotational stiffness respectively. a (t) is set proportionally, so the desired joint torque τ des (t) is calculated by the variable Cartesian impedance controller as formula (3):

[0086]

[0087] Where q(t) is the joint position vector, and the force feedback data during the operation is calibrated by offline sensor measurement, and the force feedback model is formed according to the difference between the operation type and the object. T (q(t)) is the Jacobian matrix of the robot arm, It is an expectation attitude and actual posture The position and orientation error vector between is its time derivative. K(t) and D(t) are the variable stiffness and damping matrices. D(t) is the damping matrix of the end effector and can be defined by its relationship with K(t). M(q(t)) is the joint mass matrix. is the centrifugal Coriolis term, and g(q(t)) represents the contribution of gravity compensation torque. is the zero-space impedance used to control the redundancy of the robot. Satisfying formula (4):

[0088]

[0089] Formula 4 and are joint position error and velocity error respectively. K q and D q is the joint space stiffness matrix and damping matrix. In this patent, q dTo force the joints away from their position limits, it is usually set to 0, but this can be changed as needed, for example, to optimize the internal robot configuration rather than the task objectives or to avoid collisions with the environment. N(q(t)) is the null space projection, and in this patent N(q(t)) satisfies formula (5):

[0090] N(q(t))=IJ T (q(t))J ΥT (q(t)) (5)

[0091] in Υ It represents the pseudo-inverse operator.

[0092] This application also constructs a multi-manipulator control mode, which is characterized by proposing a control architecture including three main modes, namely coordinated control and independent control.

[0093] The relationship between this architecture scheme and control mode is as follows Figure 1 The operating principles of these units and the protocols used for their mutual conversion are designed to enable users to intuitively control the multi-manipulator system. Each proposed control strategy requires two inputs obtained via joysticks.

[0094] The multi-robot control mode is mainly based on the robot id and the control owner. Robot id: This input indicates the user to select the robot to be controlled. When the user selects n ≥ 2 robots, a robot group is created. The group of robots will move together using one of the control strategies. Control owner: Defines the user hand that is selected to remotely operate the group of robots. In the following section, Ψ hand A frame used to indicate the hand that is the controlling owner of the selected strategy.

[0095] During teleoperation, the control strategy proposed is used to calculate the and K ee (t). Then, the local controller of each robot calculates the desired joint torque through equation (3).

[0096] Multi-manipulator control requires building independent controls.

[0097] The main goal of this claim is to mimic the motion of a human hand by any number of robotic end effectors. s At the beginning, the multi-manipulator is initialized to obtain the initial relative rotation between the hand frame and the end effector frame of the selected robot and translation The distance between each robot's hand and the end effector and Respectively by and Get. Figure 1 shown.

[0098] The translation vector of the hand relative to the end effector is expressed in the robot frame and satisfies formula (6):

[0099]

[0100] where Ψ bi and Represent the base frame and end effector frame of the i-th robot respectively. After initialization, the control loop Figure 1 Step 2 in the above example is executed. The expected posture of each robot is calculated as formula (7):

[0101]

[0102] In (7), the current hand rotation Multiply by the initial rotation offset from step 1 Similarly, the translation offset is added to the current basis to the hand position vector In this way, the rotation and translation of the hand can be directly transmitted to the end of the robot. In addition, the stiffness of the robot is adjusted through a mapping method to simulate the stiffness profile of the user's arm that commands the robot.

[0103] Multi-manipulator control requires the construction of coordinated control.

[0104] In this control strategy, the user manipulates the translation and rotation of the virtual frame located between the robot end effectors. In independent control, when the control strategy is at t = t s During initialization, the virtual frame Ψ ν Created at the midpoint between the robot end effectors and oriented as the command hand, satisfying formula (8)

[0105]

[0106] Where Ψ w , Ψ v , Ψ hard 、 n are the world frame, virtual frame, hand frame, end effector frame of the i-th robot, and the total number of robots controlled in this mode. Then Ψ v Linked to the technology described in the independent control claim hard :register and The virtual frame pose during the control loop is then calculated using Equation (9):

[0107]

[0108] The robot end effectors are connected to the virtual frame by virtual movement joints. Given The desired pose of the ith end effector with respect to the virtual frame is computed as (9)

[0109]

[0110] where The formula (10) is satisfied:

[0111]

[0112] Saturation between the lower limit l min and the upper limit l max to prevent the robots from colliding with each other or from reaching unreachable positions. α(t) is defined in (10) as formula (11),

[0113]

[0114] According to the joystick command, increase or decrease α(t) according to the joystick command using ε(t). ε(t) is defined as formula (12):

[0115]

[0116] In formula (12), when the joystick command is increased, take ε, when it is less, take -ε, and otherwise take 0. The pose sent to the ith robot controller as a reference is formula (13):

[0117]

[0118] where the current base coordinate system of the ith robot is multiplied by the desired transformation matrix of the robot end effector with respect to the virtual coordinate system .

[0119] The above description is only the preferred embodiment of the present application, and is not intended to limit the present application in any other form, and any modification or equivalent change made according to the technical essence of the present application still falls within the scope of the present application.

Claims

1. A shared autonomous reconfigurable control method for remote operation of a multi-arm system, specifically as follows, characterized by: 1) A scalable single-arm control mathematical model was constructed; The scalable single-arm control mathematical model of step 1) is specifically as follows: Remote impedance control is used to regulate the motion and dynamic behavior of each robot. The user’s arm stiffness index and the desired end-effector pose are sent to the robot controller as references. The desired pose frame, the robot base frame, is expressed in pose coordinates. is the homogeneous transformation matrix of the rack at time t, defined as formula (1): (1) in 、 are the rotation matrix and position vector respectively. The desired stiffness matrix of the end effector sent to the robot controller is defined as formula (2): （2） in and Represents the required one-dimensional linear stiffness and rotational stiffness, respectively, and the user stiffness index are set proportionally, so the desired joint torque The variable Cartesian impedance controller is calculated as formula (3): （3） in It is the joint position vector, which is used to calibrate the force feedback data during the operation by offline sensing measurement, and form a force feedback model based on the difference between the operation type and the object. is the Jacobian matrix of the robot arm, It is an expectation attitude and actual posture The position and orientation error vector between is its time derivative, is the variable stiffness, is the damping matrix of the end effector, which is To define the relationship, is the joint mass matrix, is the centrifugal Coriolis term, represents the contribution of gravity compensation torque, is the zero-space impedance used to control the robot's redundancy, Satisfying formula (4): （4） Formula (4) and are joint position error and velocity error, respectively. and are the joint space stiffness matrix and damping matrix, To force the joints to move away from their position limits, set to 0, but this can be changed as needed to optimize the internal robot configuration rather than the mission objectives or to avoid collisions with the environment, is the null space projection, Satisfies formula (5): （5） in It represents the pseudo-inverse operator; 2) Constructed a multi-robot control model; 3) Designed the working principles of these units and the protocols used for mutual conversion; 4) Build independent control according to the control needs of multiple robotic arms; 5) Build coordinated control based on multi-robot control needs.

2. The shared autonomous reconfigurable control method for remote operation of a multi-arm system according to claim 1, characterized in that: The multi-manipulator control mode in step 2) includes coordinated control and independent control.

3. The shared autonomous reconfigurable control method for remote operation of a multi-arm system according to claim 1, characterized in that: The working principles of these units and the protocols used for mutual conversion in step 3) are as follows: To enable users to intuitively control the multi-manipulator system, each proposed control strategy requires two inputs obtained through the joystick. The multi-manipulator control mode is based on the robot id and the control owner; Robot id: This input instructs the user to select the robot to be controlled. When the user selects n ≥ 2 robots, a robot group is created. A group of robots will move together using one of the control strategies. Control Owner: Defines the user hand that is selected to teleoperate the robot group, in the following section, A frame for indicating the hand, which is the controlling owner of the chosen strategy; During teleoperation, the control strategies proposed are used to calculate the and ,Then, the local controller of each robot calculates the desired joint torque through Equation (3).

4. The shared autonomous reconfigurable control method for remote operation of a multi-arm system according to claim 1, characterized in that: The step 4) multi-manipulator control requires the construction of independent control as follows; By using any number of robot end effectors to imitate the motion of the human hand, when controlled in time At the beginning, the multi-manipulator is initialized to obtain the initial relative rotation between the hand frame and the end effector frame of the selected robot and translation , the distance between each robot's hand and the end effector and Respectively by and get; The translation vector of the hand relative to the end effector is expressed in the robot frame and satisfies formula (6): （6） in and Represent the base frame and end effector frame of the i-th robot respectively. After initialization, the control loop step 2) starts to execute, and the desired posture of each robot is calculated as formula (7): （7） In (7), the current hand rotation Multiply by the initial rotation offset from step 1) , similarly, the translation offset is added to the current basis to the hand position vector , the rotation and translation of the hand are directly transmitted to the end of the robot.

5. The shared autonomous reconfigurable control method for remote operation of a multi-arm system according to claim 1, characterized in that: In step 4), the stiffness of the robot is adjusted by a mapping method to simulate the stiffness profile of the user's arm commanding the robot.

6. The shared autonomous reconfigurable control method for remote operation of a multi-arm system according to claim 1, characterized in that: The step 5) multi-manipulator control requires the construction of coordinated control as follows; In this control strategy, the user manipulates the translation and rotation of the virtual frame located between the robot end effectors. In independent control, when the control strategy is During initialization, the virtual frame Created at the midpoint between the robot end effectors and oriented as the command hand, satisfying formula (8) （8） In the formula 、 、 、 , n are the world frame, virtual frame, hand frame, end effector frame of the i-th robot, and the total number of robots controlled in this mode, respectively. Linked to the technology described in the independent control claim :register and Then use Equation (9.1) to calculate the virtual frame pose during the control loop: （9.1） The robot end effector is connected to the virtual frame through virtual mobile joints, given , the desired pose of the i-th end effector relative to the virtual frame is calculated as (9.2) （9.2） in Satisfies formula (10): （10） At the lower limit and upper limit Saturation between them to prevent robots from colliding with each other or commanding positions that cannot be reached, defined in (10) As shown in formula (11), （11） According to the instructions of the joystick, use Increase or decrease according to joystick command . Defined as formula (12): (12) In formula (12), when the joystick command is increasing, take , is a young age, take - , in other cases it is 0, and the posture sent to the i-th robot controller as a reference is formula (13): （13） Among them, the i-th robot The transformation from the current base coordinate system to the virtual coordinate system is multiplied by the desired coordinate system of the robot end effector relative to the virtual coordinate system The expected transformation matrix.

Images