An optimization-based robot variable stiffness vision-impedance control method and system

Abstract

The application discloses a kind of robot variable stiffness vision-impedance control method and system based on optimization, the coordinate conversion relationship between camera and six-dimensional force sensor and robot end is established, six-dimensional force sensor is calibrated, by analyzing the interactive dynamics relationship of robot and environment, and the mathematical model of robot-environment interaction is constructed, by solving vision servo acceleration model, combined with the dynamics equation of robot in assembly process, a characteristic space man-robot-environment interactive dynamics model is established;Establish the characteristic space man dynamics model as a planner to obtain the reference feature trajectory of the implicit human recognition contact dynamics;Design the characteristic space variable stiffness impedance controller based on QP online planning, which can dynamically adjust the impedance parameters during assembly process, and compensate the disturbance force caused by environmental uncertainty in real time, and maintain constant force in rich contact task, to ensure that the robot can complete the assembly task stably and with high precision.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of intelligent manufacturing, and particularly relates to a robot variable stiffness vision-impedance control method and system based on optimization. BACKGROUND

[0002] With the development of industrial manufacturing towards high precision, high efficiency and flexibility, complex component manufacturing, such as aviation panel assembly, precision electronic manufacturing, etc., puts forward higher requirements on the intelligent degree of production system. However, the manufacturing process is often subject to many constraints, such as narrow working space, numerous assembly procedures, dynamic changes in environment, etc., and the traditional manufacturing mode is difficult to meet the needs of modern industry.

[0003] At present, the manufacturing industry mainly relies on two ways of production: one is manual operation, which is flexible but low in efficiency, labor-intensive, and greatly influenced by the experience and skill level of the operator; the other is robot production line operation, which can realize efficient production, but its fixed algorithm is often only suitable for repetitive tasks. In the face of changes in production environment, unstructured manufacturing scenarios and complex assembly tasks, the adaptability and intelligent ability of traditional robot manufacturing mode are insufficient. Therefore, improving the perception ability of robots to the manufacturing environment, enabling them to adapt to changes autonomously, and improving their fine operation ability, have become urgent problems in the field of intelligent manufacturing.

[0004] In intelligent manufacturing, vision technology has been widely used in workpiece positioning, quality detection and other tasks. Robots can recognize the position, attitude and assembly quality of target objects by perceiving environmental information through vision sensors. However, real manufacturing tasks are usually rich in contact tasks, and single visual information cannot meet the requirements of fine operation. For example, in the process of aviation panel assembly, the panel is a weak rigid component. If the robot relies only on visual positioning for assembly, it may produce tremor due to insufficient control precision during contact, affecting the assembly quality, and even may cause damage to the workpiece due to excessive contact force.

[0005] On the other hand, when robots perform rich contact tasks, they face the problem of unstructured characteristics of the environment and their own dynamics constraints. Because of the coupled dynamics relationship between the robot and the workpiece, the instability of the contact force may lead to task failure. Therefore, in addition to visual information, force sensation needs to be introduced to obtain contact force information when the robot interacts with the environment, and then the robot control strategy is optimized to improve the stability and adaptability of the operation.

[0006] In summary, in the field of intelligent manufacturing, how to effectively fuse multi-modal sensing information (such as vision and force sense) and overcome the complex coupling dynamics between human-robot-environment to achieve high-precision adaptive assembly is a core technical problem that needs to be solved at present. Therefore, the present application proposes a robot variable stiffness vision-impedance control method based on optimization to improve the adaptability of robots in complex manufacturing environments. SUMMARY

[0007] To solve the above technical problems, the present application provides a robot variable stiffness vision-impedance control method and system based on optimization.

[0008] The technical solution adopted by the present application to solve its technical problems is:

[0009] A robot variable stiffness vision-impedance control method based on optimization, the method comprising the following steps:

[0010] S100: Establish the coordinate transformation relationship between the camera, the six-axis force sensor and the robot end, and calibrate the six-axis force sensor, compensate for the sensor's own zero drift and tool gravity through the force calibration algorithm;

[0011] S200: Consider the dynamics characteristics of the robot under the force state, including the guiding force applied by the human and the reaction force of the environment, and combine the robot kinematics and dynamics equations, as well as the vision servo acceleration equation, to build a complete human-robot-environment interaction dynamics model in the feature space;

[0012] S300: Based on the dynamics model, a human guiding force model is established in the feature space, the vision features are associated with the robot motion and a reference trajectory is generated, a variable stiffness vision impedance controller is designed, the stiffness matrix is optimized online through QP to ensure constant contact force and meet the torque constraint, the system equation is derived to realize external force convergence, and an energy function is constructed to analyze stability;

[0013] S400: According to the stability analysis result, introduce energy tank based online stiffness enhancement, construct energy tank state equation and upper and lower limit constraints to limit active energy injection, establish a quadratic programming QP model, optimize the stiffness parameters in real time under the passivity constraint and ensure that the total energy is bounded, and conduct passivity analysis by constructing a system storage function;

[0014] S500: Complete the gear assembly by human guiding robot, integrate six-axis force sensing and RealSense vision data, extract feature point trajectory error in real time based on AprilTag algorithm, drive QP optimization to dynamically adjust variable stiffness parameters, map to joint space combined with Jacobian pseudo-inverse to drive robot arm motion, and realize compliant trajectory tracking.

[0015] Preferably, S100 establishes the coordinate transformation relationship between the camera and the six-dimensional force sensor and the robot end, including:

[0016] S110: Install a six-dimensional force sensor and an RGB-D camera at the end of the mechanical arm, ensuring that the camera's field of view is not obstructed;

[0017] S120: Use the end camera to take pictures of the calibration board with known geometric features, and record the corresponding mechanical arm end pose at this time;

[0018] S130: Repeat S120 a preset number of times at different positions and attitudes, take pictures of the calibration board, and record the pose data of the mechanical hand end synchronously;

[0019] S140: By analyzing the captured calibration board images and recorded end pose data, the rotation matrix and translation vector of the camera relative to the mechanical arm end are calculated , and the coordinate transformation relationship between the camera and the mechanical hand end is solved using the Tsai-Lenz method combined with the eye-in-hand calibration technique , specifically:

[0020] ;

[0021] S150: Use modeling software to build an assembly model of the flange, six-dimensional force sensor and camera, and calculate the rotation matrix and translation vector of the six-dimensional force sensor relative to the camera through geometric analysis, to obtain the coordinate transformation relationship between the six-dimensional force sensor and the camera , specifically:

[0022] .

[0023] Preferably, S200 includes:

[0024] S210: When a person applies force to the robot and interacts with the environment, the dynamics of the person-guided kinematic chain robot in the joint space is described as:

[0025] ;

[0026] where, are the generalized variables of the joint angle, joint velocity and joint acceleration of the robot manipulator, represents a symmetric and positive definite inertia matrix, describes the Coriolis force and centripetal force matrix, is the gravity vector, is the joint drive command vector, is the torque exerted by a human on the end-effector, is the external moment;

[0027] S220: Solve the acceleration-level visual servoing dynamics to characterize the motion relationship between the visual features and joint variables, considering image features , the kinematic relationship between the motion of image features and the associated motion screw is:

[0028] ;

[0029] where, is the velocity of image features, is the image interaction matrix, which links the velocity of feature points to the robot motion screw, and denote the motion screw of camera and object, respectively;

[0030] S230: When the object is stationary, i.e. , the above equation can characterize the kinematic relationship between the feature velocity and joint velocity through the Jacobian function of the robot, i.e.

[0031] ;

[0032] where represents the task Jacobian matrix, is the robot Jacobian matrix, is the screw transformation matrix, in this way, the velocity of image features can be represented as a function of joint velocity, thus providing a foundation for subsequent control strategy design;

[0033] S240: Differentiate the equation in S230 with respect to time to obtain the acceleration form expression of image features:

[0034] ;

[0035] where, denotes the derivative of the task Jacobian matrix, which is the rate of change over time;

[0036] In order to facilitate the concise derivation of subsequent expressions, the above equation is rewritten as:

[0037] ;

[0038] where, is the acceleration of image features, is the acceleration coupling matrix, denotes the derivative of the image interaction matrix, which is rate of change over time, denotes the derivative of the Jacobian matrix of the robot, and rate of change over time;

[0039] S250: The human guide n-link robot dynamics equation in S210 is rearranged to derive the dynamics characteristics of the human guide robot in the feature space:

[0040] ;

[0041] The feature point velocity expression in S230 and the feature point acceleration expression in S240 are substituted into the dynamics characteristics formula to obtain the dynamic equation related to the feature motion:

[0042] ;

[0043] wherein is a comprehensive force term; the above formula is a set of nonlinear high-coupling differential equations corresponding to image features;

[0044] S260: A task Jacobian coefficient is introduced to consider a projection matrix that can convert the human-applied torque in the human tool coordinate system to the camera coordinate system, so that , wherein is the transpose of the kinematics Jacobian matrix of the human guide tool end relative to the robot joint, which is substituted into the above feature-related dynamic equation to obtain:

[0045] ;

[0046] wherein represents a virtual force acting on the image feature, the virtual force comes from the human guide torque projected into the camera coordinate system, is a virtual external force in the feature space, represents a virtual gravity, a virtual Coriolis force and a centripetal force in the feature space, represents a damping matrix, represents a feature space impedance controller, which is a virtual control input of the robot in the feature space, represents the human-applied torque represented in the camera coordinate system.

[0047] Preferably, S300 comprises:

[0048] S310: In the feature space, the virtual human guide force and acting on the image feature, for ease of analysis, the human guide force projected into the feature space is modeled as:

[0049] ;

[0050] where, and represent the velocity and acceleration variables of the human-guided feature, respectively, denotes the damping matrix;

[0051] By the above equation, the visual feature variable and the robot end-effector can move in the same direction as the human-guided external force torque, while the human-guided feature trajectory , which is required as a reference trajectory for the subsequent controller design, can be obtained;

[0052] S320: A variable-stiffness visual impedance controller is designed to act on the feature space based on S310, and the stiffness is adjusted by the online optimized QP method to ensure that the robot end-effector maintains a constant force when performing a contact-rich task; a feature space impedance controller , which is in the form of:

[0053] ;

[0054] where, is the feature tracking error vector, is the rate of change of the feature tracking error, and represent the stiffness and damping matrices of the impedance control, respectively;

[0055] S330: The equations in S310 and S320 are brought into S260 to obtain:

[0056] ;

[0057] Observing the above equation, it is further derived that:

[0058] ;

[0059] where, is the second derivative of the rate of change of the feature tracking error, representing the acceleration of the change of the feature tracking error;

[0060] Since the performance of the impedance system depends on its parameters, it is desired that the virtual external force in the feature space can converge to the required external force ;

[0061] S340: To solve the force convergence problem in S320, the following QP problem is constructed for online adjustment of the stiffness of the impedance system:

[0062] ;

[0063] ;

[0064] wherein, and denote the minimum and maximum allowable stiffness, respectively, denotes the maximum external moment that the robot can exert, and are weighting matrices used to regulate the external moment and the minimum stiffness, respectively;

[0065] S350: After the controller optimization described above, in order to facilitate the subsequent stability analysis, the energy function of the impedance system in the feature space is defined as:

[0066] ;

[0067] The derivative of which with respect to time is:

[0068] ;

[0069] Since the sign of the term is not known in advance, and it is not feasible to define a proper storage function for the relative port, it is not possible to guarantee the passivity of the impedance system with respect to the port , wherein denotes the derivative of the variable stiffness , which is the rate of change of the variable stiffness.

[0070] Preferably, S400 comprises:

[0071] S410: In order to guarantee the stability of the variable impedance system in the feature space, the variable stiffness is first rewritten as , wherein and denote the constant stiffness component and the time-varying stiffness component, respectively, and without loss of generality, let ;

[0072] Subsequently, the concept of a virtual energy tank is introduced to separate the potential active behavior of the time-varying stiffness component, the energy of which is defined as , wherein denotes its dynamic state:

[0073] ;

[0074] wherein, denotes the rate of change of the energy tank dynamic state, is the energy flow switch parameter, is the energy exchange term;

[0075] wherein is defined as:

[0076] ;

[0077] where represents the lower bound of the energy tank, is the spatial-temporal stiffness matrix, in addition, to ensure the total energy of the whole interconnected system remains bounded, when reaching a certain upper bound of the energy tank , the overloading of the energy tank is guaranteed by the following way:

[0078] ;

[0079] S420: Therefore, the online optimization of stiffness under the passive constraint is carried out, and the energy tank constraint is incorporated into the optimization problem, which can be expressed as the following QP problem:

[0080] ;

[0081] ;

[0082] S430: The final feature space impedance controller is expressed as:

[0083] ;

[0084] The expression of the virtual external force in the feature space is modified as:

[0085] ;

[0086] By implementing the feature space variable stiffness visual impedance controller based on the online QP-based stiffness optimization, as described in the above two equations, it can be dynamically adjusted to maintain constant force in contact-intensive tasks, and the energy tank is initialized to satisfy ; when the energy of the virtual energy tank , the stiffness can be continuously optimized online, on the contrary, when , the stiffness degenerates to , and the controller is simplified to a constant stiffness impedance controller;

[0087] S440: For the modified energy tank constraint controller and the virtual external force equation, i.e., the equations shown in S420, S430, the dynamic state of the energy tank, i.e., the equation shown in S410, the storage function of the whole interconnected system is selected as:

[0088] ;

[0089] The differential of the total energy with respect to time is given by:

[0090] ;

[0091] wherein, represents the energy of the virtual energy tank

[0092] Since , we have:

[0093] ;

[0094] Thus we have:

[0095] ;

[0096] The above passive condition inequality shows that there exists:

[0097] ;

[0098] wherein, is the storage function value at time t, is the storage function value at initial time, is the integral variable, is the power exchange between the system and the external environment;

[0099] The above passive condition inequality proves that the modified enhanced control framework can achieve variable stiffness in the visual impedance model while safely interacting with any passive environment.

[0100] Preferably, S500 comprises:

[0101] S510: A six-axis force sensor is installed between the handle and the flange for real-time measurement of the force and torque applied by the person; a RealSense D435i camera is installed at the front end of the flange to capture image information of the gear and gear shaft; the end effector of the manipulator is connected to the six-axis force sensor through the flange, ensuring the measurement accuracy and stability of the force sensor;

[0102] S520: During the experiment, the operator guides the robot to assemble the gear into the gear shaft, while the RGB-D camera captures the image information of the gear and gear shaft in real time and records the corresponding four sets of feature point trajectory data, which include the relative position and attitude information of the gear and gear shaft, for subsequent visual servo control. By recording these data, the operation skills and dynamic characteristics in the human-guided assembly process are analyzed, providing a reference for the variable stiffness visual impedance control strategy;

[0103] S530: According to the feature point trajectory data recorded during the human-guided process, the parameters of the variable stiffness visual impedance controller are initialized, including the virtual stiffness matrix , the virtual damping matrix , desired contact force ;

[0104] S540: Feature point extraction and tracking The RealSense D435i camera captures images of the gear and gear shaft in real time at a frequency of 30 Hz. The AprilTag algorithm is used to identify four key feature points of the gear and gear shaft from the images. The position information of these feature points will be used as input for the visual servo controller. The error between the current feature point position and the hand-guided trajectory is calculated , where is the current feature point position, is the hand-guided trajectory; by monitoring the error of the feature points in real time, the controller dynamically adjusts the motion trajectory of the robot to ensure that the feature points can closely follow the hand-guided trajectory;

[0105] S550: Variable stiffness visual impedance control According to the feature point error and the QP-based variable stiffness strategy, the virtual stiffness matrix , the virtual damping matrix , the variable stiffness strategy is:

[0106] ;

[0107] ;

[0108] Through the variable stiffness parameter, the controller can respond to contact interference in the assembly process in real time, ensuring that the robot exhibits compliance during contact and avoiding damage to the gear and gear shaft;

[0109] S560: According to the updated impedance parameters and feature point error information, the control signal is calculated:

[0110] ;

[0111] where the control signal is used to guide the motion of the robot's end effector, so that the feature points can closely follow the hand-guided teaching trajectory;

[0112] S570: Convert the controller in the feature space to the control signal in the joint space:

[0113] ;

[0114] where is the pseudo-inverse of the Jacobian matrix and the inverse inertia matrix. By converting the control signal from the feature space to the joint space, the robot can adjust the motion of the end effector according to the calculated joint space control signal;

[0115] S580: The robot adjusts the motion of the end effector according to the calculated joint space control signal Adjusting the motion of the end effector so that the feature points can closely follow the demonstration trajectory guided by the human hand.

[0116] An optimization-based robot variable stiffness visual-impedance control system comprises a Sawyer robot and an end effector, the end effector comprises a flange, a six-axis force sensor and a depth camera, the flange is used to integrate the six-axis force sensor and the camera at the end of the robot, the six-axis force sensor is used to perceive the force applied by a person, and the depth camera is used to detect feature points. The Sawyer robot is used to perform the steps of an optimization-based robot variable stiffness visual-impedance control method.

[0117] A computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of an optimization-based robot variable stiffness visual-impedance control method when executing the computer program.

[0118] A computer readable storage medium stores a computer program, and the computer program implements the steps of an optimization-based robot variable stiffness visual-impedance control method when executed by a processor.

[0119] The optimization-based robot variable stiffness visual-impedance control method and system described above, by analyzing the interactive dynamics between the robot and the environment, and constructing a mathematical model of the robot-environment interaction, a feature space human-robot-environment interactive dynamics model is established by solving the visual servo acceleration model, combined with the dynamics equation of the robot in the assembly process; a feature space human guide dynamics model is established as a planner to obtain a reference feature trajectory that implicitly identifies the contact dynamics of the human; in addition, a QP-based optimization method is introduced, and a feature space variable stiffness impedance controller based on QP online planning is designed, which can dynamically adjust the impedance parameters in the assembly process and compensate for the disturbance force caused by environmental uncertainty in real time, maintain the constant force of the end contact, and thus improve the stability and precision of the rich contact task, and finally the robot can stably, intelligently, flexibly and accurately complete the assembly work. BRIEF DESCRIPTION OF DRAWINGS

[0120] Figure 1 A flowchart of an optimization-based robot variable stiffness visual-impedance control method in an embodiment of the present application;

[0121] Figure 2 A control block diagram of a feature-based human guide impedance learning adaptive control algorithm in an embodiment of the present application. DETAILED DESCRIPTION

[0122] In order for those skilled in the art to better understand the technical solutions of the present application, the present application will be further described in detail below with reference to the drawings.

[0123] In one embodiment, as shown in Figure 1 and Figure 2 An optimized robot variable stiffness vision-impedance control method, the method comprising the following steps:

[0124] S100: Establish the coordinate transformation relationship between the camera and the six-dimensional force sensor and the robot end, ensure the effective fusion of the subsequent vision-force information; at the same time, calibrate the six-dimensional force sensor, compensate the sensor itself zero drift and tool gravity through the force calibration algorithm, improve the measurement accuracy and ensure the reliability of the force data;

[0125] S200: Considering the dynamics characteristics of the robot under the force state, including the guiding force applied by the human and the reaction force of the environment, combining the robot kinematics and dynamics equations, and the vision servo acceleration equation, a complete human-robot-environment interaction dynamics model in the feature space is constructed;

[0126] S300: Based on the dynamics model, a human guiding force model is established in the feature space, the vision features are associated with the robot motion and a reference trajectory is generated, a variable stiffness vision impedance controller is designed, the stiffness matrix is optimized online through QP (Quadratic Programming) to ensure the constant contact force and meet the torque constraint, the system equation is derived to realize the external force convergence, and the energy function is constructed to analyze the stability;

[0127] S400: According to the stability analysis result, introduce the online optimization of enhanced stiffness based on energy tank, construct the energy tank state equation and the upper and lower limit constraints to limit the active energy injection, establish the quadratic programming QP model, optimize the stiffness parameters in real time under the passivity constraint and ensure the total energy is bounded, and carry out passivity analysis through the construction of system storage function;

[0128] S500: The human-guided robot completes the gear assembly, integrates the six-dimensional force sensing and RealSense vision data, extracts the feature point trajectory error in real time based on the AprilTag algorithm, drives the QP optimization to dynamically adjust the variable stiffness parameters, maps to the joint space combined with the Jacobian pseudo-inverse to drive the robot arm motion, and realizes the compliant trajectory tracking.

[0129] Specifically, in view of the following problems existing in the robot vision-force sensing fusion human-guided learning assembly: (1) human-robot-environment interaction coupling nonlinear dynamics; (2) vision-force sensing data is heterogeneous and difficult to fuse; (3) contact uncertain dynamic problems generated in the assembly task, the present application proposes an optimized robot variable stiffness vision-impedance control method.

[0130] Inspired by the fact that humans naturally integrate visual and force information when performing assembly tasks and adjust their actions according to environmental feedback, the present application integrates visual and force sensors on a robot and enables the robot to learn human impedance characteristics by imitating human interaction learning mechanisms, to fuse visual and force data in a feature space, and to eliminate the inherent differences in perception modes and data dimension heterogeneity of visual sensors and force sensors. By analyzing the dynamic relationship between the robot and the environment and constructing a mathematical model of robot-environment interaction, a feature space human-robot-environment interaction dynamics model is established by solving the visual servoing acceleration model in combination with the dynamics equation of the robot in the assembly process; a feature space human dynamics model is established as a planner to obtain a reference feature trajectory that implicitly identifies the dynamic contact of humans; in addition, a QP-based optimization method is introduced to design a feature space variable stiffness impedance controller based on QP online planning, which can dynamically adjust the impedance parameters during assembly and compensate for the disturbance force caused by environmental uncertainty in real time, maintain constant contact force at the end, and thus improve the stability and precision of the rich contact task, ultimately enabling the robot to complete the assembly task stably, intelligently, flexibly, and with high precision.

[0131] In one embodiment, the coordinate transformation relationship between the camera and the six-dimensional force sensor and the robot end is established in S100, including:

[0132] S110: A six-dimensional force sensor and an RGB-D camera are installed at the end of the mechanical arm to ensure that the field of view of the camera is not obstructed;

[0133] S120: An image of a calibration board with known geometric features (such as N points) is captured using the end camera, and the corresponding end pose of the mechanical arm at this time is recorded; the feature points on the calibration board should have a known geometric relationship so that the pose of the camera relative to the calibration board can be determined by image processing algorithms;

[0134] S130: To improve the accuracy and reliability of the calibration, S120 is repeated a predetermined number of times (12 times) at different positions and attitudes to capture images of the calibration board and record the pose data of the end of the robot in synchronization; this helps to reduce noise interference and improve the stability of the solution;

[0135] S140: By analyzing the captured calibration board images and recorded end pose data, the rotation matrix and the translation vector of the camera relative to the end of the mechanical arm are calculated, and the Tsai-Lenz method combined with the hand-eye calibration technique is used to solve the coordinate transformation relationship between the camera and the end of the robot, which is specifically:

[0136] ;

[0137] S150: Construct the assembly model of the flange, six-axis force sensor and camera using modeling software, and calculate the rotation matrix of the six-axis force sensor relative to the camera through geometric analysis and translation vector to obtain the coordinate transformation relationship between the six-axis force sensor and the camera , specifically:

[0138] .

[0139] In one embodiment, S200 includes:

[0140] S210: When a person applies force to the robot and makes it interact with the environment, the dynamics of the human-guided kinematic chain robot in joint space is described as:

[0141] ;

[0142] wherein , respectively, are the joint angle, joint velocity and joint acceleration of the robot manipulator, represents a symmetric and positive definite inertia matrix, describes the Coriolis force and centripetal force matrix, is the gravity vector, is the joint driving instruction vector, is the torque exerted by the human on the end effector, is the external torque;

[0143] S220: Solve the acceleration level of visual servoing dynamics to represent the motion relationship between visual features and joint variables, considering image features The kinematic relationship between the motion of the image feature and the associated motion screw is:

[0144] ;

[0145] wherein is the velocity of the image feature, is the image interaction matrix, which is used to associate the velocity of the feature point with the robot motion screw, and respectively represent the motion screw of the camera and the object, therefore, the subtraction of the two reflects the relative motion between the two;

[0146] S230: When the object is stationary, i.e. , the above formula can be used to represent the kinematic relationship between the feature velocity and the joint velocity through the Jacobian function of the robot, i.e.:

[0147] ;

[0148] where represents the task Jacobian matrix, is the robot Jacobian matrix, is the screw transformation matrix (in this formula, is a constant matrix because the camera is rigidly mounted on the end effector of the robot), in this way, the velocity of image features can be expressed as a function of joint velocity, thus providing a basis for subsequent control strategy design;

[0149] S240: Differentiate the equation in S230 with respect to time to obtain the acceleration form expression of the image feature:

[0150] ;

[0151] where, denotes the derivative of the task Jacobian matrix, which is the rate of change with respect to time;

[0152] In order to facilitate the concise derivation of subsequent expressions, the above formula is rewritten as:

[0153] ;

[0154] where, is the acceleration of the image feature, is the acceleration coupling matrix, denotes the derivative of the image interaction matrix, which is the rate of change with respect to time, denotes the derivative of the robot Jacobian matrix, which is the rate of change with respect to time;

[0155] S250: Rearrange the human-guided n-link robot dynamics equation in S210 to derive the dynamics characteristics of the human-guided robot in the feature space:

[0156] ;

[0157] Substitute the feature point velocity expression in S230 and the feature point acceleration expression in S240 into the dynamics characteristics formula to obtain the dynamic equation related to the feature motion:

[0158] ;

[0159] where is the comprehensive force term; the above formula is a set of nonlinear high-coupling differential equations corresponding to image features;

[0160] S260: Jacobian of the task is introduced , consider a projection matrix that can convert the human-applied torque in the human tool frame to the camera frame, so that , where, is the transpose of the kinematic Jacobian matrix of the human-to-tool end relative to the robot joints, substitute the above feature-related dynamic equation, to obtain:

[0161] ;

[0162] where, represents the virtual force acting on the image feature, the virtual force comes from the human guide torque projected into the camera coordinate system, is the virtual external force in the feature space, represent the virtual gravity, virtual Coriolis force and centripetal force in the feature space, represents the damping matrix, represents the feature space impedance controller, which is the virtual control input of the robot in the feature space, represents the human-applied torque expressed in the camera coordinate system.

[0163] In one embodiment, S300 includes:

[0164] S310: In the feature space, the virtual human guide force and acting on the image feature, for the convenience of analysis, the human guide force projected into the feature space is modeled as:

[0165] ;

[0166] where, respectively represent the velocity and acceleration variables of the human guide feature, represents the damping matrix;

[0167] Through the above formula, the visual feature variable and the robot end effector can move in the same direction with the human guide external force moment, at the same time, the human guide feature trajectory can be obtained, which serves as the reference trajectory required for subsequent controller design;

[0168] S320: Design a variable-stiffness visual impedance controller acting on the feature space based on S310, the stiffness is adjusted by the online optimized QP method to ensure that the robot end effector maintains a constant force when performing a contact-rich task; a feature space impedance controller is designed, which is in the form of:

[0169] ;

[0170] where, is the feature tracking error vector, is the rate of change of the feature tracking error, and denote the stiffness and damping matrices of the impedance control, respectively;

[0171] S330: Substitute the equations in S310 and S320 into S260 to get:

[0172] ;

[0173] Observe the above equation, further derivation is:

[0174] ;

[0175] where, is the second derivative of the rate of change of the feature tracking error, denoting the acceleration of the change of the feature tracking error;

[0176] Since the performance of the impedance system depends on its parameters, it is expected that the virtual external force in the feature space can converge to the required external force ;

[0177] S340: To solve the force convergence problem in S320, the following QP problem is constructed to adjust the stiffness of the impedance system online:

[0178] ;

[0179] ;

[0180] where, and denote the minimum and maximum allowable stiffness, respectively, denotes the maximum external torque that the robot can apply, and are the weighting matrices for adjusting the external torque and the minimum stiffness, respectively;

[0181] S350: After completing the above controller optimization, in order to facilitate the subsequent stability analysis, the energy function of the feature space impedance system is defined as:

[0182] ;

[0183] Its derivative with respect to time is:

[0184] ;

[0185] Since the term The symbols are not known in advance and it is not feasible to define a proper storage function for the relative port, thus guaranteeing the passivity of the impedance system with respect to the ports , where denotes the derivative of the variable stiffness , which is the rate of change of the variable stiffness.

[0186] In one embodiment, S400 comprises:

[0187] S410: To guarantee the stability of the feature space variable impedance system, first, the variable stiffness is rewritten as , where and denote the constant stiffness component and the time-varying stiffness component, respectively, without loss of generality, let ;

[0188] Subsequently, the concept of virtual energy tank is introduced to separate the potential active behavior of the time-varying stiffness component, whose energy is defined as , where denotes its dynamic state:

[0189] ;

[0190] , where denotes the rate of change of the energy tank dynamic state, is the energy flow switch parameter, is the energy exchange term;

[0191] , where is defined as:

[0192] ;

[0193] , where denotes the lower bound of the energy tank, is the spatiotemporal stiffness matrix, in addition, to ensure that the total energy of the entire interconnected system remains bounded, when a certain energy tank upper bound is reached, the following approach is used to guarantee the avoidance of energy tank overload:

[0194] ;

[0195] S420: Therefore, under the passivity constraint, the online optimization of the stiffness is carried out, and the energy tank constraint is incorporated into the optimization problem, which can be expressed as the following QP problem:

[0196] ;

[0197] ;

[0198] S430: The final feature space impedance controller is expressed as:

[0199] ;

[0200] The virtual external force expression in the feature space is modified as:

[0201] ;

[0202] By implementing the feature space variable stiffness visual impedance controller based on QP-based online stiffness optimization, as described in the above two equations, can be dynamically adjusted to maintain constant force in contact-intensive tasks, and the energy tank is initialized to satisfy ; when the energy of the virtual energy tank , the stiffness can be continuously optimized online, and conversely, when , the stiffness degenerates to , and the controller is simplified to a constant stiffness impedance controller;

[0203] S440: For the modified energy tank constraint controller and the virtual external force equation, i.e., the equation shown in S420, S430, the dynamic state of the energy tank, i.e., the equation shown in S410, the storage function of the entire interaction system is selected as:

[0204] ;

[0205] The differential of the total energy with respect to time is given by:

[0206] ;

[0207] where denotes the time derivative of the energy of the virtual energy tank;

[0208] Since , we have:

[0209] ;

[0210] Thus we have:

[0211] ;

[0212] The above equation shows that the following passivity condition exists:

[0213] ;

[0214] where is the storage function value at time t, is the stored function value at the initial time instant, is the integral variable, is the power exchange between the system and the external environment;

[0215] The passivity condition inequality above proves that the modified augmented control framework can achieve variable stiffness in the visual impedance model while safely interacting with any passive environment.

[0216] In one embodiment, S500 comprises:

[0217] S510: A six-axis force sensor is installed between the handle and the flange to measure the force and torque applied by the human in real time; a RealSense D435i camera is installed at the front end of the flange to capture image information of the gear and gear shaft; the end effector of the robot is connected to the six-axis force sensor through the flange to ensure the measurement accuracy and stability of the force sensor;

[0218] S520: During the experiment, the operator guides the robot to assemble the gear into the gear shaft, while the RGB-D camera captures the image information of the gear and gear shaft in real time and records the corresponding four sets of feature point trajectory data, which include the relative position and attitude information of the gear and gear shaft, for subsequent visual servo control. By recording these data, the operation skills and dynamic characteristics in the human-guided assembly process are analyzed to provide reference for the variable stiffness visual impedance control strategy;

[0219] S530: According to the feature point trajectory data recorded during the human-guided process, the parameters of the variable stiffness visual impedance controller are initialized, including the virtual stiffness matrix , the virtual damping matrix , and the desired contact force ;

[0220] S540: Feature point extraction and tracking The RealSense D435i camera collects images of the gear and gear shaft in real time at a frequency of 30Hz, and uses the AprilTag algorithm to identify four key feature points of the gear and gear shaft from the images. The position information of these feature points will be used as the input of the visual servo controller; the error between the current feature point position and the human hand guide trajectory is calculated , where is the current feature point position, is the human hand guide trajectory; by monitoring the error of the feature points in real time, the controller dynamically adjusts the motion trajectory of the robot to ensure that the feature points can closely follow the trajectory of the human hand guide;

[0221] S550: Variable stiffness visual impedance control according to the feature point error and the stiffness strategy based on QP optimization, dynamically adjusting the virtual stiffness matrix , the virtual damping matrix , the stiffness strategy is:

[0222] ;

[0223] ;

[0224] Through the stiffness parameter, the controller can respond to the contact interference in the assembly process in real time, ensure that the robot shows compliance in the contact process, and avoid damage to the gear and gear shaft;

[0225] S560: According to the updated impedance parameter and the feature point error information, the control signal is calculated:

[0226] ;

[0227] Wherein, the control signal is used to guide the motion of the robot end effector, so that the feature point can closely follow the teaching trajectory guided by the human hand;

[0228] S570: Convert the controller in the feature space Into the control signal in the joint space:

[0229] ;

[0230] Wherein is the pseudo-inverse of the Jacobian matrix and the inverse inertia matrix, by converting the control signal from the feature space to the joint space, the robot can adjust the motion of the end effector according to the calculated joint space control signal;

[0231] S580: The robot adjusts the motion of the end effector according to the calculated joint space control signal , so that the feature point can closely follow the teaching trajectory guided by the human hand.

[0232] In the assembly process, the controller continuously adjusts the motion trajectory of the robot end to ensure that the feature point can closely follow the teaching trajectory guided by the human hand. Through adaptive impedance control, the robot can automatically adjust the force and position in the contact process, realize stable and accurate assembly. The human-guided demonstration ) method has significant advantages in feature point tracking error, assembly accuracy and stability compared with the classical visual servo control ), inductance learning control ), can realize high-precision and stable assembly task.

[0233] Compared with the prior art, the advantages of the present application are:

[0234] (1) The present application provides a unified visual-force framework that integrates visual and force information in feature space, effectively addressing the data heterogeneity problem between visual information and six-dimensional force information. In traditional methods, visual and force information are difficult to directly integrate and process due to differences in data type and source. However, the present application takes full advantage of the high precision of visual information and the high sensitivity of force information by integrating them in feature space, achieving their complementarity. This integration not only improves the adaptability and robustness of the robot in complex tasks, but also enables the robot to complete tasks more accurately. For example, in assembly tasks, visual information can help the robot quickly locate the target, while force information can provide fine adjustments during contact to ensure the precision and quality of assembly.

[0235] (2) The present application considers the influence of the environment and introduces a human-guided learning control strategy to adaptively counteract the identified environmental disturbance dynamics. In actual work tasks, there is a high degree of nonlinear coupling between the human-robot-environment, especially in tasks that require fine manipulation, such as assembly, drilling, etc. These tasks involve contact dynamics, i.e., the contact state between the robot and the environment changes constantly as the operation progresses. By introducing human-guided learning control, the present application can learn the operation skills of human operators in complex tasks and adaptively adjust control parameters to complete high-quality assembly tasks. This strategy not only improves the flexibility and adaptability of the robot, but also exhibits similar flexibility and adaptability to humans in actual operations, ensuring the smooth progress of the task.

[0236] (3) The present application significantly improves the robustness and stability of the robot system in complex environments through a variable stiffness visual impedance control strategy based on QP optimization. In actual operations, the contact dynamics between the robot and the environment are complex and variable. Traditional rigid control methods are prone to cause excessive contact force when faced with such complex environments, resulting in damage to the robot or the object being manipulated. However, the present application uses variable stiffness impedance control to dynamically adjust the stiffness and damping characteristics of the robot according to real-time contact states, enabling the robot to exhibit compliance during contact, avoiding system instability or task failure due to excessive contact force.

[0237] In one embodiment, an optimization-based robot variable stiffness visual-impedance control system includes a Sawyer robot and an end effector, which contains a flange, a six-axis force sensor, and a depth camera. The flange is used to integrate the six-axis force sensor and the camera at the end of the robot, the six-axis force sensor is used to sense the force applied by the human, and the depth camera is used to detect feature points. The Sawyer robot is used to perform the steps of an optimization-based robot variable stiffness visual-impedance control method.

[0238] The specific definition of the robot variable stiffness vision-impedance control system based on optimization can refer to the definition of the robot variable stiffness vision-impedance control method based on optimization, which will not be repeated here. Each module in the robot variable stiffness vision-impedance control system based on optimization can be realized by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to call and execute the operation corresponding to each module by the processor.

[0239] A computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the robot variable stiffness vision-impedance control method based on optimization when executing the computer program.

[0240] A computer readable storage medium stores a computer program, and the computer program implements the steps of the robot variable stiffness vision-impedance control method based on optimization when executed by a processor.

[0241] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0242] The above describes in detail the robot variable stiffness vision-impedance control method and system based on optimization provided by the present application. The principles and implementation manners of the present application are described by using specific examples in this paper, and the above description of the examples is only used to help understand the core idea of the present application. It should be pointed out that, for those skilled in the art, some improvements and modifications can be made to the present application without departing from the principles of the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. An optimization-based visual-impedance control method for variable stiffness of a robot, characterized in that, The method includes the following steps: S100: Establish coordinate transformation relationships between the camera, the six-dimensional force sensor, and the robot end effector. Simultaneously, calibrate the six-dimensional force sensor and compensate for the sensor's own zero drift and tool gravity through a force calibration algorithm. S200: Considering the dynamic characteristics of the robot under force, including the guiding force applied by the human and the reaction force of the environment, and combining the robot's kinematics and dynamic equations, as well as the visual servo acceleration equation, a complete human-robot-environment interaction dynamic model in the feature space is constructed. S300: Based on the dynamic model, a human guidance force model is established in the feature space. Visual features are associated with robot motion and a reference trajectory is generated. A variable stiffness visual impedance controller is designed. The stiffness matrix is ​​optimized online through QP to ensure constant contact force and satisfy torque constraints. The system equations are derived to achieve external force convergence. An energy function is constructed to analyze stability. S400: Based on the stability analysis results, online optimization based on energy tank stiffness enhancement is introduced. The energy tank state equation and upper and lower limit constraints are constructed to limit active energy injection. A quadratic programming QP model is established to optimize stiffness parameters in real time under passive constraints and ensure that the total energy is bounded. Passive analysis is performed by constructing a system storage function. S500: The robot assembles gears using a human-guided robot. It integrates six-dimensional force sensing and RealSense visual data, extracts feature point trajectory errors in real time based on the AprilTag algorithm, drives QP optimization to dynamically adjust variable stiffness parameters, and combines Jacobi pseudo-inverse mapping to the joint space to drive the robot arm movement, achieving compliant trajectory tracking.

2. The method according to claim 1, characterized in that, The S100 establishes coordinate transformation relationships between the camera, the six-dimensional force sensor, and the robot's end effector, including: S110: A six-dimensional force sensor and an RGB-D camera are installed at the end of the robotic arm to ensure that the camera's field of view is unobstructed; S120: Use the end-effector camera to capture an image of a calibration plate with known geometric features and record the corresponding end-effector pose at that time; S130: Repeat the S120 preset number of times under different positions and postures, take images of the calibration board, and simultaneously record the pose data of the robot end effector; S140: By analyzing the captured calibration board images and recorded end-effector pose data, the rotation matrix of the camera relative to the robotic arm end effector is calculated. Translation vector By using the Tsai-Lenz method combined with hand-eye calibration technology, the coordinate transformation relationship between the camera and the robotic arm end effector was solved. Specifically: ； S150: Using modeling software, an assembly model of the flange, six-dimensional force sensor, and camera is constructed. The rotation matrix of the six-dimensional force sensor relative to the camera is calculated through geometric analysis. Translation vector The coordinate transformation relationship between the six-dimensional force sensor and the camera was obtained. Specifically: 。 3. The method according to claim 2, characterized in that, S200 includes: S210: When a human applies forces to a robot and causes it to interact with its environment, in the joint space, the human-guided... The dynamic characteristics of the linkage robot are described as follows: ； in, These are the generalized variables of the robot's joint angles, joint velocities, and joint accelerations. This represents a symmetric and positive definite inertia matrix. The Coriolis force and centripetal force matrix are described. It is a gravity vector. It is a joint drive command vector. It is the torque applied by the human to the end effector. It is an external torque; S220: Solve for the visual servo dynamics at the acceleration level to characterize the motion relationship between visual features and joint variables, considering... Image features Motion of image features and associated motion spinors The kinematic relationship is as follows: ； in, It is the speed of image feature generation. It is an image interaction matrix used to correlate the velocity of feature points with the robot's motion spinor. and These represent the motion spins of the camera and the object, respectively. S230: When the object is stationary... In this case, the above equation can be used to characterize the kinematic relationship between the characteristic velocity and the joint velocity through the robot's Jacobian function, that is: ； in The Jacobian matrix represents the task. It is the Jacobian matrix of robots. It is a screw transformation matrix. In this way, the velocity of image features can be expressed as a function of joint velocity, thus providing a basis for subsequent control strategy design. S240: Differentiating the equation in S230 with respect to time yields an accelerated form of the image features: ； in, This represents the derivative of the task Jacobian matrix, which is... Rate of change over time; To facilitate the concise derivation of subsequent expressions, the above expression is rewritten as: ； in, It is the acceleration of image features. Here is the acceleration coupling matrix. This represents the derivative with respect to the image interaction matrix, which is... rate of change over time This represents the derivative of the robot's Jacobian matrix, which is... Rate of change over time; S250: The dynamic equations of the human-guided n-link robot in S210 are rearranged to derive the dynamic characteristics of the human-guided robot in the feature space: ； Substituting the velocity expressions for the characteristic points in S230 and the acceleration expressions for the characteristic points in S240 into the dynamic characteristic formula, we obtain the dynamic equations related to the characteristic motion: ； in This is a combined force term; the above formula is a set of terms related to... Nonlinear, highly coupled differential equations corresponding to each image feature; S260: By introducing the task Jacob coefficient Consider a projection matrix that represents the torque applied by the human in the human-tool coordinate system. Transform to the camera coordinate system, therefore we have , ,in, Substituting the transpose of the kinematic Jacobian matrix of the human-guided tool end relative to the robot joint into the above feature-related dynamic equations, we obtain: ； in, This represents a virtual force acting on image features, originating from a human-guided torque projected onto the camera coordinate system. It is a virtual external force in the characteristic space. Represents virtual gravity, virtual Coriolis force, and centripetal force in characteristic space. Represents the damping matrix. This represents the characteristic space impedance controller, which is the virtual control input for the robot in the characteristic space. This represents the torque applied by the human in the camera coordinate system.

4. The method according to claim 3, characterized in that, The S300 includes: S310: Virtual human guidance force in feature space And acting on image features, for ease of analysis, the human guiding force projected onto the feature space is modeled as: ； in, These represent the velocity and acceleration variables of human guiding characteristics, respectively. Represents the damping matrix; Based on the above equation, the visual feature variables and the robot's end effector can move in the same direction as the external torque guided by the human, and at the same time, the characteristic trajectory guided by the human can be obtained. This trajectory serves as a reference trajectory for subsequent controller design; S320: A variable-stiffness visual impedance controller based on the S310 is designed and operates in the feature space. The stiffness is adjusted using an online optimized QP method to ensure that the robot end effector maintains a constant force when performing contact-intensive tasks. A feature space impedance controller is designed. Its form is: ； in, It is the feature tracking error vector. The rate of change of feature tracking error. and These represent the stiffness and damping matrices for impedance control, respectively. S330: Substituting the formulas in S310 and S320 into S260, we get: ； Observing the above formula, we can further deduce: ； in, It is the derivative of the rate of change of the feature tracking error, which represents the acceleration of the change of the feature tracking error; Since the performance of an impedance system depends on its parameters, it is desirable that the virtual external force in the characteristic space converges to the desired external force. ; S340: To solve the force convergence problem in S320, the following QP problem is constructed for adjusting the stiffness of the impedance system online: ； ； in, and These represent the minimum and maximum allowable stiffness, respectively. This indicates the maximum external torque that the robot can apply. and These are weighted matrices used to adjust the external torque and minimum stiffness, respectively. S350: After completing the above controller optimization, to facilitate subsequent stability analysis, the energy function of the characteristic space impedance system is defined as follows: ； Its derivative with respect to time is: ； Due to the item The sign of the impedance system is unknown beforehand, and it is not feasible to define a suitable energy storage function for the relative ports. Therefore, it is impossible to guarantee the impedance system with respect to the ports. The passivity, in which, Indicates the variable stiffness Differentiate the value to find the rate of change of the variable stiffness.

5. The method according to claim 4, characterized in that, The S400 includes: S410: To ensure the stability of the characteristic space variable impedance system, firstly, the variable stiffness... Rewritten as ,in and Let the constant stiffness component and the time-varying stiffness component be respectively. Without loss of generality, let ; Subsequently, the concept of a virtual energy tank is introduced to separate the potential active behavior of time-varying stiffness components, whose energy is defined as... ,in Indicate its dynamic state: ； in, This represents the rate of change of the dynamic state of the energy tank. For energy flow switching parameters, It is an energy exchange term; in Defined as: ； in This indicates the lower limit of the energy tank. For the spacetime stiffness matrix, in addition, to ensure that the total energy of the entire interconnected system remains bounded, when a certain energy tank upper limit is reached... To prevent the energy tank from overloading, the following methods should be used: ； S420: Therefore, online stiffness optimization under passive constraints, incorporating the energy tank constraint into the optimization problem, can be expressed as the following QP problem: ； ； S430: The final characteristic space impedance controller is described as follows: ； The expression for the virtual external force in the characteristic space is modified as follows: ； By implementing a QP-based online stiffness optimization feature-space variable stiffness visual impedance controller, as described in the two equations above, It can be dynamically adjusted to maintain a constant force in contact-intensive tasks, and the energy tank initializes to meet the following conditions. When the virtual energy tank's energy When the stiffness is constant, it can be continuously optimized online; conversely, when... At that time, the stiffness degenerates into The controller is simplified to a constant stiffness impedance controller; S440: For the modified energy tank constraint controller and virtual external force equations, i.e., equations in S420 and S430, the energy tank dynamic state is shown in equation S410, and the storage function of the entire interconnected system. Selected as: ； Total Energy Differential with respect to time It is given by the following formula: ； in, This represents the energy in the virtual energy tank. Find the time derivative; because Therefore: ； Therefore, we can conclude that: ； The above equation indicates the existence of the following passive condition: ； in, The value of the storage function at time t. The stored function value at the initial moment. For integration variables, For power exchange between the system and the external environment; The above passive conditional inequalities prove that the modified enhanced control framework can achieve variable stiffness in the visual impedance model while safely interacting with any passive environment.

6. The method according to claim 5, characterized in that, The S500 includes: S510: A six-dimensional force sensor is installed between the handle and the flange to measure the force and torque applied by the user in real time; a RealSense D435i camera is installed at the front of the flange to capture image information of the gears and gear shafts; the end effector of the robot is connected to the six-dimensional force sensor through the flange to ensure the measurement accuracy and stability of the force sensor. S520: During the experiment, the operator guides the robot to assemble the gear into the gear shaft. At the same time, the RGB-D camera captures the image information of the gear and gear shaft in real time and records the corresponding four sets of feature point trajectory data. These feature point trajectory data include the relative position and attitude information of the gear and gear shaft, which are used for subsequent visual servo control. By recording these data, the operator's skills and dynamic characteristics in the human-guided assembly process are analyzed, providing a reference for the variable stiffness visual impedance control strategy. S530: Based on the feature point trajectory data recorded during human guidance, initialize the parameters of the variable stiffness visual impedance controller. These parameters include the virtual stiffness matrix. Virtual damping matrix Expected contact force ; S540: Feature Point Extraction and Tracking. The RealSense D435i camera acquires images of gears and gear shafts in real-time at a frequency of 30Hz. The AprilTag algorithm is used to identify four key feature points from the images of the gears and gear shafts. The positional information of these feature points will be used as input to the visual servo controller. The error between the current feature point position and the trajectory of the hand-operated indicator is calculated. ,in This is the current location of the feature point. It is a hand-guided display trajectory; by monitoring the error of the feature points in real time, the controller dynamically adjusts the robot's motion trajectory to ensure that the feature points can closely follow the hand-guided trajectory; S550: Variable stiffness visual impedance control based on feature point error And a variable stiffness strategy based on QP optimization to dynamically adjust the virtual stiffness matrix Virtual damping matrix The variable stiffness strategy is as follows: ； ； By using variable stiffness parameters, the controller can respond to contact interference that occurs during the assembly process in real time, ensuring that the robot exhibits compliance during contact and avoiding damage to gears and gear shafts. S560: Calculate the control signal based on the updated impedance parameters and characteristic point error information: ； Among them, control signals Used to guide the movement of the robot's end effector, enabling feature points to closely follow the hand-guided teaching trajectory; S570: Controller in feature space Converted into control signals for joint space: ； in It is the pseudo-inverse of the Jacobian matrix and the inverse inertial matrix. By converting the control signal from the feature space to the joint space, the robot can adjust the motion of the end effector according to the calculated joint space control signal. S580: The robot controls the joint space based on the calculated signals. Adjust the movement of the end effector so that the feature points can closely follow the reading trajectory guided by the hand.

7. An optimized robot variable stiffness vision-impedance control system, characterized in that, The device includes a Sawyer robot and an end effector. The end effector comprises a flange, a six-dimensional force sensor, and a depth camera. The flange is used to integrate the six-dimensional force sensor and the camera at the end of the robot. The six-dimensional force sensor is used to sense the force applied by a person, and the depth camera is used to detect feature points. The Sawyer robot is used to perform the steps of the method as described in any one of claims 1 to 6.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images