A visual-haptic guided exoskeleton robot force / position hybrid control method and system

Abstract

The application discloses a kind of visual haptic guidance's outer limb robot force-position hybrid control method and system, it is related to robot force-position hybrid control technical field.The method includes the following steps: from visual angle, the wrist posture information of human arm in the process of gripping different objects, the posture information of five fingers and the generated motion trajectory are acquired;Human wrist motion trajectory is learned using outer limb robot arm, and human five-finger gripping motion trajectory is learned using outer limb robot hand;The fingertip force information of outer limb robot when gripping different objects is acquired as finger force feedback;A force-position hybrid controller is established, force-position control is carried out using the force-position hybrid controller, and the posture of outer limb robot hand is adjusted according to finger force feedback, to obtain the optimal motion trajectory.The application improves the gripping accuracy of outer limb, and provides a good suggestion for future outer limb force-position hybrid control.

Description

Technical Field

[0001] This invention relates to the field of force-position hybrid control technology for robots, and in particular to a method and system for force-position hybrid control of an extremity robot guided by visual and tactile perception. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Cerebrovascular diseases such as stroke and cerebral infarction, spinal vascular diseases, encephalitis, brain abscess, peripheral neuritis, and trauma can all cause limb paralysis, resulting in the loss of some or all of the upper limb's motor function. This severely impacts the patient's ability to live independently and significantly reduces their quality of life. Therefore, providing assistive technologies and equipment for patients with upper limb motor function loss is a shared aspiration of society, families, and patients.

[0004] Exolimb robots are highly intelligent devices that can provide effective motor function assistance to people with movement disorders. They can provide grasping assistance for hemiplegic patients, enabling them to complete basic daily tasks, and are of great significance in assisting the elderly and disabled. However, many technical challenges remain to be addressed. For example, exolimb robots lack effective sensory guidance, resulting in limited spatial positioning capabilities; their high degrees of freedom and difficulty in control make it challenging to achieve simultaneous and accurate control of posture and force. Therefore, improving the perception and control capabilities of exolimb robots has become a technological bottleneck in this field.

[0005] Vision and touch are excellent channels for robots to interact with their environment. Robots can perceive the external environment through vision, including the position, shape, size, and color of objects, and through touch, they can perceive physical properties such as weight and temperature. However, for some complex grasping tasks, a single-modal perception system is insufficient; various sensors are needed to acquire information from different modalities. Therefore, how to achieve hybrid control optimization of force and position in extremity robots based on a multimodal perception system has become a pressing problem to be solved in current technology. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the present invention aims to provide a visual-tactile guided force-position hybrid control method and system for extremity robots. This control method uses tactile and visual information as guidance to establish a force-position hybrid controller, and optimizes the controller's parameters through a reinforcement learning mechanism. Compared with traditional force-position hybrid controllers, this controller can learn directly from experience by observing and interacting with the environment, making the most environmentally responsive control strategy, thereby independently solving operational tasks.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0008] The first aspect of this invention provides a method for force-position hybrid control of an extremity robot guided by visual and tactile perception, comprising the following steps:

[0009] From a visual perspective, obtain wrist posture information, finger posture information, and generated motion trajectory information of the human arm during the process of grasping different objects;

[0010] The exolimb robot arm learns the motion trajectory of the human wrist, the exolimb robot hand learns the grasping motion trajectory of the human five fingers, and the error between the grasping motion trajectory of the human five fingers and the motion trajectory of the exolimb robot hand is obtained.

[0011] Acquire fingertip force information when the exolimb robot grasps different objects as finger force feedback;

[0012] A force-position hybrid controller is established based on reinforcement learning and proportional-integral-differential (PID). The force-position hybrid controller is used to perform force-position control based on the error between the human five-finger grasping motion trajectory and the motion trajectory of the extremity robot hand. Based on the finger force feedback, the posture of the extremity robot hand is adjusted to obtain the optimal motion trajectory.

[0013] The robot continues to learn and adjust using the optimal motion trajectory until it can achieve stable grasping and meets the preset grasping success rate of the exolimb robot.

[0014] Furthermore, the exolimb robotic arm learns the movement trajectory of the human wrist through a dynamic motion primitive algorithm.

[0015] Furthermore, the exolimb robot hand learns the human five-finger grasping motion trajectory through a trainable time warp algorithm and a Gaussian mixture regression algorithm based on expectation maximization.

[0016] Furthermore, the specific steps for processing the human five-finger grasping motion trajectory using the Gaussian mixture regression algorithm are as follows: First, similar kinematic information and motion characteristics are obtained through a trainable time warp algorithm; then, the Gaussian mixture regression algorithm based on expectation maximization is used for processing, thereby fitting all trajectories into one.

[0017] Furthermore, the force-position hybrid controller takes the error between the human five-finger grasping motion trajectory and the motion trajectory of the exolimb robot hand as input, and takes the finger force of the exolimb robot hand and the motion speed of the end effector as feedback. It uses reinforcement learning to iterate the controller's control parameters based on the feedback data, thereby enabling the robot to make the optimal motion plan according to the environment.

[0018] Furthermore, based on the optimal motion trajectory generated by the force-position hybrid controller, the joint angle transformation trajectory is obtained through inverse kinematics calculation, and then the joint angle transformation trajectory is transmitted to the hand of the extremity robot, so that the extremity robot can complete the grasping according to the joint angle transformation trajectory.

[0019] A second aspect of the present invention provides a visual-tactile guided force-position hybrid control system for an extremity robot, comprising:

[0020] The visual information acquisition module is configured to acquire wrist posture information, finger posture information, and generated motion trajectory of the human arm during the process of grasping different objects from a visual perspective.

[0021] The machine learning module is configured to learn the motion trajectory of the human wrist using the exolimb robot arm, and the exolimb robot hand learns the grasping motion trajectory of the human five fingers, and obtain the error between the grasping motion trajectory of the human five fingers and the motion trajectory of the exolimb robot hand.

[0022] The finger force information acquisition module is configured to acquire fingertip force information of the exolimb robot when grasping different objects as finger force feedback.

[0023] The control module is configured to establish a force-position hybrid controller based on reinforcement learning and proportional-integral-derivative (PID). The force-position hybrid controller performs force-position control based on the error between the human five-finger grasping motion trajectory and the motion trajectory of the extremity robot hand. Based on the finger force feedback, the posture of the extremity robot hand is adjusted to obtain the optimal motion trajectory.

[0024] The effect verification module is configured to continue learning and adjusting using the optimal motion trajectory until the exolimb robot can achieve stable grasping and meets the preset grasping success rate of the exolimb robot.

[0025] Furthermore, in the finger force information acquisition module, pressure film sensors are attached to the five fingers of the exolimb robot's hand to acquire fingertip force information as finger force feedback when the exolimb robot grasps different objects.

[0026] Furthermore, the finger pressure information acquisition module also includes a microcontroller. When the exophoric robot hand performs a grasping action, the pressure film sensor transmits the fingertip pressure data to the microcontroller in real time, and then the microcontroller transmits it to the host computer through serial communication.

[0027] Furthermore, the force-position hybrid controller includes a proportional-derivative controller and a proportional-integral force controller. The proportional-derivative controller is used to generate a partial motion trajectory of the extremity robot based on the error between the human five-finger grasping motion trajectory and the extremity robot hand motion trajectory. The proportional-integral force controller is used to change the motion trajectory based on finger force feedback to reduce the error.

[0028] The above one or more technical solutions have the following beneficial effects:

[0029] This invention discloses a visual-tactile guided force-position hybrid control method and system for extremity robots. By having a healthy person grasp different objects in space and move them to different positions, the movement information of the person's arm and five fingers is recorded. Then, through skill learning, the human hand movements are transferred to the extremity robot. For multi-degree-of-freedom extremity-assisted grasping tasks, a visual-tactile guided force-position hybrid controller is proposed. The optimal parameters of this controller are obtained through reinforcement learning, improving the precise grasping control capability of the extremity robot. This invention can provide new technical support for people with limb paralysis, improving their self-care ability in daily life, and has high application value.

[0030] To improve the accuracy of extremity grasping objects, this invention incorporates force feedback, enabling hybrid force-position control of the extremities. Furthermore, it continuously adjusts controller parameters through reinforcement learning to reduce the error between the extremity's motion trajectory and the human arm's motion trajectory. This invention improves the grasping accuracy of the extremities and provides valuable suggestions for future hybrid force-position control of extremities.

[0031] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0032] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0033] Figure 1 This is a schematic diagram of the external limb space demonstration process in Embodiment 2 of the present invention;

[0034] Figure 2 This is a schematic diagram of the environmental setup for the human arm and external limbs in Embodiment 2 of the present invention;

[0035] Figure 3 This is a schematic diagram of the pasting of reflective marking points on a human hand and a schematic diagram of the angle of the metacarpophalangeal joints in Embodiment 2 of the present invention;

[0036] Figure 4 This is a schematic diagram of the hardware components of the external limb in Embodiment 2 of the present invention;

[0037] Figure 5 This is a schematic diagram of the force-position hybrid control strategy based on reinforcement learning in Embodiment 2 of the present invention;

[0038] Figure 6 This is a schematic diagram of the human arm data acquisition process in Embodiment 2 of the present invention;

[0039] Among them, 1. 7-DOF robotic arm, 2. underactuated prosthetic hand, 3. pressure membrane sensor, 4. motion capture system, 5. frustum, 6. cylinder, 7. sphere, 8. reflective marker point. Detailed Implementation

[0040] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0041] It should be noted that the embodiments of the present invention involve information related to the human body's grasping process. When the above embodiments of the present invention are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0042] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0043] Example 1:

[0044] Embodiment 1 of the present invention provides a force-position hybrid control method for an extremity robot guided by visual and tactile perception, comprising the following steps:

[0045] S1: Obtain wrist posture information, finger posture information, and generated motion trajectory from a visual perspective during the process of human arm grasping different objects;

[0046] S2: The exolimb robot arm learns the motion trajectory of the human wrist, the exolimb robot hand learns the grasping motion trajectory of the human five fingers, and the error between the grasping motion trajectory of the human five fingers and the motion trajectory of the exolimb robot hand is obtained.

[0047] S3: Acquire fingertip force information when the exolimb robot grasps different objects as finger force feedback, i.e., tactile information;

[0048] S4: A force-position hybrid controller is established based on reinforcement learning and proportional-integral-derivative (PID). The force-position hybrid controller is used to perform force-position control based on the error between the human five-finger grasping motion trajectory and the motion trajectory of the extremity robot hand. Based on the finger force feedback, the posture of the extremity robot hand is adjusted to obtain the optimal motion trajectory.

[0049] S5: Continue learning and adjusting using the optimal motion trajectory until the exolimb robot can achieve stable grasping and meets the preset grasping success rate of the exolimb robot.

[0050] In S1, the spatial motion information of the human arm when grasping an object is obtained from a visual perspective through a motion capture system, specifically including:

[0051] The spatial motion trajectory of the human arm and wrist is used for trajectory learning of the spatial position of the end effector of the extremity robot arm.

[0052] The trajectory of the change in the bending angle of the metacarpophalangeal joints when the five fingers grasp an object. This trajectory is used as a reference for learning the bending angle of the five fingers when the hand of an exolimb robot grasps an object.

[0053] In step 2, the exolimb robot arm learns the motion trajectory of the human wrist through dynamic motion primitives, and the exolimb robot hand learns the human five-finger grasping motion trajectory through a trainable time warp algorithm and a Gaussian mixture regression algorithm based on expectation maximization. The error between the human five-finger grasping motion trajectory and the exolimb robot hand motion trajectory is obtained through root mean square error.

[0054] In this embodiment, when collecting data using a motion capture system, reflective markers are attached to the subject's wrist and finger joints. The motion data of the wrist markers is transferred to the exolimb robot arm, and the angle change trajectory of the metacarpophalangeal joints is transferred to the exolimb robot hand. The formula for calculating the metacarpophalangeal joint angle α is as follows:

[0055]

[0056] Among them, a, b, and c are as follows Figure 3 In this diagram, point a is a marker on the human wrist, point b is a marker at a metacarpophalangeal joint of the hand, and point c is a marker at the interphalangeal joint of the finger corresponding to point b. The motion trajectory of the human arm and wrist first undergoes coordinate transformation to align with the default spatial coordinate system of the extremity robot arm. Then, it is processed using a dynamic motion primitive algorithm, where the X, Y, and Z coordinates of the trajectory are substituted into the algorithm for processing. During processing, the parameters in the dynamic motion primitive are set as follows: α / β = 4, and the time gain term s is set to 1. The initial system of this algorithm can be represented as:

[0057]

[0058] Where y represents the system state, g represents the objective function, α and β are the system gain terms, x represents the introduced system (i.e., a typical dynamic system), and f represents the time-varying forced function. The motion trajectory of the human wrist is processed by a dynamic motion primitive algorithm to filter out the jittering part in the human arm motion trajectory, preventing unstable phenomena such as jittering, pauses, and jamming from occurring during the process of the extremity robot arm reproducing the human hand motion trajectory.

[0059] The human five-finger grasping motion trajectory is processed by Gaussian mixture regression algorithm to filter out some erroneous data in the process of human hand grasping objects. While obtaining the most characteristic data in the entire process trajectory, the smoothness of the trajectory is guaranteed, so that the exolimb robot hand can replicate the grasping motion of human hand well and stably.

[0060] In this embodiment, the specific steps for processing the human five-finger grasping motion trajectory using the Gaussian mixture regression algorithm are as follows:

[0061] First, a trainable time warp algorithm is used to obtain similar kinematic information and motion features. Specifically, the trainable time warp algorithm is first used to process multiple trajectories of a human hand grasping the same object, obtaining relatively similar kinematic information and motion features. The formula for the trainable time warp algorithm is as follows:

[0062]

[0063]

[0064]

[0065] in, It is a set of optimized time warp functions. Synchronization signal The within-group mean square error, where T is the synchronization signal. The length of N is the synchronization signal. Quantity, Synchronization signal The t-th sample, r[t] is the t-th sample of the average signal.

[0066] Then, a Gaussian mixture regression algorithm based on expectation maximization is used to process the data, thereby fitting all the trajectories into a single curve. Specifically, the Gaussian mixture regression algorithm based on expectation maximization is used to fit 15 sets of trajectories into a single curve, which is then used for learning the hand of the extremity robot. Given a dataset consisting of M data points of dimension T, where t is the time step and s is the joint angle, the Gaussian mixture model formula is as follows:

[0067] p(k) = π k (6)

[0068]

[0069] Among them, {π k ,μ k ,Σ k} are the parameters defining the weights, average, and covariance matrix of the k-th Gaussian, respectively, where Σ is a D×D dimensional matrix and ξ is a dimensional Gaussian matrix. j It is a point in the dataset.

[0070] Then, regression calculations are performed based on the Gaussian mixture, using the following formula:

[0071]

[0072] Among them, h k Let h be the weight of the k-th mixture component, satisfying 0 ≤ h k ≤1 and ξ O ξ is the target variable. I It is the input feature vector.

[0073] The human hand data, processed by the algorithm, is transferred to the extremity robot, enabling the extremity to initially reach the designated position.

[0074] In S3, pressure film sensors are attached to the five fingers of the exoskeleton robot's hand to detect the finger force when the robot grasps an object, providing feedback to adjust the robot's hand posture. In this invention, the pressure film sensors are connected to a microcontroller and record the finger force information of the five fingers of the exoskeleton robot's hand in real time.

[0075] In S4, the force-position hybrid controller takes the error between the human five-finger grasping motion trajectory and the motion trajectory of the extremity robot hand as input, and takes the finger force of the extremity robot hand and the motion speed of the end effector as feedback. It uses reinforcement learning to iterate the controller's control parameters based on the feedback data, so that the robot can make the optimal motion plan according to the environment.

[0076] In S5, the joint angle transformation trajectory is obtained by inverse kinematics calculation based on the optimal motion trajectory generated by the force-position hybrid controller. Then, the joint angle transformation trajectory is transmitted to the hand of the extremity robot so that the extremity robot can complete the grasping according to the joint angle transformation trajectory.

[0077] Repeat steps S2-S4 until the error between the human hand's movement trajectory and the exolimb robot's hand movement trajectory approaches zero, at which point the exolimb robot can achieve stable grasping. In this embodiment, a stable grasp is defined as the process by which the exolimb robot grasps the grasping object, moves it to a designated position, and returns it. The grasping object must not slip or fall during this process. The root mean square error is used to calculate the success rate of the exolimb's grasping, and the formula is as follows:

[0078]

[0079] Example 2:

[0080] Embodiment 2 of the present invention provides a visual-tactile guided force-position hybrid control system for an extremity robot, such as... Figure 1 As shown, it includes:

[0081] The visual information acquisition module is configured to acquire wrist posture information, finger posture information, and generated motion trajectory of the human arm during the process of grasping different objects from a visual perspective.

[0082] The machine learning module is configured to learn the motion trajectory of the human wrist using the exolimb robot arm, and the exolimb robot hand learns the grasping motion trajectory of the human five fingers, and obtain the error between the grasping motion trajectory of the human five fingers and the motion trajectory of the exolimb robot hand.

[0083] The finger force information acquisition module is configured to acquire fingertip force information of the exolimb robot when grasping different objects as finger force feedback.

[0084] The control module is configured to establish a force-position hybrid controller based on reinforcement learning and proportional-integral-derivative (PID). The force-position hybrid controller performs force-position control based on the error between the human five-finger grasping motion trajectory and the motion trajectory of the extremity robot hand. Based on the finger force feedback, the posture of the extremity robot hand is adjusted to obtain the optimal motion trajectory.

[0085] The effect verification module is configured to continue learning and adjusting using the optimal motion trajectory until the exolimb robot can achieve stable grasping and meets the preset grasping success rate of the exolimb robot.

[0086] In this embodiment, the visual-tactile guided force-position hybrid control system for the extremity robot is used to control the extremity robot. The extremity robot includes a multi-degree-of-freedom underactuated prosthetic hand as the extremity robot hand and a 7-degree-of-freedom robotic arm as the extremity robot arm. Pressure film sensors are attached to the five fingers of the extremity robot hand.

[0087] This embodiment designs a device for recording the movement trajectory of a human hand, such as... Figure 2As shown, the system includes a 7-DOF robotic arm 1, an underactuated prosthetic hand 2, a pressure film sensor 3, a motion capture system 4, grasping props, and reflective markers 8. The 7-DOF robotic arm 1 and the underactuated prosthetic hand 2 are mounted on the extremity robot. The 7-DOF robotic arm 1 is used to bring the extremity robot's hand to a specific position, and the underactuated prosthetic hand 2 is used to grasp objects. The pressure film sensor 3 records the finger force information when the extremity robot's hand grasps an object. The motion capture system 4 is a motion capture system used to collect the movement trajectory of the human hand, and it has eight cameras. The grasping props include objects such as a frustum 5, a cylinder 6, and a ball 7. The reflective markers 8 are affixed to the human hand. The testing process is as follows. Figure 6 As shown, after the subject is in position, pressure sensors are attached to their five fingers to start detecting the force of the five fingers. After three seconds, the subject begins to move the object. After five seconds, the subject releases the object to reset it and stops detecting. After three seconds, the object is reset. After four seconds, the above steps are repeated until five grasps are completed. The endpoint position is then changed. After changing three positions, the test ends.

[0088] First, reflective markers are affixed to the joints of the five fingers and wrist of a healthy person, such as... Figure 3 As shown, a motion capture system is used to capture the motion information of a human hand grasping an object. Then, a 7-DOF exolimb robot, in conjunction with its underactuated prosthetic hand, learns skills based on the movements of a healthy human arm. Next, pressure film sensors are attached to the five fingers of the exolimb robot's hand to detect changes in finger force in real time when grasping an object. Then, a reinforcement learning-based PID parallel force-position hybrid controller is added to the exolimb system. The error between the exolimb end effector's motion trajectory and the human hand's motion trajectory serves as the controller's input, while fingertip force acts as negative feedback, jointly adjusting the exolimb's motion trajectory and reducing motion error. Finally, the root mean square error (RMSE) method is used to analyze the positional error of the exolimb grasping the object to a specific location, obtaining the grasping success rate.

[0089] like Figure 4 As shown, the visual information acquisition module mainly consists of a motion capture system and a host computer. In this embodiment, the motion capture system uses eight cameras to capture the movements of the human arm.

[0090] The finger force information acquisition module mainly consists of pressure film sensors and a 32-bit microcontroller. Pressure film sensors are attached to the five fingers of the exolimb robot's hand to acquire fingertip force information as feedback when the robot grasps different objects. During grasping actions, the pressure film sensors transmit fingertip pressure data to the 32-bit microcontroller in real time, which then transmits the data to the host computer via serial communication. In this embodiment, the output force of the pressure film sensors is first filtered by a low-pass filter before being used as the feedback input to the force controller and reinforcement learning control strategy.

[0091] The control module is a force-position hybrid controller based on reinforcement learning. It takes the error between the movement trajectory of the extremities and the movement trajectory of the human hand as input and the finger force information as negative feedback, and continuously and accurately adjusts the movement trajectory of the extremities to ensure the accuracy of grasping.

[0092] The human hand motion trajectory is processed by a trainable time warp algorithm and a Gaussian mixture regression algorithm based on expectation maximization, while the human arm motion trajectory is processed by dynamic motion primitives, thereby reconstructing the spatial motion trajectory of the extremity robot, enabling the extremity robot to learn the human arm grasping motion more stably and accurately.

[0093] The force-position hybrid controller includes a proportional-derivative (PD) controller and a proportional-integral (PI) force controller. The PD controller is used to generate a partial motion trajectory of the extremity robot based on the error between the human five-finger grasping motion trajectory and the extremity robot's hand motion trajectory. The PI force controller is used to change the motion trajectory based on finger force feedback to reduce the error.

[0094] Reinforcement learning has two main objectives: to generate the motion trajectories of the five fingers of the exolimb robot's hand based on trajectory errors; and to fine-tune the control parameters of the force controller based on the contact force feedback of the fingertips and the motion speed, preventing the exolimb robot's fingers from damaging objects due to excessive contact force or causing objects to slip due to insufficient contact force. The motion trajectory generated by the reinforcement learning-based force controller is used to obtain the joint angle transformation trajectory through inverse kinematics calculation, and then the joint angle transformation trajectory is transmitted to the exolimb robot's hand to complete the grasping process.

[0095] In this embodiment, reinforcement learning and PID parallel force-position hybrid control are combined, such as... Figure 5 As shown, where x d Let x represent the ideal spatial position and motion trajectory of the extremity robot hand, i.e., the spatial position and motion trajectory of a human hand, and let x represent the actual spatial position and motion trajectory of the extremity robot hand. e x represents the error between the ideal spatial position trajectory and the actual spatial position trajectory of the exoskeleton robot's hand. p The spatial position and motion trajectory of the exolimb robot hand calculated by the force-position hybrid controller, q p The trajectory of the joint angle transformation of the extremity robot's hand is calculated using inverse kinematics, where F represents the finger force feedback when the extremity robot's hand grasps an object, v represents the velocity of the extremity robot's hand when grasping an object, and r represents the joint angle transformation trajectory. x For position / direction commands, r f These are the parameters for the controller. The reinforcement learning strategy serves two purposes: firstly, to generate the motion trajectory r of the exoskeleton robot's hand.x Secondly, the parameter r of the fine-tuning force control method. f .

[0096] This embodiment adds a selection matrix to the force-position hybrid controller to define the degree of control for position and force in each direction. The control law consists of the PD controller acting on position, the PI controller acting on force, the selection matrix, and the position direction command r. x The composition, and the formula, can be expressed as:

[0097]

[0098] Where u is the vector of the driving force.

[0099] The selection matrix S can be represented as:

[0100] S = diag(s1,…,s6), s j ∈[0,1] (11)

[0101] The value in S corresponds to the degree of control the controller has in a given direction.

[0102] For PD controllers, only proportional gain is available. It is controllable, and the derivative gain... Based on Calculated. Settings The critical damping relationship is as follows:

[0103]

[0104] For PI controllers, only proportional gain is available. It is controllable, integral gain. It is relative to The calculations are as follows, and the correspondence between the two is as follows:

[0105]

[0106] In summary, the relevant parameters of the PID parallel force-position controller can be expressed as follows:

[0107]

[0108] To enable a PID parallel force-position controller to self-optimize and obtain optimal parameters, reinforcement learning is a good application method. In reinforcement learning, a reward function needs to be designed to minimize the trajectory error of the extremities. The reward function can be expressed as:

[0109] R(s,r)=ω1L m (||x e / x max || 1,2)+ω2L m (||r / r max ||2)+ω3L m (||F / F max ||2)+ω4ρ+ω5κ(15)

[0110] Among them, L m (y) = y → x, x ∈ [0, 1], therefore the closer to the target, the smaller the contact force and the higher the reward. ||·|| 1，2 For L1 and L2 norms; x e is the distance between the end effector of the extremity robot and the target at time step t; r is the action taken by the extremity robot; F is the contact force; ρ is the penalty given in each timing step to encourage rapid task completion; κ is a reward, which can be defined as:

[0111]

[0112] Finally, each component is weighted by a weight ω, where all ω are hyperparameters.

[0113] The force-position hybrid controller in this embodiment can not only generate the movement trajectory of the five fingers according to the trajectory error, but also fine-tune the control parameters of the force controller according to the contact force feedback of the fingertip force and the movement speed, so as to prevent the object from being damaged due to excessive contact force between the finger and the object, or the object from slipping due to insufficient contact force.

[0114] The steps and methods involved in the above embodiment two correspond to those in embodiment one. For specific implementation details, please refer to the relevant description section of embodiment one.

[0115] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0116] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A visual-haptic guided exo-limb robot force-position hybrid control method, characterized by, The method comprises the following steps: Obtain wrist posture information, five-finger posture information and generated motion trajectory of human arm in the process of gripping different objects from a visual perspective; Learn human wrist motion trajectory by using an exoskeleton robot arm, and learn human five-finger gripping motion trajectory by using an exoskeleton robot hand, and obtain the error of the human five-finger gripping motion trajectory and the exoskeleton robot hand motion trajectory; Obtain fingertip force information of the exoskeleton robot when gripping different objects as fingertip force feedback; Establish a force-position hybrid controller based on reinforcement learning and proportional-integral-derivative, and use the force-position hybrid controller to perform force-position control according to the error of the human five-finger gripping motion trajectory and the exoskeleton robot hand motion trajectory, and adjust the posture of the exoskeleton robot hand according to the fingertip force feedback to obtain an optimal motion trajectory, specifically as follows: The force-position hybrid controller takes the error of the human five-finger gripping motion trajectory and the exoskeleton robot hand motion trajectory as input, takes the fingertip force of the exoskeleton robot hand and the motion speed of the end effector as feedback, and iterates the control parameters of the controller according to the feedback data by using reinforcement learning, so that the robot can make optimal motion planning according to the environment; Continue learning and adjustment by using the optimal motion trajectory until the exoskeleton robot can complete stable gripping and meet the preset gripping success rate of the exoskeleton robot.

2. The visuo-haptically guided exo-limb robotic force-position hybrid control method of claim 1, wherein, The exoskeleton robot arm learns the human wrist motion trajectory by using a dynamic motion primitive algorithm.

3. The visuo-haptically guided exo-limb robotic force-position hybrid control method of claim 1, wherein, The exoskeleton robot hand learns the human five-finger gripping motion trajectory by using a trainable time warping algorithm and a Gaussian mixture regression algorithm based on expectation maximization.

4. The visuo-haptically guided exo-limb robotic force-position hybrid control method according to claim 3, wherein, The specific steps of processing the human five-finger gripping motion trajectory by using the Gaussian mixture regression algorithm are as follows: first, obtain similar kinematic information and motion characteristics by using the trainable time warping algorithm; and then process the kinematic information and the motion characteristics by using the Gaussian mixture regression algorithm based on expectation maximization, so as to fit all the trajectories into one trajectory.

5. The visuo-haptically guided exo-limb robotic force-position hybrid control method of claim 1, wherein, According to the optimal motion trajectory generated by the force-position hybrid controller, joint angle transformation trajectory is obtained by inverse kinematics calculation, and then the joint angle transformation trajectory is transmitted to the exoskeleton robot hand, so that the exoskeleton robot can complete gripping according to the joint angle transformation trajectory.

6. A tactile-guided exo-limb robot force / position hybrid control system, characterized by, It comprises: A visual information acquisition module configured to obtain wrist posture information, five-finger posture information and generated motion trajectory of human arm in the process of gripping different objects from a visual perspective; A machine learning module configured to learn human wrist motion trajectory by using an exoskeleton robot arm, and learn human five-finger gripping motion trajectory by using an exoskeleton robot hand, and obtain the error of the human five-finger gripping motion trajectory and the exoskeleton robot hand motion trajectory; A fingertip force information acquisition module configured to obtain fingertip force information of the exoskeleton robot when gripping different objects as fingertip force feedback; A control module configured to establish a force-position hybrid controller based on reinforcement learning and proportional-integral-derivative, and use the force-position hybrid controller to perform force-position control according to the error of the human five-finger gripping motion trajectory and the exoskeleton robot hand motion trajectory, and adjust the posture of the exoskeleton robot hand according to the fingertip force feedback to obtain an optimal motion trajectory, specifically as follows: The force-position hybrid controller takes the error between the human five-finger grasping trajectory and the exoskeleton robot hand trajectory as input, takes the force of the exoskeleton robot hand and the motion speed of the end effector as feedback, and iterates the control parameters of the controller according to the feedback data by using reinforcement learning, so that the robot can make the optimal motion planning according to the environment; The effect verification module is configured to continue learning and adjusting using the optimal motion trajectory until the exoskeleton robot can complete stable grasping and meet the preset grasping success rate of the exoskeleton robot.

7. The haptically guided exo-limb robotic force-position hybrid control system of claim 6, wherein, In the finger force information acquisition module, pressure film sensors are pasted at the five fingers of the exoskeleton robot hand, which are used to obtain the fingertip force information of the exoskeleton robot when grasping different objects as finger force feedback.

8. The haptically guided exo-limb robotic force-position hybrid control system of claim 7, wherein, The finger force information acquisition module further includes a single-chip microcomputer, and when the exoskeleton robot hand performs grasping action, the pressure film sensors transmit the fingertip pressure data to the single-chip microcomputer in real time, and then the single-chip microcomputer transmits the data to the upper computer through serial communication.

9. The haptically guided exo-limb robotic force-position hybrid control system of claim 6, wherein, The force-position hybrid controller includes a proportional-derivative controller and a proportional-integral force controller, wherein the proportional-derivative controller is used to generate part of the motion trajectory of the exoskeleton robot according to the error between the human five-finger grasping trajectory and the exoskeleton robot hand trajectory, and the proportional-integral force controller is used to change the motion trajectory according to the finger force feedback to reduce the error.

Images