Cooperative control method and system between robot arm and hand

Abstract

The invention provides a cooperative control method and system between an arm and a hand of a robot, and relates to the technical field of robots. Human hand posture data and arm posture data in the teaching data are detected; extracting a finger joint angle sequence and an arm joint angle sequence; mapping the finger joint angle sequence and the arm joint angle sequence into a manipulator joint angle and a mechanical arm joint angle, and generating a robot state sequence; a manipulator joint angle change value and a mechanical arm joint angle change value in the process that the robot makes contact with the object are recorded; training a low-level controller according to the robot state action pair sequence until a robot target action is generated; constructing a hierarchical control framework according to the target task of the target action of the robot; and based on a high-level behavior selection strategy, taking maximization of a reward function value as a target, training the hierarchical control architecture, and outputting a robot arm-hand cooperative control strategy by utilizing the trained hierarchical control architecture.

Description

Technical Field

[0001] This invention relates to the field of robotics, and in particular to a method and system for the coordinated control of a robot arm and hand. Background Technology

[0002] A robotic arm refers to the arm portion of a robot that can perform movements similar to a human arm to execute various tasks, such as grasping and carrying. The coordinated control between the robotic arm and the hand allows the robotic arm and hand to work together through appropriate control methods to achieve more precise and natural operation.

[0003] With the rapid development of robotics technology, robotic upper limb systems are increasingly being used in industrial production, medical services, and home services. Therefore, coordinated control between the robotic arm and hand is of great significance in improving production efficiency and reducing human workload.

[0004] Traditional methods primarily rely on precise kinematic and dynamic modeling. However, the highly redundant nature of robot upper limbs, coupled with the complex coupling between the robotic arm and dexterous hand, makes accurate modeling extremely difficult. Furthermore, environmental uncertainty poses significant challenges to traditional control methods, making it difficult for the system to achieve the expected control performance in practical applications. Deep learning-based robot control methods require large amounts of training data, resulting in lengthy learning processes, and the learned strategies often lack sufficient generalization ability, making it difficult to handle unfamiliar task scenarios. In summary, existing robot upper limb control methods still have significant shortcomings in system modeling, learning efficiency, real-time performance, and human-robot collaboration. These shortcomings are particularly prominent in handling the collaborative control of the robotic arm and dexterous hand, environmental adaptability, and task generalization, leading to a decline in the performance of the robot upper limb system, further reducing robot productivity and increasing human workload. Summary of the Invention

[0005] To address the significant shortcomings of existing robot upper limb control methods in system modeling, learning efficiency, real-time performance, and human-robot collaboration, particularly in handling the collaborative control of the robotic arm and dexterous hand, environmental adaptability, and task generalization, which leads to a decline in the performance of the robot upper limb system, further reducing robot production efficiency and increasing the human burden, this invention provides a collaborative control method and system between a robot arm and hand.

[0006] The technical solutions provided by the embodiments of the present invention are as follows: First aspect: This invention provides a method for coordinated control between a robot arm and a hand, comprising: S1: Collect teaching data on how humans move objects to their target positions; S2: Detect human hand and arm posture data in the teaching data; S3: Extract finger joint angle sequences and arm joint angle sequences from hand posture data and arm posture data; S4: By kinematic reorientation, the finger joint angle sequence and arm joint angle sequence are mapped to the robotic hand joint angle and robotic arm joint angle, respectively, and the robot state sequence is generated by combining human hand posture data and arm posture data. S5: Record the changes in the joint angles of the robotic hand and the robotic arm during the robot's contact with the object. The changes in the joint angles of the robotic hand and the robotic arm constitute the robot's action sequence. S6: Train the low-level controller based on the robot state-action pair sequence consisting of robot state sequence and robot action sequence until the robot target action is generated. The low-level controller includes: encoder, controller and discriminator. S7: Based on the target task of the robot's target action, construct a hierarchical control architecture, which includes: a high-level policy network, an interaction evaluation network, and a state representation module. S8: Based on the high-level behavior selection strategy, with the goal of maximizing the reward function value, the hierarchical control architecture is trained, and the trained hierarchical control architecture is used to output the robot arm hand-hand collaborative control strategy.

[0007] The second aspect: An embodiment of the present invention provides a collaborative control system between a robot arm and a hand, comprising: processor; The memory stores computer-readable instructions, which, when executed by a processor, implement a collaborative control method between the robot arm and hand, as described in the first aspect.

[0008] Third aspect: The present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a collaborative control method between a robot arm and a hand as described in the first aspect.

[0009] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following: In this embodiment of the invention, teaching data of humans moving objects to their target positions is collected, and then the human hand and arm posture data in the teaching data are detected in real time to ensure that the robot can accurately perform tasks in actual operation. Next, finger joint angle sequences and arm joint angle sequences are extracted from the hand and arm posture data and mapped to robotic hand joint angles and robotic arm joint angles, respectively generating robot state sequences and robot action sequences, thus realizing natural interaction between the robot and the object. Based on the robot state-action pair sequence composed of the robot state sequence and robot action sequence, the low-level controller is trained until the robot's target action is generated. Based on the target task of the robot's target action, a hierarchical control architecture is constructed, further enhancing the robot's intelligence. Finally, based on a high-level behavior selection strategy, with the goal of maximizing the reward function value, the hierarchical control architecture is trained, and the trained hierarchical control architecture outputs a robot arm-hand collaborative control strategy, thereby significantly improving the efficiency of the robot's long-term operation. This invention proposes a hierarchical control architecture, which designs independent low-level controllers for the arm and dexterous hand, and coordinates and controls them at a high level. This effectively reduces the control complexity of the system, improves the system's adaptability in different task scenarios, and achieves seamless coordination between the arm and dexterous hand. By combining and calling high-level strategies, the system's real-time response capability is guaranteed, and the stability of action execution is ensured, which greatly improves the robot's work efficiency and reduces labor costs. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A flowchart illustrating a collaborative control method between a robot arm and a hand, provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a low-level controller training method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a high-level behavior selection strategy training structure provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a collaborative control system between a robot arm and a hand, provided as an embodiment of the present invention. Detailed Implementation

[0012] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0013] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0014] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0015] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0016] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0017] Reference manual attached Figure 1 The diagram shows a flowchart illustrating a collaborative control method between a robot arm and a hand provided in an embodiment of the present invention.

[0018] This invention provides a method for collaborative control between a robot arm and a hand. This method can be implemented by a collaborative control device between the robot arm and hand, which can be a terminal or a server. The processing flow of the collaborative control method between the robot arm and hand may include the following steps:

[0019] S1: Collect teaching data on how humans move objects to their target positions.

[0020] Specifically, teaching data is collected using an RGB-D camera, showing how humans move objects to their target positions.

[0021] Among them, an RGB-D camera is a type of camera that combines color (RGB) images and depth (D) information. It can acquire the color of objects in an image and their distance (depth) from the camera. This type of camera helps robots capture the shape, size, and relative position of objects, providing robots with more accurate environmental perception. Teaching data refers to a series of task data demonstrated to a robot through human operation or guidance.

[0022] It's worth noting that by using an RGB-D camera to collect teaching data, the robot is provided with detailed visual and depth information, enabling it to accurately understand the shape, size, and position of objects, as well as how they interact with humans. This data provides precise input for subsequent robot motion learning, allowing the robot to mimic human behavior and perform tasks efficiently in similar situations.

[0023] In one possible implementation, the teaching data specifically includes: RGB-D camera video stream, human demonstration action sequence, object pose information, and target pose information.

[0024] In this context, RGB-D camera video stream refers to a series of continuous video frames, which include not only color information but also depth data for each pixel. Human demonstration action sequence refers to the process of a human performing a task through natural movements (such as carrying objects or placing items). Object pose information refers to the position and orientation of an object in three-dimensional space, typically including the object's coordinates (e.g., X, Y, Z coordinates) and orientation (e.g., described by rotation angle). Target pose information refers to the spatial position information of the target object or target location in the task.

[0025] It should be noted that by using motion data demonstrated by humans, the pose information of objects in space, and the target pose information required by the task, the robot is provided with detailed information needed to perform the task, enabling the robot to accurately learn and imitate human actions and achieve correct operation of objects.

[0026] S2: Detect human hand and arm posture data in the teaching data.

[0027] Specifically, MediaPipe technology is used to detect human hand and arm posture data in the teaching data.

[0028] MediaPipe is an open-source, cross-platform framework primarily used for building multimedia processing applications, particularly in computer vision and machine learning. It enables real-time human pose detection, facial tagging, and gesture recognition. Human hand pose data refers to the spatial position and movement information of the hand extracted using computer vision technologies (such as MediaPipe). Arm pose data refers to the arm's pose information in three-dimensional space, typically including the angles of the upper limb joints and the direction of movement of the elbow and shoulder.

[0029] Specifically, human hand posture data includes the coordinates of key points on the hand, while arm posture data includes the position and posture of human upper limb joints.

[0030] It should be noted that by using MediaPipe technology to detect hand and arm posture data in real time, the robot can accurately acquire details of human movements. This process can not only efficiently capture the joint angles of fingers and upper limbs, but also provide dynamic information in space, ensuring that the robot can accurately imitate and synchronize human movements.

[0031] S3: Extract finger joint angle sequences and arm joint angle sequences from hand posture data and arm posture data.

[0032] Among them, the finger joint angle sequence refers to the angle changes of each joint in the human finger during movement. Each finger joint (such as the interphalangeal joint, metacarpophalangeal joint, etc.) can produce different angle changes, and these changes form a sequence throughout the entire hand movement. The arm joint angle sequence refers to the sequence of angle changes of each joint in the human arm (such as the shoulder, elbow, wrist, etc.).

[0033] It's worth noting that by extracting the joint angle sequences of the hand and arm, it's possible to accurately record and reproduce every detail of a human's movements when performing a task. This data extraction provides robots with precise motion control information, enabling them to mimic complex human movements and execute tasks with high accuracy.

[0034] In one possible implementation, the finger joint angle sequence and the arm joint angle sequence are specifically as follows: in, θ h A sequence representing the angles of the finger joints. θ hi Indicates the first i The angle of each finger joint , n This indicates the total number of finger joint angles. ψ h This represents the sequence of arm joint angles. ψ hj Indicates the first j An arm joint angle , m This indicates the total number of angles of the arm joints.

[0035] S4: By kinematic retargeting, the finger joint angle sequence and arm joint angle sequence are mapped to the robotic hand joint angle and robotic arm joint angle, respectively, and the robot state sequence is generated by combining human hand posture data and arm posture data.

[0036] Kinematic retargeting refers to mapping human posture data to the robot system using a kinematic model to ensure the robot can perform tasks that match human movements appropriately. Robotic hand joint angles refer to the angles of the joints in the robotic hand (such as finger joints and metacarpophalangeal joints), while robotic arm joint angles refer to the angles of the joints in the robot arm (such as shoulder, elbow, and wrist). Robot state sequences refer to the robot's state data at different points in time during task execution, typically including information such as the robot's position, posture, and joint angles.

[0037] It's important to note that kinematic retargeting precisely maps human hand and arm posture data onto the robot, enabling the robot to simulate and perform complex human movements. This method allows the robot to convert hand and arm movements into corresponding joint angles of the robotic hand and arm, generating accurate robot state sequences.

[0038] In one possible implementation, S4 specifically includes: S401: Set distance threshold.

[0039] The distance threshold refers to a preset distance value used to determine the contact state between objects or robots during a specific task or operation.

[0040] Those skilled in the art can set the distance threshold according to the actual situation, and the present invention does not impose any limitations.

[0041] S402: Determine whether the robot is in contact with an object using a contact state detection algorithm. in, S con Indicates the contact status. S con When =1, it indicates that the robot has come into contact with the object. S con When =0, it means that the robot and the object are not in contact. p e This indicates the position information of the robot's end effector. p o Indicates the position of an object. δ This represents the distance threshold.

[0042] It should be noted that contact state detection algorithms are algorithms that determine whether contact has occurred by analyzing factors such as the relative position, distance, and speed between the robot and the object. These algorithms typically utilize sensor data (such as RGB-D cameras, force sensors, and collision sensors) to monitor the interaction between the robot's end effector and the object.

[0043] S403: When the robot comes into contact with an object, record the contact position and combine it with human hand pose data and arm pose data to generate a robot state sequence: in, S t express t The robot's state at any given moment. θ r This represents the sequence of joint angles of the robotic arm. ψ r This represents the sequence of joint angles of the robotic arm. q o Indicates the posture of an object. p g Indicates the target location. q g Indicates the target's attitude. p c Indicates the location of the contact point.

[0044] It should be noted that this algorithm enables robots to mimic and replicate human actions, reducing the complexity of robot control and allowing robots to perform tasks more flexibly and naturally, especially in complex scenarios such as grasping, carrying, and fine manipulation.

[0045] S5: Record the changes in the joint angles of the robotic hand and the robotic arm during the robot's contact with the object. The changes in the joint angles of the robotic hand and the robotic arm constitute the robot's action sequence.

[0046] The joint angle variation values ​​of a robotic hand refer to the changes in angle of each joint of the robot's hand during task execution. By recording these changes, the robot can adjust its hand movements as needed to ensure precise task completion, such as grasping and placing objects. The joint angle variation values ​​of a robotic arm refer to the changes in angle of each joint of the robot's arm (such as the shoulder, elbow, and wrist). Recording these changes helps adjust the position and posture of the robotic arm, enabling the robot to perform more complex actions, such as object handling and precise positioning.

[0047] Among them, robot action sequence refers to a series of actions that occur in chronological order when a robot is performing a task.

[0048] It's worth noting that by recording changes in the joint angles of the robotic hand and arm, the robot's motion evolution during contact with objects can be precisely tracked. This allows the robot to continuously adjust and optimize its motion strategies in complex tasks, thereby improving the accuracy and efficiency of task execution.

[0049] Reference manual attached Figure 2The diagram shows a structural schematic of a low-level controller training method provided by an embodiment of the present invention.

[0050] Figure 2 In this process, the collected human teaching data is preprocessed to generate a sequence of robot state-action pairs. Then, the sequence of robot state-action pairs is encoded and represented by an encoder. Combined with kinematic constraints, dynamic constraints, and style rewards, the low-level controller is trained, and the loss function of the controller is updated according to the current state and action to help optimize the controller's performance. This continues until the arm controller and the dexterous hand controller generate actions. After the system executes the generated actions, it enters a new state s. At the same time, the discriminator evaluates the performance of the system in the current state and updates its own evaluation criteria. If the discriminator evaluates that the system's performance is biased (such as failing to achieve the expected goal), it updates the loss function, adjusts the system's feedback, and evaluates whether its behavior is real and effective, ensuring that the actions taken meet the predetermined goals or task requirements.

[0051] It should be noted that the system continuously improves its performance by constantly evaluating the current state, optimizing the controller, executing actions, and updating feedback.

[0052] S6: Train the low-level controller based on the robot state-action pair sequence consisting of the robot state sequence and the robot action sequence until the robot target action is generated. The low-level controller includes: encoder, controller and discriminator.

[0053] The low-level controller is part of the robot control system, responsible for executing the actual motion control. The encoder is a component used to map the robot's state-action sequence to the latent behavior space. The controller is responsible for generating the robot's target motion; it calculates and generates control commands based on the robot's current state and the preset target. The discriminator is the part used to evaluate the realism of the target motion; it generates style reward signals by learning whether the robot's executed actions conform to the expected goal.

[0054] It should be noted that by training the low-level controller, the robot can automatically generate target actions according to task requirements. In conjunction with the encoder, controller, and discriminator, the low-level controller can efficiently map robot state-action pairs to the potential behavior space and optimize control strategies.

[0055] In one possible implementation, an encoder is used to map a sequence of robot state-action pairs to a latent behavior space and is encoded using a multilayer perceptron architecture.

[0056] The controller generates the robot's target motion based on a conditional Gaussian policy. The controller includes an arm controller and a dexterous hand controller.

[0057] The discriminator is used to evaluate the authenticity of the robot's target actions and generate style reward signals.

[0058] In one possible implementation, S6 specifically includes: S601: Initialize the parameters of the encoder, controller, and discriminator respectively.

[0059] S602: Calculate the potential representation of the robot's state-action sequence using an encoder.

[0060] S603: Combine the latent representation to construct the loss function of the encoder.

[0061] S604: Train the encoder with the objective of minimizing the encoder's loss function: in, L E Represents the encoder loss function. λ KL express KL Divergence weighting coefficient D KL express KL divergence, N (0, I ) represents the standard normal distribution. E () indicates expectation. s Indicates the robot's state. a This indicates the robot's actions.

[0062] S605: Train the controller with the objective of minimizing the total loss function value of the controller: in, L Represents the total loss function. L task This represents the task-related loss function. L style Represents the style loss function, L constraint Represents the constraint loss function. λ 1 indicates the weight of the style loss function. λ 2 represents the weight of the constraint loss function. a t Indicates time step t The action at that time s t Indicates time step t The state at that time, z t Indicates time step t Noise at that time Lkin Indicates kinematic constraints, L dyn Indicates dynamic constraints. D Indicates the discriminator, π Represents the policy function. θ Indicates the parameters of the strategy function. z Indicates noise.

[0063] S606: Train the discriminator with the goal of maximizing its loss function value and calculate its style reward.

[0064] The objective function is as follows: in, L D This represents the loss function value of the discriminator. E Expressing expectations, s Indicates state, a Indicates an action, π Represents the policy function. θ Indicates the parameters of the strategy function. π θ ( s , z ) indicates based on state s and noise z The generated strategy D denoted as the discriminator, and log represents the logarithmic function.

[0065] The style rewards are as follows: in, r s Representing state s The style of the reward function.

[0066] It should be noted that this process not only improves the robot's adaptability, enabling it to cope with changing environments, but also continuously optimizes motion quality through style reward signals from the discriminator, ensuring that the robot can perform tasks efficiently and accurately. Through continuous training and adjustments, the low-level controller can significantly improve the robot's motion accuracy and task completion efficiency.

[0067] S7: Based on the target task of the robot's target action, construct a hierarchical control architecture, which includes: a high-level policy network, an interaction evaluation network, and a state representation module.

[0068] The hierarchical control architecture is a method for handling tasks through control modules at different levels. It decomposes complex tasks into multiple layers, each handling different control and decision-making processes. The high-level policy network is the upper-level network in the hierarchical control architecture, responsible for deciding the global policy for the task. The interaction evaluation network is used to evaluate the robot's target actions and the quality of its interactions with the environment or task. The state representation module is part of the hierarchical control architecture, responsible for transforming the robot's current state into a representation that the high-level policy network can understand.

[0069] It should be noted that by constructing a hierarchical control architecture, complex tasks are divided into control strategies at different levels, enabling the robot to efficiently handle various tasks. The high-level policy network is responsible for generating global decision-making schemes, ensuring that the robot can make optimal choices based on the overall goal, while the interactive evaluation network evaluates and optimizes the robot's actions in real time.

[0070] Reference manual attached Figure 3 The diagram shows a schematic representation of a high-level behavior selection strategy training method provided by an embodiment of the present invention.

[0071] Figure 3 The system first extracts state features from human teaching data and constructs a corresponding task structure based on task requirements. Combining interaction style rewards and task completion rewards, the hierarchical control architecture is trained to maximize the reward function value. The trained discriminator then evaluates and optimizes the robot's actions in real time. Through continuous interactive training and task feedback mechanisms, the system can continuously adjust its strategy based on real-time data, improving the accuracy and efficiency of task execution. Furthermore, the system considers various factors in task construction, such as environmental conditions, object distance, and hand-object docking, ensuring the task can flexibly adapt to changes in complex environments. The task completion evaluation stage also ensures continuous checking and optimization of system performance during training, achieving long-term self-improvement. The overall scalability and modular design of the structure enable the system to cope with different task requirements and exhibit excellent adaptability and robustness in practical applications.

[0072] S8: Based on the high-level behavior selection strategy, with the goal of maximizing the reward function value, the hierarchical control architecture is trained, and the trained hierarchical control architecture is used to output the robot arm hand-hand collaborative control strategy.

[0073] Among them, the high-level behavior selection strategy refers to the strategy by which the robot selects the optimal behavior based on the task objective and the environmental state in the high-level control architecture. The reward function is a core component of reinforcement learning and is used to measure the robot's performance in the process of performing the task.

[0074] It should be noted that by combining a high-level behavior selection strategy with reinforcement learning to optimize robot behavior, the robot can efficiently perform complex tasks in varied environments. The high-level policy network maximizes the success rate of the target task based on the reward function, while the interactive evaluation network evaluates the robot's actions and task progress in real time, further optimizing the control strategy. The hierarchical control architecture design enables the robot to flexibly respond to different task requirements while efficiently selecting and executing the optimal action.

[0075] In one possible implementation, the formula for calculating the reward function value is as follows: in, r Represents the reward function, r g This represents the task reward function. r s Represents the state reward function, r b This represents the discriminator reward function. w task Indicates task weight, f task This indicates the assessment of task progress, where g represents the objective. s I Represents the sequence of interactive states. D I This indicates an interactive rating.

[0076] In one possible implementation, S8 specifically includes: S801: Initialize the parameters of the high-level policy network and the interaction evaluation network respectively.

[0077] S802: Generate low-level behavior sequences through a high-level policy network based on the target task.

[0078] The target task is the specific task or goal that the robot needs to complete. This can be any predefined task, such as object handling, object placement, or grasping operations. The low-level behavior sequence is the sequence of specific actions or instructions that the robot needs to execute.

[0079] S803: The trained controller executes low-level behavior sequences to generate the target action.

[0080] S804: Through an interactive evaluation network, the target action is interactively scored until the reward function value is greater than the preset reward function value, thus completing the training of the hierarchical control architecture.

[0081] Those skilled in the art can set the value of the preset reward function according to the actual situation, and the present invention does not limit it.

[0082] It should be noted that the hierarchical control architecture design enables the robot to flexibly respond to different task requirements, while efficiently selecting and executing the optimal action. In this way, the robot can continuously learn and adjust during task execution, maximizing efficiency and accuracy, thereby improving the quality and flexibility of task completion.

[0083] In this invention, the complex arm-hand collaborative control task is decomposed into two levels: high-level behavior selection and low-level action execution, effectively reducing the system's control complexity. The modular learning method based on behavior encoding significantly improves the system's generalization ability, enabling the robot to adapt to unseen objects and task scenarios. The introduced interactive evaluation mechanism ensures the naturalness and fluency of action execution, allowing the robot to exhibit a more human-like operating style. Furthermore, the hierarchical architecture design of this invention improves the system's maintainability and scalability, simplifies the development process, and facilitates rapid deployment and migration in different application scenarios. This innovative control method provides a new approach to solving the challenges of upper limb collaborative control in robots and has significant application value in the fields of industrial automation and service robots.

[0084] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following: In this embodiment of the invention, teaching data of humans moving objects to their target positions is collected, and then the human hand and arm posture data in the teaching data are detected in real time to ensure that the robot can accurately perform tasks in actual operation. Next, finger joint angle sequences and arm joint angle sequences are extracted from the hand and arm posture data and mapped to robotic hand joint angles and robotic arm joint angles, respectively generating robot state sequences and robot action sequences, thus realizing natural interaction between the robot and the object. Based on the robot state-action pair sequence composed of the robot state sequence and robot action sequence, the low-level controller is trained until the robot's target action is generated. Based on the target task of the robot's target action, a hierarchical control architecture is constructed, further enhancing the robot's intelligence. Finally, based on a high-level behavior selection strategy, with the goal of maximizing the reward function value, the hierarchical control architecture is trained, and the trained hierarchical control architecture outputs a robot arm-hand collaborative control strategy, thereby significantly improving the efficiency of the robot's long-term operation. This invention proposes a hierarchical control architecture, which designs independent low-level controllers for the arm and dexterous hand, and coordinates and controls them at a high level. This effectively reduces the control complexity of the system, improves the system's adaptability in different task scenarios, and achieves seamless coordination between the arm and dexterous hand. By combining and calling high-level strategies, the system's real-time response capability is guaranteed, and the stability of action execution is ensured, which greatly improves the robot's work efficiency and reduces labor costs.

[0085] Reference manual attached Figure 4 The diagram shows a structural schematic of a collaborative control system between a robot arm and a hand provided by the present invention.

[0086] The present invention also provides a collaborative control system 20 between a robot arm and a hand, applied to the aforementioned collaborative control method between a robot arm and a hand, comprising: Processor 201.

[0087] The memory 202 stores computer-readable instructions, which, when executed by the processor 201, implement the collaborative control method between the robot arm and the hand as described in the method embodiment.

[0088] The robotic arm and hand collaborative control system 20 provided by the present invention can execute the above-mentioned robotic arm and hand collaborative control method and achieve the same or similar technical effects. To avoid repetition, the present invention will not elaborate further.

[0089] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following: In this embodiment of the invention, teaching data of humans moving objects to their target positions is collected, and then the human hand and arm posture data in the teaching data are detected in real time to ensure that the robot can accurately perform tasks in actual operation. Next, finger joint angle sequences and arm joint angle sequences are extracted from the hand and arm posture data and mapped to robotic hand joint angles and robotic arm joint angles, respectively generating robot state sequences and robot action sequences, thus realizing natural interaction between the robot and the object. Based on the robot state-action pair sequence composed of the robot state sequence and robot action sequence, the low-level controller is trained until the robot's target action is generated. Based on the target task of the robot's target action, a hierarchical control architecture is constructed, further enhancing the robot's intelligence. Finally, based on a high-level behavior selection strategy, with the goal of maximizing the reward function value, the hierarchical control architecture is trained, and the trained hierarchical control architecture outputs a robot arm-hand collaborative control strategy, thereby significantly improving the efficiency of the robot's long-term operation. This invention proposes a hierarchical control architecture, which designs independent low-level controllers for the arm and dexterous hand, and coordinates and controls them at a high level. This effectively reduces the control complexity of the system, improves the system's adaptability in different task scenarios, and achieves seamless coordination between the arm and dexterous hand. By combining and calling high-level strategies, the system's real-time response capability is guaranteed, and the stability of action execution is ensured, which greatly improves the robot's work efficiency and reduces labor costs.

[0090] It should be understood that the processor in the embodiments of the present invention can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0091] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0092] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. Semiconductor media can be solid-state drives.

[0093] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0094] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0095] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0096] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0097] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0098] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0099] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0100] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0101] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0102] This invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a collaborative control method between a robot arm and a hand as described in the method embodiment.

[0103] The present invention provides a computer-readable storage medium that can realize the steps and effects of the collaborative control method between the robot arm and the hand in the above-described method embodiments. To avoid repetition, the present invention will not repeat them.

[0104] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following: In this embodiment of the invention, teaching data of humans moving objects to their target positions is collected, and then the human hand and arm posture data in the teaching data are detected in real time to ensure that the robot can accurately perform tasks in actual operation. Next, finger joint angle sequences and arm joint angle sequences are extracted from the hand and arm posture data and mapped to robotic hand joint angles and robotic arm joint angles, respectively generating robot state sequences and robot action sequences, thus realizing natural interaction between the robot and the object. Based on the robot state-action pair sequence composed of the robot state sequence and robot action sequence, the low-level controller is trained until the robot's target action is generated. Based on the target task of the robot's target action, a hierarchical control architecture is constructed, further enhancing the robot's intelligence. Finally, based on a high-level behavior selection strategy, with the goal of maximizing the reward function value, the hierarchical control architecture is trained, and the trained hierarchical control architecture outputs a robot arm-hand collaborative control strategy, thereby significantly improving the efficiency of the robot's long-term operation. This invention proposes a hierarchical control architecture, which designs independent low-level controllers for the arm and dexterous hand, and coordinates and controls them at a high level. This effectively reduces the control complexity of the system, improves the system's adaptability in different task scenarios, and achieves seamless coordination between the arm and dexterous hand. By combining and calling high-level strategies, the system's real-time response capability is guaranteed, and the stability of action execution is ensured, which greatly improves the robot's work efficiency and reduces labor costs.

[0105] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0106] The following points need to be explained: (1) The accompanying drawings of the embodiments of the present invention only involve the structures involved in the embodiments of the present invention. Other structures can refer to the general design.

[0107] (2) For clarity, the thickness of layers or regions is enlarged or reduced in the drawings used to describe embodiments of the invention, i.e., these drawings are not drawn to scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being “above” or “below” another element, the element may be “directly” located “above” or “below” the other element or there may be intermediate elements.

[0108] (3) Where there is no conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other to obtain new embodiments.

[0109] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for coordinated control between a robotic arm and a hand, characterized in that, include: S1: Collect teaching data on how humans move objects to their target positions; S2: Detect human hand posture data and arm posture data in the teaching data; S3: Extract finger joint angle sequences and arm joint angle sequences from the hand posture data and the arm posture data; S4: By kinematic reorientation, the finger joint angle sequence and the arm joint angle sequence are mapped to robotic hand joint angle and robotic arm joint angle, respectively, and the robot state sequence is generated by combining the human hand posture data and the arm posture data. S5: Record the changes in the joint angles of the robotic hand and the robotic arm during the process of the robot contacting the object, wherein the changes in the joint angles of the robotic hand and the robotic arm constitute the robot's action sequence; S6: Train the low-level controller based on the robot state-action pair sequence composed of the robot state sequence and the robot action sequence until the robot target action is generated, wherein the low-level controller includes: an encoder, a controller and a discriminator; S7: Based on the target task of the robot's target action, construct a hierarchical control architecture, which includes: a high-level policy network, an interaction evaluation network, and a state representation module. S8: Based on the high-level behavior selection strategy, with the goal of maximizing the reward function value, the hierarchical control architecture is trained, and the trained hierarchical control architecture is used to output the robot arm hand-hand collaborative control strategy.

2. The collaborative control method between the robot arm and hand according to claim 1, characterized in that, The teaching data specifically includes: RGB-D camera video stream, human demonstration action sequence, object pose information, and target pose information.

3. The collaborative control method between the robot arm and hand according to claim 1, characterized in that, The finger joint angle sequence and the arm joint angle sequence are specifically as follows: in, θ h A sequence representing the angles of finger joints. θ hi Indicates the first i The angle of each finger joint , n This indicates the total number of finger joint angles. ψ h This represents the sequence of arm joint angles. ψ hj Indicates the first j An arm joint angle , m This indicates the total number of angles of the arm joints.

4. The collaborative control method between the robot arm and hand according to claim 1, characterized in that, S4 specifically includes: S401: Set distance threshold; S402: Determine whether the robot is in contact with an object using a contact state detection algorithm. in, S con Indicates the contact status. S con When =1, it indicates that the robot has come into contact with the object. S con When =0, it means that the robot and the object are not in contact. p e This indicates the position information of the robot's end effector. p o Indicates the position of an object. δ Indicates the distance threshold; S403: When the robot comes into contact with an object, record the contact position and, combined with the human hand posture data and the arm posture data, generate the robot state sequence: in, S t express t The robot's state at any given moment. θ r This represents the sequence of joint angles of the robotic arm. ψ r This represents the sequence of joint angles of the robotic arm. q o Indicates the posture of an object. p g Indicates the target location. q g Indicates the target's attitude. p c Indicates the location of the contact point.

5. The collaborative control method between the robot arm and hand according to claim 1, characterized in that, The encoder is used to map the robot state-action pair sequence to a potential behavior space and is encoded using a multilayer perceptron structure; The controller generates the robot's target action according to a conditional Gaussian policy, wherein the controller includes an arm controller and a dexterous hand controller; The discriminator is used to evaluate the authenticity of the robot's target action and generate a style reward signal.

6. The collaborative control method between the robot arm and hand according to claim 1, characterized in that, S6 specifically includes: S601: Initialize the parameters of the encoder, the controller, and the discriminator respectively; S602: Calculate the potential representation of the robot state-action pair sequence using the encoder; S603: Construct the loss function of the encoder by combining the latent representation; S604: Train the encoder with the objective of minimizing the loss function value of the encoder: in, L E Represents the encoder loss function. λ KL express KL Divergence weighting coefficient D KL express KL divergence, N (0, I ) represents the standard normal distribution. E () indicates expectation. s Indicates the robot's state. a Indicates robot actions; S605: Train the controller with the objective of minimizing the total loss function value of the controller: in, L Represents the total loss function. L task This represents the task-related loss function. L style Represents the style loss function, L constraint Represents the constraint loss function. λ 1 indicates the weight of the style loss function. λ 2 represents the weight of the constraint loss function. a t Indicates time step t The action at that time s t Indicates time step t The state at that time, z t Indicates time step t Noise at that time L kin Indicates kinematic constraints, L dyn Indicates dynamic constraints. D Indicates the discriminator, π Represents the policy function. θ Indicates the parameters of the strategy function. z Indicates noise; S606: Train the discriminator with the objective of maximizing the loss function value of the discriminator, and calculate the style reward of the discriminator; The objective function is specifically: in, L D This represents the loss function value of the discriminator. E Expressing expectations, s Indicates state, a Indicates an action, π Represents the policy function. θ Indicates the parameters of the strategy function. π θ ( s , z ) indicates based on state s and noise z The generated strategy D The discriminant is represented by log, and the logarithm function is represented by log. The style reward is specifically as follows: in, r s Representing state s The style of the reward function.

7. The method for coordinated control between a robot arm and a hand according to claim 1, characterized in that, The specific formula for calculating the reward function value is as follows: in, r Represents the reward function, r g This represents the task reward function. r s Represents the state reward function, r b This represents the discriminator reward function. w task Indicates task weight, f task This indicates the assessment of task progress, where g represents the objective. s I Represents the sequence of interactive states. D I This indicates an interactive rating.

8. The collaborative control method between a robot arm and a hand according to claim 1, characterized in that, S8 specifically includes: S801: Initialize the parameters of the high-level policy network and the interaction evaluation network respectively; S802: Generate a low-level behavior sequence through the high-level policy network according to the target task; S803: The trained controller executes the low-level behavior sequence to generate the target action; S804: The target action is interactively scored through the interactive evaluation network until the reward function value is greater than the preset reward function value, thus completing the training of the hierarchical control architecture.

9. A collaborative control system between a robotic arm and a hand, characterized in that, include: processor; A memory storing computer-readable instructions, which, when executed by the processor, implement the collaborative control method between the robot arm and the hand as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the collaborative control method between the robot arm and the hand as described in any one of claims 1 to 8.

Images