A human-robot collaboration-oriented robot intelligent article delivery method and system

Abstract

The application discloses a kind of human-computer cooperation-oriented robot intelligent article delivery method and system, belong to between robot and person article intelligent delivery, method includes: construct mechanical hand with proximity sensing and pressure sensing, and fusion vision is carried out multimodal perception, real-time sensing human hand's approach;Based on object, human hand and the virtual bounding box of robot, the human and robot digital twin system in object delivery process is constructed, and the arrival state of human hand is evaluated in real time;According to the different article delivery process of person-person, the pose of receiver to delivery point and the delivery time when deliverer completes are core indexes, data is collected, and dataset is constructed, and reinforcement learning algorithm is developed to train person-like delivery time selection model, which can enable robot to have the ability of active selection of delivery time.

Description

Technical Field

[0001] This invention belongs to the field of robot human-robot collaboration technology, and particularly relates to a robot intelligent item delivery method and system for human-robot collaboration. Background Technology

[0002] With the development of collaborative robot technology, more and more robots will participate in our daily lives and production. In human-robot collaborative tasks, the transfer of objects between robots and humans is a crucial step in the process. Developing and designing reasonable end effectors, selecting appropriate sensing channels, and developing robust algorithms for grasping states and human intent perception are of great significance for ensuring the efficiency of object transfer between robots and humans and improving the user experience. Object transfer between people can be divided into three processes: approach, arrival, and transfer. However, current methods for robot-to-human object transfer mostly focus on the transfer process, typically using vision- or force-based methods to assess whether the robot has completed the transfer. Furthermore, these methods often employ preset fixed times or passively trigger the robot to open its gripper only after receiving force or voice cues from the human. This passive transfer method is highly inconsistent with the multimodal inference-based active transfer method between humans and significantly reduces the user experience.

[0003] Therefore, it is necessary to establish a robot intelligent item transfer algorithm for human-robot collaboration. This system should include multimodal human hand proximity perception, arrival status assessment based on digital twins of the human hand and the item, datasets containing transfer times for different items and people, and a robot autonomous transfer control algorithm based on reinforcement learning. By designing a robot end effector with intelligent proximity perception capabilities, utilizing arrival status assessment based on digital twins and spatial relationships, and dynamic prediction of transfer timing based on reinforcement learning, the robot can achieve human-like autonomous control of approach, arrival, and transfer, ultimately improving the interactive experience of collaborative robots.

[0004] There have been attempts to improve the ability of robots to transfer items by addressing their structural design, such as in the application published with number […].

[0005] Patent application CN111301838A discloses an intelligent robot for delivering items. However, this solution only focuses on the robot's structural body, making it difficult for the robot to actively sense a person's approach, arrival, and delivery status.

[0006] To estimate the proximity of a human hand, CN113681565A discloses a human-robot collaborative method and apparatus for transferring objects between a robot and a human. This method uses visual recognition of the human hand's pose to control a robotic arm to move to the hand's position, thus transferring the object. However, this method focuses primarily on the proximity process during transfer, with the human passively waiting. The actual object transfer process involves perceiving the proximity of the human hand, assessing the transfer location, and actively selecting the timing of the transfer. To improve the user experience, a more comprehensive algorithm and system for object transfer is needed, enabling the robot to actively choose the timing of the transfer. Summary of the Invention

[0007] In view of the above, the purpose of this invention is to provide a robot intelligent object transfer method and system for human-robot collaboration, which can realize rapid perception of human hand approach, accurate estimation of human hand arrival at the transfer point, and active selection of transfer timing, ensuring the robustness of the robot transfer process, thereby improving the robot's interactive experience.

[0008] To achieve the above-mentioned objectives, according to one aspect of the present invention, a method for robot intelligent item delivery oriented towards human-robot collaboration is provided, the method comprising the following steps:

[0009] S1, construct a robotic hand with proximity and pressure sensing, and integrate vision for multimodal perception to detect the approach of a human hand in real time;

[0010] S2, based on the virtual bounding box of objects, human hands and robots, constructs a digital twin system of human and robot in the process of object transfer. This digital twin system can evaluate the arrival status of human hands in real time based on the visually updated position of human hands and the position of grasped objects updated based on robot kinematics, as well as the spatial geometric relationship between the two.

[0011] S3. Data is collected based on the receiver's pose at the transfer point and the timing of the transferor releasing the object during the transfer of different objects between people. The core indicators include three indicators: the hand position based on visual computing, the hand proximity state based on capacitive variable sensing, and the time the hand stays at that position. A dataset is constructed based on this data. An improved DDPG algorithm is developed based on Deep Deterministic Policy Gradient (DDPG). The constructed dataset is used to train a human-like transfer timing selection model, which enables the robot to actively select the transfer timing.

[0012] Furthermore, S1 includes: selecting suitable proximity capacitor plate material and piezoelectric material, and designing a robotic arm to achieve the coverage of the fingers by the proximity capacitor plate material. The piezoelectric material is attached to the surface of the capacitor plate material as a pressure sensor. The proximity capacitance will change when the human hand approaches, and the proximity state of the human hand can be realized in real time and in multimodal perception by combining a visual sensor.

[0013] Furthermore, S1 specifically includes: a robotic hand structure covered with a metal material as the electrode material for proximity capacitance sensing, and equipped with a visual sensor, and a multimodal human hand proximity sensing algorithm that integrates capacitance change and machine vision algorithm;

[0014] In the multimodal human hand proximity perception algorithm, the distance between the human hand and the robot gripper is calculated using the following formula, where d refers to the calculated distance between the human hand and the robot gripper, d1 refers to the human hand distance calculated based on the visual algorithm, and d2 refers to the human hand distance based on the proximity capacitance test.

[0015]

[0016] In the visual algorithm, the real-time distance between the human hand and the robotic arm is calculated according to the formula d1=inf{‖xy‖2:x∈traA(t),y∈traB(t)}, where traA(t) and traB(t) are the trajectories of the human hand A and the object or gripper B within the time interval t[t0, t1]. Based on the change in proximity capacitance C, the equation is referenced. Where ∈ is the dielectric constant, k is the electrostatic constant, and S is the area of ​​the coated plate. Solve for d2.

[0017] Furthermore, S2 includes: constructing a virtual bounding box of the human hand, the object being grasped, and the robotic arm containing the robotic hand, and establishing the spatial relationship between the object being grasped and each joint of the robotic arm according to the forward kinematics of the robotic arm, thereby constructing a digital twin system of the human hand, the object being grasped, and the robotic arm. The calculation formula is the same as the calculation formula of d1 in step S1. Solving the distance relationship between the bounding box of the human hand and the object being grasped, and evaluating the arrival state of the human hand according to the distance relationship between the object being grasped and the human hand in the digital space.

[0018] Furthermore, in step S3, the intelligent transfer algorithm for items is implemented using the human hand pose recognition module, the DDPG module, and the robot control module;

[0019] The human hand pose recognition module is used to identify the pose of the receiver when it stops at the grasping point. The timing starts from the moment the receiver's hand stops moving, the time the transmitter releases the object is recorded, and the time the receiver's hand stays at that position is calculated to build a dataset.

[0020] The DDPG module contains an improved DDPG algorithm and a reward value function; the improved DDPG algorithm includes the following steps:

[0021] First, the human-machine intelligent object transfer problem is described as a Markov decision problem. Let a be the action performed by the agent at each time step. t The corresponding state is:

[0022] s t =[θ,d,d min ,K i [P,Q]

[0023] In the formula: θ is the current joint angle of the robot; d is the minimum distance between the bounding boxes of the human hand and each joint of the robot; d min K represents the minimum value among the minimum distances d between the joints of the human hand and the robot in the current state; i For d min The corresponding joint number; P is the position of the robotic arm's end effector, including the coordinates of x, y, and z; Q is the robot's end effector posture, represented by quaternions x, y, z, and w.

[0024] Next, construct the reward value function, with action a. t =[Δθ,ΔP,ΔQ] Reward value R k The solution can be obtained using the following formula:

[0025] R k =r k +γr k-1 +…+γ N-k r N

[0026] In the formula: γ∈[0,1] is the decay factor; the reward for the k-th step is:

[0027]

[0028] Where s is the current state; P0 is the end-effector pose when the robotic arm reaches the transfer state; t0 represents the current position of the human hand's end effector; t0 represents the moment the robot reaches the transfer position. Indicates the current time, m e and m k and represent the weights of the human hand's end-effector position and the timing of the transfer in the reward function, respectively. During the dataset construction process, the receiver rates the human hand's end-effector pose and the robot's transfer timing using a given Likert rating scale. This rating, quantified, is m. e and m k The value;

[0029] The improved DDPG algorithm enhances receiver posture and delivery timing. Since the reward value function is the human rating of delivery timing under different delivery postures, the trained delivery timing selection model will gradually converge toward the delivery timing with the highest score.

[0030] When the robot control module calls the timing selection model to control the robot's object transfer process, it combines multimodal fusion proximity perception and, after completing the state assessment, uses the achieved hand posture as input to generate the optimal control time for the robot's gripper to open, thus enabling the robot to actively transfer objects.

[0031] This invention also proposes a robotic intelligent item delivery system for human-robot collaboration, the system comprising a hierarchical human hand proximity sensing module, a system digital twin module, an active instruction delivery module, a robot control module, and a data acquisition module; wherein,

[0032] The hierarchical human hand proximity sensing module is used to perform multimodal sensing of the proximity of human hands during the process of the robot passing items to a human.

[0033] The system's digital twin module is used to assess the arrival status of the hand and provide training material for the model that selects the timing of the delivery.

[0034] The active instruction transmission module is used to output transmission trigger node instructions based on the input hand posture;

[0035] The robot control module is used to control the robot to move to the contact point and trigger the node to complete the transfer task according to the output transfer command;

[0036] The data acquisition module is used to provide empirical data on the timing of interpersonal object transfers during neural network training, and also to provide quantitative data for the construction of the reward value function.

[0037] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:

[0038] By utilizing a constructed robotic hand covered with a proximity capacitance sensor and a robot's vision sensor, a hierarchical real-time human hand proximity perception method is established based on human hand visual recognition algorithms and proximity capacitance change patterns. Simultaneously, by constructing a digital twin system of the object being transferred, the robot, and the human hand, the proximity status of the human hand is evaluated in real time. Furthermore, by collecting the transfer response time and scores for different objects and different human hands during human-to-human object transfer, an improved DDPG algorithm is designed and trained using the scores within the dataset as a reward function to obtain an intelligent transfer model. This model formalizes human-to-human object transfer experience and digitizes the evaluation of the object transfer process, enabling the robot to possess autonomy during object transfer.

[0039] This method digitizes the assessment of human hand proximity. Compared to traditional passive triggering methods such as voice, this method enables robots to proactively initiate actions, thus improving the human-computer interaction experience. Compared to traditional methods based on visual recognition of hand movement trajectories, this algorithm combines proximity capacitance sensing to compensate for potential visual obstruction at the proximity end, significantly enhancing the system's robustness.

[0040] Compared with traditional force sensor-triggered transmission, the reinforcement learning network designed in this invention introduces human evaluation of the timing of transmission. That is, the transmission model trained by the robot using this network can enable it to actively choose the timing of transmission that satisfies most people, which can greatly improve the robot's interactive experience. Attached Figure Description

[0041] Figure 1 A flowchart of a robot intelligent item transfer method for human-robot collaboration provided in an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram of the intelligent sensing robotic arm structure in the robot intelligent item transfer method for human-robot collaboration according to an embodiment of the present invention;

[0043] Figure 3 This is an algorithm structure diagram of a robot intelligent item delivery method for human-robot collaboration provided in an embodiment of the present invention;

[0044] Figure 4 This is a block diagram of a robot intelligent item delivery method system for human-machine collaboration provided in an embodiment of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0046] In this invention, the terms "first," "second," etc. (if present) in the invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0047] Figure 1 This is a flowchart illustrating a robot-based intelligent item delivery method for human-robot collaboration, provided as an embodiment of the present invention. (See also...) Figure 1 , combined Figures 2-3 The transmission method in this embodiment will be described in detail. The method includes steps S1-S3.

[0048] Step S1: Construct a robotic hand with proximity and pressure sensing, and integrate vision for multimodal perception to detect the approach of a human hand in real time.

[0049] According to an example of the present invention, step S1 includes steps S11-S13.

[0050] In step S11, a robotic hand structure is constructed using metallic materials such as copper foil and silver foil as the contact capacitance sensing electrode material, a piezoelectric material as the pressure sensor, and a vision sensor. A multimodal human hand proximity perception algorithm, fusion of capacitance change and machine vision, is employed. Specifically, refer to... Figure 2 .

[0051] In step S12, when a human hand approaches, a visual algorithm is used to estimate the distance between them. Let d1 represent the distance calculated based on the visual algorithm, according to the formula d1=inf{‖xy‖2:x∈traA(t),y∈traB(t)}, where traA(t) and traB(t) are the trajectories of human hand A and object (or gripper) B within the time interval t[t0, t1], respectively. The real-time distance between the human hand and the robotic arm is then calculated. Let d2 represent the distance between the human hand and the robotic arm based on proximity capacitance testing, based on changes in proximity capacitance, referring to the equation. Where ∈ is the dielectric constant, k is the electrostatic constant, S is the area of ​​the coated plate, and C is the capacitance value detected in real time. Solve for d2.

[0052] Specifically, in step S13, the results of human hand proximity perception are optimized according to the multimodal fusion method. A hierarchical fusion method is constructed based on vision and proximity sensing using the following formula to optimize the accuracy of the distance between the human hand and the robot gripper, where d refers to the solved distance between the human hand and the robot gripper.

[0053]

[0054] Step S2: Construct a virtual bounding box for the human hand, the object being grasped, and the robotic arm (including the robotic hand), and establish the spatial relationship between the object being grasped and each joint of the robotic arm according to the forward kinematics of the robotic arm. In this way, a digital twin system of the human hand, the object being grasped, and the robotic arm is constructed to evaluate in real time whether the human hand has reached the transfer point.

[0055] According to an example of the present invention, step S2 includes step S21.

[0056] Step S21: Assess the hand's arrival status based on the distance relationship between the grasped object and the hand within the digital space. Specifically, calculate the distance relationship between the hand's bounding box and the grasped object using the formula for d1, and determine whether the transfer point has been reached based on the distance and dwell time.

[0057] In this embodiment, step S3 involves collecting data based on the receiver's pose at the transfer point and the timing of the transferor releasing the object during the person-to-person transfer of different objects, using these as core indicators. A dataset is then constructed, which can be used to train the subsequently developed reinforcement learning algorithm to obtain a human-like transfer timing selection model. (Reference) Figure 3 The intelligent transmission algorithm is implemented using a human hand pose recognition module, a DDPG module, and a robot control module.

[0058] Specifically, in the hand pose recognition module, during a pre-experiment involving passing objects between people, the module identifies the recipient's pose when they stop at the grasping point. Timing begins at the moment the recipient's hand stops moving, and the time it takes for the passer to release the object is recorded to construct a dataset. Simultaneously, after each pass, the recipient evaluates the timing of the pass; this rating serves as the correlation coefficient for the reward function in subsequent reinforcement learning training.

[0059] Specifically, the DDPG module includes an improved DDPG algorithm and a reward function. The reward function is the score given by the receiver regarding the timing of the transfer during the dataset construction process. The main function of the DDPG module is to reinforce the receiver's posture and the timing of the transfer using a reinforcement learning algorithm. Since the reward function is the human score for the timing of the transfer under different postures, the trained model for selecting the timing of the transfer will gradually converge towards the timing with the highest score.

[0060] The DDPG module contains an improved DDPG algorithm and a reward value function; the improved DDPG algorithm includes the following steps:

[0061] First, the human-machine intelligent object transfer problem is described as a Markov decision problem. Let a be the action performed by the agent at each time step. t The corresponding state is:

[0062] s t =[θ,d,d min ,K i [P,Q]

[0063] In the formula: θ is the current joint angle of the robot; d is the minimum distance between the bounding boxes of the human hand and each joint of the robot; d min K represents the minimum value among the minimum distances d between the joints of the human hand and the robot in the current state; i For d min The corresponding joint number; P is the position of the robotic arm's end effector, including the coordinates of x, y, and z; Q is the robot's end effector posture, represented by quaternions x, y, z, and w.

[0064] Next, construct the reward value function, with action a. t =[Δθ,ΔP,ΔQ] Reward value R kThe solution can be obtained using the following formula:

[0065] R k =r k +γr k-1 +…+γ N-k r N

[0066] In the formula: γ∈[0,1] is the decay factor; the reward for the k-th step is:

[0067]

[0068] Where s is the current state; P0 is the end-effector pose when the robotic arm reaches the transfer state; t0 represents the current position of the human hand's end effector; t0 represents the moment the robot reaches the transfer position. Indicates the current time, m e and m k and represent the weights of the human hand's end-effector position and the timing of the transfer in the reward function, respectively. During the dataset construction process, the receiver rates the human hand's end-effector pose and the robot's transfer timing using a given Likert rating scale. This rating, quantified, is m. e and m k The value; specifically, the robot control module directly calls the transfer timing selection model to control the robot's item transfer process. Specifically, combined with the multimodal fusion proximity perception of this scheme, after completing the arrival state assessment, the reached hand posture is used as input to generate the optimal control time for the robot's gripper to open, realizing the robot's active item transfer.

[0069] Figure 4 This is a block diagram of a robotic intelligent goods delivery system for human-robot collaboration, provided in an embodiment of the present invention. (See also...) Figure 4 The robot intelligent item delivery system 400 for human-robot collaboration includes a hierarchical human hand proximity sensing module 410, a system digital twin module 420, an active instruction transmission module 430, a robot control module 440, and a data acquisition module 450.

[0070] The hierarchical hand proximity sensing module 410, for example, performs operation S1 to perform multimodal sensing of the proximity state of the human hand during the process of the robot passing items to the human.

[0071] The system's digital twin module 420, for example, performs operation S2 to assess the arrival status of the hand and provide material for training the intelligent delivery model.

[0072] The active instruction transmission module 430, for example, performs operation S3 to output a transmission trigger node instruction based on the input hand posture.

[0073] The robot control module 440, for example, performs operation S3 to control the robot to move to the contact point and trigger the node to complete the transfer task according to the output transfer instruction.

[0074] The data acquisition module 450 performs operations S2 and S3, for example, to provide empirical data on the timing of interpersonal object transfers during neural network training, and to provide quantitative data for the construction of the reward function.

[0075] The robotic intelligent goods delivery system 400 for human-robot collaboration is used to perform the above-mentioned tasks. Figures 1-3 The illustrated embodiment presents a method for intelligent transfer of objects between robots and humans. For details not covered in this embodiment, please refer to the foregoing. Figures 1-3 The robot intelligent item delivery algorithm for human-robot collaboration shown in the embodiment will not be described in detail here.

[0076] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for intelligent robot item delivery for human-robot collaboration, characterized in that, The method includes the following steps: S1, construct a robotic hand with proximity and pressure sensing, and integrate vision for multimodal perception to detect the approach of a human hand in real time; S2, based on the virtual bounding box of objects, human hands and robots, constructs a digital twin system of human and robot in the process of object transfer. This digital twin system can evaluate the arrival status of human hands in real time based on the visually updated position of human hands and the position of grasped objects updated based on robot kinematics, as well as the spatial geometric relationship between the two. S3. Data is collected based on the core indicators of the receiver's pose at the transfer point and the timing of the transferor releasing the object during the transfer of different objects between people. The core indicator data includes three core indicators: the position of the human hand based on visual computing, the proximity state of the human hand based on capacitive variable sensing, and the time the human hand stays at the position. A dataset is constructed based on this data. An improved DDPG algorithm is developed based on Deep Deterministic Policy Gradient (DDPG). The constructed dataset is used to train a human-like transfer timing selection model. This model enables the robot to actively select the transfer timing. In step S3, the intelligent object transfer algorithm is implemented using the human hand pose recognition module, DDPG module, and robot control module; The human hand pose recognition module is used to identify the pose of the receiver when it stops at the grasping point. The timing starts from the moment the receiver's hand stops moving, the time the transmitter releases the object is recorded, and the time the receiver's hand stays at that position is calculated to build a dataset. The DDPG module contains an improved DDPG algorithm and a reward value function; the improved DDPG algorithm includes the following steps: First, the human-machine intelligent object transfer problem is described as a Markov decision problem. Let a be the action performed by the agent at each time step. t The corresponding state is: In the formula: θ is the current joint angle of the robot; d is the minimum distance between the bounding boxes of the human hand and each joint of the robot; d min K represents the minimum value among the minimum distances d between the joints of the human hand and the robot in the current state; i For d min The corresponding joint number; P is the position of the robotic arm's end effector, including the coordinates of x, y, and z; Q is the robot's end effector posture, represented by quaternions x, y, z, and w. Next, construct the reward value function, with the action being... Reward value R k The solution can be obtained using the following formula: In the formula: γ∈[0,1] is the decay factor; the reward for the k-th step is: Where s is the current state; P0 is the end-effector pose when the robotic arm reaches the transfer state; t0 represents the current position of the human hand's end effector; t0 represents the moment the robot reaches the transfer position. Indicates the current time, m e and m k and represent the weights of the human hand's end-effector position and the timing of the transfer in the reward function, respectively. During the dataset construction process, the receiver rates the human hand's end-effector pose and the robot's transfer timing using a given Likert rating scale. This rating, quantified, is m. e and m k The value; The improved DDPG algorithm enhances receiver posture and delivery timing. Since the reward value function is the human rating of delivery timing under different delivery postures, the trained delivery timing selection model will gradually converge toward the delivery timing with the highest score. When the robot control module calls the timing selection model to control the robot's object transfer process, it combines multimodal fusion proximity perception and, after completing the state assessment, uses the achieved hand posture as input to generate the optimal control time for the robot's gripper to open, thus enabling the robot to actively transfer objects.

2. The method according to claim 1, characterized in that, S1 includes: selecting suitable proximity capacitor plate material and piezoelectric material, and designing a robotic arm to achieve the coverage of the fingers by the proximity capacitor plate material. The piezoelectric material is attached to the surface of the capacitor plate material as a pressure sensor. The proximity capacitance will change when the human hand approaches. The proximity state of the human hand is realized in real time and multimodal perception by combining with a visual sensor.

3. The method according to claim 2, characterized in that, The S1 includes: a robotic hand structure covered with a metal material as the electrode material for proximity capacitance sensing, and equipped with a visual sensor, and a multimodal human hand proximity sensing algorithm that integrates capacitance change and machine vision algorithm. In the multimodal human hand proximity perception algorithm, the distance between the human hand and the robot gripper is calculated using the following formula, where d refers to the calculated distance between the human hand and the robot gripper, d1 refers to the human hand distance calculated based on the visual algorithm, and d2 refers to the human hand distance based on the proximity capacitance test. In visual algorithms, according to the formula Where traA(t) and traB(t) are the trajectories of hand A and object or gripper B within the time interval t [t0, t1], respectively, the real-time distance between the hand and the robotic arm is calculated; based on the change in proximity capacitance C, the equation is referenced. ,in Let be the dielectric constant, k be the electrostatic constant, and S be the area of ​​the coated plate. Solve for d2.

4. The method according to claim 3, characterized in that, S2 includes: constructing a virtual bounding box for the human hand, the object being grasped, and the robotic arm containing the robotic hand; establishing the spatial relationship between the object being grasped and each joint of the robotic arm according to the forward kinematics of the robotic arm; thereby constructing a digital twin system of the human hand, the object being grasped, and the robotic arm; the calculation formula is the same as the calculation formula for d1 in step S1; solving the distance relationship between the bounding box of the human hand and the object being grasped; and evaluating the arrival state of the human hand according to the distance relationship between the object being grasped and the human hand in the digital space.

5. A transfer system for a robot intelligent item transfer method for human-robot collaboration according to any one of claims 1-4, characterized in that, The system includes a hierarchical hand proximity sensing module, a system digital twin module, an active command transmission module, a robot control module, and a data acquisition module; among which... The hierarchical human hand proximity sensing module is used to perform multimodal sensing of the proximity of human hands during the process of the robot passing items to a human. The system's digital twin module is used to assess the arrival status of the hand and provide training material for the model that selects the timing of the delivery. The active instruction transmission module is used to output transmission trigger node instructions based on the input hand posture; The robot control module is used to control the robot to move to the contact point and trigger the node to complete the transfer task according to the output transfer command; The data acquisition module is used to provide empirical data on the timing of interpersonal object transfers during neural network training, and also to provide quantitative data for the construction of the reward value function.

Images