Control method for autonomous navigation of dual-arm care robot for manned holding and transfer

Abstract

This invention provides a control method for autonomous navigation and human-carrying transport using a dual-arm nursing robot. A musculoskeletal biomechanics dataset is constructed, enabling an LSTM neural network to predict human muscle activation. Combined with a human-machine contact biomechanics model, a human-machine contact comfort prediction model is built to determine the optimal holding posture parameters. Based on this, the Cartesian coordinates of the target contact points on the robot's left and right forearms are calculated. During the lifting process, Openpose continuously monitors the posture angles of the simplified four-link human body, ensuring the human body returns to the planned posture. A target detection algorithm identifies the target carrier in real time. After obtaining the current posture of the person being cared for, the robot chassis autonomously navigates from its current position to a distance in front of the target carrier. Placement is then performed, and successful placement is verified by visually detecting the distance between key points on the human body and the end effector of the robotic arm. This achieves a closed-loop process from human body recognition, compliant lifting, autonomous navigation to precise placement of the target carrier.

Description

Technical Field

[0001] This invention relates to the field of autonomous mobile robots and nursing transport equipment, specifically a control method for autonomous navigation and human-carrying transport of a dual-arm nursing robot. Background Technology

[0002] Currently, significant progress has been made both domestically and internationally in the fields of humanoid care robots and carrying / transfer technologies. However, research on dedicated transfer robots that integrate human carrying and autonomous navigation is still in its early stages. In terms of configuration design, robots represented by Japan's RIBA / Robear and Germany's GARMI have achieved high-load physical care functions, capable of actions such as picking up and turning over. However, their configurations primarily focus on flexible carrying with two arms, lacking systematic design and safety mechanism research for tightly coupled human-machine interactions. Domestically, robots from companies like UBTECH and Logic have made breakthroughs in arm coordination and walking stability, but still have shortcomings in active compliant carrying, dynamic center of gravity adjustment, and precise control of human-machine contact forces in human transfer scenarios. At the level of multimodal perception and interactive control, existing research mainly relies on the fusion of vision and force. While tactile electronic skin and whole-body distributed sensor arrays have been initially applied to human-machine contact detection, a closed-loop control framework integrating multi-source information from "human-machine-environment" has not yet been formed for carrying / transfer tasks. Especially in close human-machine interaction, how to deeply integrate tactile, force, and visual-language-motion models to achieve coordinated control of autonomous navigation, obstacle avoidance, and human-carrying operations remains a current technological bottleneck. Furthermore, existing robot systems have significant limitations in cross-task skill transfer and the reliability of active perception and decision-making under occlusion conditions, making it difficult to meet the comprehensive requirements of high safety, high comfort, and high autonomy in elderly care and medical transport scenarios.

[0003] Therefore, the systematic research on manned carrying and transport robots is becoming an important direction for the evolution of humanoid robots from single-function execution to embodied intelligent care systems. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the technical problem this invention aims to solve is to provide a control method for autonomous navigation and human-carrying transport of a dual-arm nursing robot.

[0005] The technical solution adopted by the present invention to solve the aforementioned technical problem is as follows: In a first aspect, the present invention provides a control method for autonomous navigation and human-carrying transport of a dual-arm nursing robot, wherein the control method comprises: The constructed musculoskeletal biomechanics dataset incorporates the activation levels of the sternocleidomastoid, external oblique, gluteus maximus, and rectus femoris muscles into an LSTM neural network. The LSTM neural network is then trained to predict the activation levels of human muscles. Then, a comfort prediction model for holding a dual-arm nursing robot is constructed by combining the human-machine contact mechanics model. The comfort experience in the holding state is analyzed, and the optimal holding posture parameters are determined, which are denoted as the planned posture. Based on the optimal holding posture parameters, and combined with the real-time three-dimensional position of the human body key points in the robot base coordinate system, the Cartesian coordinates of the target contact points of the robot's left forearm and right forearm are calculated. In the control of the lifting process, the lifting trajectory is divided into three stages: pre-tensioning stage, lifting stage, and hovering stage. During the lifting process, Openpose continuously monitors the posture angles (θ2, θ3) of the simplified four-bar linkage of the human body and compares them with the planned values. If the actual posture deviates from the planned value by more than ±5°, the target posture of the ends of the arms is readjusted through inverse kinematics to restore the human body to the planned posture. Target carriers are identified in real time using target detection algorithms; After obtaining the current position of the patient, the robot chassis autonomously navigates from its current position to a distance in front of the target carrier and maintains the preparatory position. In the navigation of the dual-arm nursing robot chassis, a combination of global path planning and local DWA obstacle avoidance is used for navigation. The placement process then proceeds in three stages: descent, release, and retraction. Once the retraction is complete, the successful placement is verified by visually detecting the distance between key points on the human body and the end effector of the robotic arm.

[0006] Furthermore, the process of determining the optimal holding posture parameters is as follows: the patient's height, weight, body mass index (BMI), sitting height, and shoulder width are constructed into a patient condition vector. The conditional variational autoencoder (CVAE) is used to learn the expert teaching trajectory collected by teleoperation, generating a complete transfer trajectory adapted to the patient's individual characteristics. The robot learns from the expert teleoperation teaching data and automatically establishes a mapping relationship between "patient individual parameters → complete transfer trajectory", thereby obtaining a continuous, smooth, and human-like transfer trajectory.

[0007] Furthermore, the conditional variational autoencoder (CVAE) includes a conditional coding network, a trajectory encoder, a conditional prior network, and a trajectory decoder. The patient's conditional vector is simultaneously introduced into the trajectory encoder, the conditional prior network, and the trajectory decoder to model the distribution of individualized transfer actions. Using five individual parameters of the patient—height, weight, BMI, sitting height, and shoulder width—as input, the parameters are first mapped to a high-dimensional conditional embedding through a conditional encoding network. Then, in the offline training phase, the trajectory encoder learns the motion distribution characteristics under different patient conditions from the teleoperation teaching trajectory and, together with the conditional prior network, constrains the latent space, so that the latent variable z can compactly represent the individualized motion style. In the online inference phase, only the patient condition vector of the new patient needs to be input into the conditional prior network to sample the implicit expression of the motion adapted to the patient from the latent space. Then, the trajectory decoder reconstructs the complete and continuous sequence of joint and end-effector poses step by step through the time step, and finally generates an individualized prior trajectory that inherits the humanoid coordination of expert teaching and precisely matches the current patient's body shape in terms of spatial scale and motion rhythm.

[0008] Furthermore, in the chassis navigation of the dual-arm nursing robot, the tactile constraint physical model is used to predict the changing trends of ZMP, contact area, pressure center and peak contact force in the short time domain, thereby enabling the active correction of the pose of the two arms to meet future stability trends.

[0009] Furthermore, the tactile constraint physical model is a ZMP-based tactile constraint physical model, and the active correction process is as follows: A comprehensive contact state vector is constructed based on the robot's joint motion state, end-effector pose, tactile observation, and individual patient conditions. Contact force information acquired by distributed tactile sensors f i and its location r i Real-time calculation of contact resultant force F t Effective contact area A t Contact pressure center m t and zero torque point z t ; Finally, the current state s t Recent historical status and planned action sequence a t:t+H-1 Common input multi-step sequence prediction network based on Transformer architecture fψ ( This study utilizes a self-attention mechanism to model the cross-temporal coupling relationship between actions, contact states, and stability, thereby enabling the model to be applied in the future short-term domain. H Internally, the evolution trends of ZMP, contact resultant force, contact area, contact pressure center, and end-effector pose are jointly predicted. This provides a basis for decision-making regarding subsequent forward-looking online trajectory deformation; After obtaining the predicted values ​​of ZMP, contact resultant force, contact area, contact pressure center, peak force, and end pose in the short time domain, a cost function is set to correct the online trajectory deformation. The cost function decomposes the error into a set of directly perceptible and predictable physical quantities: 1) Introducing ZMP error to directly penalize the deviation of the future equilibrium point, with dynamic stability as the highest priority; 2) Introducing peak contact force, contact area, and contact pressure center errors to explicitly elevate the tactile information, which was originally a feedback signal, to the optimization target, enabling the controller to avoid the risks of excessive local pressure, insufficient support, or abnormal load migration in advance; 3) Degrading the prior trajectory from a hard tracking command to a soft constraint, retaining the human-like movement style only under the premise of ensuring the aforementioned physical safety; 4) Introducing a smoothness penalty on the control increment itself to prevent the correction action from being too abrupt or violently jittering, thus obtaining a controller that can foresee action risks and actively seek a safe, smooth, and human-like compromise optimal solution.

[0010] Furthermore, the cost function J t for:

[0011] in, For the desired ZMP, To predict peak contact force, and These are the desired contact area and the desired pressure center, respectively. These are the weighting coefficients for each item; This represents the control action at time t-1; This represents the optimal control action at time t; For prior control input; Indicates the prior end pose; Indicates the predicted ZMP coordinates. Indicates the predicted effective contact area. Indicates the predicted center of pressure. This indicates the predicted end-effector pose.

[0012] Furthermore, the control positions for each stage of the lifting trajectory are as follows: The first stage is pre-tensioning, where the two arms slowly apply a small force to eliminate gaps and slightly compress the flexible layer, allowing the human body to fit against the robot's forearms; the second stage is lifting, where the end effector moves along an S-shaped velocity curve in the vertical direction, with acceleration gradually increasing from 0 and then gradually decreasing, with the peak lifting speed not exceeding 0.1 m / s; the third stage is hovering, where lifting stops when the human's feet are about 5 cm off the ground, maintaining the posture for 1 second to wait for dynamic stabilization. During the lifting process, position and force are controlled simultaneously. The left arm mainly bears the vertical load, using force control to maintain the set support force, while the right arm mainly plays a stabilizing and guiding role, using position control to track the trajectory, and simultaneously adjusting the lateral force through impedance control to prevent the human body from sliding sideways; the force distribution between the two arms is coordinated through a task priority framework: the left arm prioritizes ensuring vertical support force, and the right arm prioritizes ensuring posture stability.

[0013] Furthermore, during the placement process, in the descent phase, the end effector moves along an S-shaped speed curve in the vertical direction. When the end effector approaches the upper part of the carrier contact surface, it switches to force control mode. The actual contact force is estimated by the joint torque sensor, and the end effector height is adjusted by the PI force controller. The desired contact force is set. Once the contact force reaches the set value and lasts for 0.5 seconds, it is determined that "stable contact has been achieved". In the release phase, the support force of the two arms decreases linearly to 0 within 2 seconds. At the same time, the tactile sensors of the two arms monitor the symmetry of the pressure distribution. If the pressure difference between the left and right arms exceeds 30%, the process pauses and the chassis position is finely adjusted. In the retraction phase, the robotic arm slowly opens outward to separate the forearm from the human body, then lifts upward and retracts to a safe folded posture.

[0014] Secondly, the present invention provides a dual-arm nursing robot, wherein the robot performs the aforementioned control method.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention provides a dual-arm nursing robot that integrates anthropomorphic dual-arm coupled drive joints, a large-area flexible array tactile skin, and a vision-language-tactile-motion (VLTA) multimodal model, enabling autonomous navigation and human-carrying transport.

[0016] This invention achieves personalized optimal holding posture planning for different body types by using LSTM-based real-time comfort prediction and five-dimensional parametric mesh search (taking human-machine contact position (thigh contact distance d1, back contact distance d2) and human posture angles (thigh-horizontal plane angle θ2, back-horizontal plane angle θ3, human tilt angle α) as input and outputting the activation levels of four core muscle groups: sternocleidomastoid, external oblique, gluteus maximus, and rectus femoris (comfort is judged based on activation levels and human-machine contact mechanics model). It integrates YOLOv5s target detection and DWA local path planning (YOLOv5s is first responsible for the perception stage, identifying and outputting the bounding box of the target carrier, and combining depth information to calculate its three-dimensional position in the world coordinate system). The pose information, including the orientation angle, is then passed to the DWA local path planner as a navigation target. Based on this, and guided by the global path, the DWA samples and generates an unobstructed local motion trajectory in real time within the robot's velocity space, driving the robot chassis to autonomously and safely reach the target point. This achieves a closed-loop process from human recognition, compliant picking up, autonomous navigation to precise placement of the target carrier. Compared with existing nursing robots, this invention achieves breakthrough improvements in three core indicators: holding comfort, transfer safety, and operational autonomy. It effectively solves prominent problems in traditional solutions such as high contact impact, uncomfortable posture, and reliance on manual operation, providing a safe, comfortable, and intelligent robotic transfer solution for elderly care scenarios. Attached Figure Description

[0017] Figure 1 Simplified diagram of a human body linkage; Figure 2 This is a schematic diagram of the 17 links in the human body.

[0018] Figure 3 This is a schematic diagram of chessboard image acquisition.

[0019] Figure 4 Schematic diagram of robot coordinate system transformation.

[0020] Figure 5 Schematic diagram of human holding posture parameters.

[0021] Figure 6 A schematic diagram of the remotely operated equipment.

[0022] Figure 7 This is a schematic diagram of the structure of a conditional variational autoencoder (CVAE) according to an embodiment of the present invention.

[0023] Figure 8 This is a schematic diagram of an online trajectory deformation correction process according to an embodiment of the present invention. Detailed Implementation

[0024] Specific embodiments of the present invention are given below. These specific embodiments are only used to further illustrate the present invention and do not limit the scope of protection of the claims of this application.

[0025] This invention utilizes a Long Short-Term Memory (LSTM) network to predict the activation levels of key muscle groups (sternocleidomastoid, external oblique, gluteus maximus, and rectus femoris) in real time. An LSTM model is trained based on an offline-constructed musculoskeletal biomechanics dataset and combined with a human-machine contact biomechanics model (local pressure) to construct a comfort prediction model for a dual-arm nursing robot. This model can quickly evaluate comfort scores under different holding posture parameters (human-machine contact position, human posture angle), thereby optimizing and determining the optimal holding posture online. This significantly reduces the time cost of traditional simulation calculations and provides a feasible path for real-time comfort control.

[0026] This invention addresses the issues of information redundancy and high computational complexity in transfer tasks caused by the 18 keypoints (or 17 simplified links) output by Openpose. It employs a simplified human posture model specifically designed for nursing transfers—using only five keypoints—left ankle, left knee, left hip, neck, and left ear—to form a four-link model. This simplified model retains the main posture features of the human body during transfer (such as the relative angles of the torso, thigh, and calf) while significantly reducing the computational complexity of subsequent obstacle avoidance algorithms and motion planning. Simultaneously, it completes the calibration of the Kinect camera (internal and external parameters), the unified calibration of the robot's base coordinate system and the world coordinate system, and establishes a kinematic model of the human-machine-environment system, realizing the coordinate representation of human keypoints, robot end effectors, and obstacles in a unified space.

[0027] This invention organically integrates the YOLOv5s target detection algorithm (for identifying placement targets such as chairs and sofas) with the Openpose human pose estimation algorithm (for identifying the posture of the person being cared for), constructing a closed-loop task framework of "human body recognition → picking up action → finding target location → putting down human body action". This enables the robot to perform fully autonomous operations in complex home environments: first, it visually perceives key points of the human body and plans a compliant holding trajectory; after picking up the human body, it dynamically re-identifies the target location (such as a wheelchair or sofa); combining a unified kinematic model (a model that places the robot, environment, and caregiver in the same world coordinate system after coordinate transformation) to complete chassis path planning and robotic arm placement trajectory generation; finally, it achieves stable placement under force / position hybrid control.

[0028] This application does not explicitly rely on complex end-to-end large models for skill transfer. Instead, it achieves cross-scenario task adaptation through modular, reusable algorithmic components and a unified multi-coordinate system framework. A unified "human-machine-environment" coordinate representation: Through a multi-coordinate system calibration method, human keypoints, robot end effectors, and environmental targets are uniformly mapped to the world coordinate system. This allows the two different task stages of "hugging" and "releasing" to share the same set of human pose estimation (Openpose) and target detection (YOLOv5s) perception results, realizing inter-task reuse of the perception modules (i.e., human pose estimation and target detection functions). A general comfort prediction model: The dual-arm nursing robot's holding comfort prediction model, built based on an LSTM neural network, is deeply optimized for the core action of holding. The optimal holding pose parameters it searches for can serve as key intermediate states in different scenarios, providing a general target for motion planning, thereby reducing the complexity of reprogramming for different tasks. Generalized application of classic navigation algorithms: The combination of A* global path planning and DWA local obstacle avoidance algorithm is a mature and universal navigation framework in the field of mobile robots, which has the ability to migrate across different indoor environments.

[0029] This application integrates a large-area flexible tactile sensor array into the forearm of a robotic arm to acquire tactile information such as contact pressure distribution and pressure center in real time. These tactile signals are unaffected by visual occlusion and can directly reflect the human-machine contact state, providing the most direct decision-making basis for force / position hybrid control. Simplification and dynamic tracking of human posture: After acquiring key points of the human body through Openpose, it is further simplified into a four-bar model. Even if part of the human body is occluded during transport, as long as the key points are captured, the system can still estimate the main human posture. More importantly, the system dynamically recalculates the target pose at a frequency of 50ms, achieving dynamic tracking of the target and reducing task failures caused by brief occlusion or movement.

[0030] This application features forward-looking predictive control: within a higher-level framework, a ZMP-based tactile constraint physical model predicts the ZMP and contact force in the short-term time domain and performs online trajectory deformation correction. This closed-loop control of "prediction + correction" upgrades decision-making from a passive "response" to an active "predictive" approach, significantly improving contact stability. Phased, multi-constraint safe execution: In the execution of picking up and placing actions, phased control, force / position hybrid control, kinematic constraints, and force threshold monitoring are introduced. This multi-level safety constraint and anomaly handling mechanism provides a fundamental safety guarantee for the reliable execution of decisions.

[0031] The present invention provides a control method for autonomous navigation, human-carrying, and transporting dual-arm nursing robots. It establishes a unified framework to enable the robot to autonomously track and locate the human body based on its position. Regardless of the location of the person being cared for, the robot needs to actively locate the person's position and move autonomously to pick up the person and place them in the designated location. The main process flow is described below.

[0032] The first step is to visually identify human joints. During this process, the robot needs to acquire the coordinates of the human joints and the end effector of the robotic arm in real time. To ensure the safety of the person being cared for, the human posture linkages are simplified and incorporated into the human-machine-environment system kinematic model. The Openpose algorithm is one of the most effective algorithms for identifying human posture joints and simplified human linkages (its principle is to first extract feature maps (FFs) using VGG-19; then proceed through multiple parallel branches: branch 1: predicting the keypoint confidence map (the probability of each joint's position); branch 2: predicting the joint affinity domain (PAF) to determine the connection relationship between keypoints; finally, generating a complete human skeleton through bipartite graph matching (Hungarian algorithm)). Simplified human posture linkages can be obtained from the correct connections of human posture keypoints. Human posture keypoint models can be divided into three types: 17 keypoints, 18 keypoints, and 25 keypoints.

[0033] Since the original 18 key points could form 17 links, but this resulted in information redundancy and high computational cost, this invention chose a simplified model using only 5 key points: left ankle, left knee, left hip, neck, and left ear, to form a four-link model (see [link name]). Figure 1 This simplified model is sufficient to represent the main postures of the human torso, thighs, and calves during the transfer process, while reducing the computational burden of subsequent motion planning.

[0034] The second step is to obtain coordinates and calibrate multiple coordinate systems.

[0035] The workspace of the nursing robot involves four coordinate systems.

[0036] Transformation link: pixel coordinates ( u , v ) + depth z c → Coordinate points in the camera coordinate system ( x c , y c , z c → Coordinates of a point in the world coordinate system x w , y w ,z w → Coordinate point O in the robot's base coordinate system r ( x r ,y r ,z r ).

[0037] I. Pixel coordinates → Camera coordinate system 1. Obtain depth value The Kinect camera simultaneously outputs a color image (1920×1080) and an aligned depth image (each pixel corresponds to a depth value in mm). For the pixel coordinates (u,v) of a human keypoint detected by Openpose, the depth value z of that point is read from the depth image. c (That is, the Z coordinate of the point in the camera coordinate system).

[0038] 2. Pinhole camera model Let the camera focal length be... f (Unit: mm), pixel physical size is dx , dy (mm / pixel), then:

[0039]

[0040] Intrinsic parameter matrix:

[0041] Where c x ,c y These are the pixel coordinates of the image center (optical axis projection point).

[0042] 3. Conversion Formula From pixel coordinates = ( u , v ) and depth z c To obtain the points in the camera coordinate system:

[0043]

[0044] Where z c For depth values:

[0045] Solve for x c y c The coordinate transformation can then be completed.

[0046] II. Camera Coordinate System → World Coordinate System 1. Camera extrinsic parameter calibration The camera is fixedly mounted in front of the robot's workspace, and the world coordinate system is defined as a corner point on a checkerboard calibration board (the 6th orange corner point from the top is the origin). O w (Z-axis points forward of the robot, Y-axis points vertically upward) Multiple chessboard images with different poses were acquired using the traditional calibration method (Zhang Zhengyou calibration method). Figure 3 As shown), the extrinsic parameters are solved using OpenCV's calibrateCamera() function:

[0047] Where R is a 3×3 rotation matrix and T is a 3×1 translation vector, representing the transformation from the world coordinate system to the camera coordinate system.

[0048] 2. Coordinate Transformation Given a point Pc=(x) in the camera coordinate system c ,y c ,z c Find its coordinates P in the world coordinate system according to the following formula. w= (x w ,y w ,z w )

[0049] 3. Actual calibration process 1. Use a standard black and white checkerboard (size 13×9, side length 42mm), with a total of 96 corner points.

[0050] 2. Select three specific corner points on the chessboard to define the axes of the world coordinate system: Origin (Ow): the 6th orange corner point from the top. Z-axis point: the 3rd orange corner point from the top (the line connecting it to the origin is the Z-axis). X-axis point: the 6th blue corner point from the top (the Y-axis is determined according to the right-hand rule).

[0051] 3. Detect corner points using OpenCV's findChessboardCorners() function, and then call solvePnP() or calibrateCamera() to obtain extrinsic parameters.

[0052] III. World Coordinate System → Robot Base Coordinate System Since the origin of the nursing robot's base coordinate system only moves relative to the X and Y axes of the world coordinate system, it is only necessary to calculate the relative position of the origin of the nursing robot's base coordinate system in the world coordinate system at the initial state. Then, the movement distance of the Mecanum wheel in the X and Y axes is superimposed with the initial position, and finally, the spatial position of the nursing robot's base coordinate origin in the selected fixed world coordinate system can be obtained in real time.

[0053] like Figure 4 As shown, the coordinates of the origin of the nursing robot's base coordinate system in the world coordinate system at its initial position are (x... i ,y i ,z i When the nursing robot moves, its base coordinate system origin moves only in the x and y directions. Based on the movement information from the Mecanum wheel, the distances relative to its initial position in these two directions during movement can be obtained. Assuming the movement is x... p and y p Finally, we can obtain the position of the origin of the base coordinate system in world coordinates during the movement as (x i +x p ,y i +y p ,z i (The subscript i represents the current position). Based on the standard DH matrix parameters of the nursing robot, the relationship between the robot arm's end effector and the base coordinate system, as well as the relationship between the origin of each joint of the robot arm and the base coordinate system, can be obtained. The origin position of the robot base after movement is O'. r .

[0054] Ultimately, the kinematic information of the nursing robot, the simplified link space information of the patient's posture, and the spatial information of obstacles are all represented in the world coordinate system, thereby establishing a kinematic model of the human-machine-environment system in the same coordinate system.

[0055] The third step is to plan the holding posture and predict comfort.

[0056] The purpose of this step is to determine the optimal contact position between the robot's arms and the human body, as well as the best posture when the human body is picked up, so that the person being cared for feels comfortable and safe throughout the transfer process, while avoiding excessive muscle tension or excessive local pressure caused by improper holding.

[0057] To acquire real-time muscle activation levels during a holding position, an LSTM neural network was constructed based on a musculoskeletal biomechanics dataset. The activation levels of the sternocleidomastoid, external oblique, gluteus maximus, and rectus femoris muscles were incorporated into the network. The LSTM network was then trained to predict the activation levels of these muscles. This LSTM, acting as a real-time predictor, was combined with an existing human-machine contact biomechanics model (responsible for calculating local pressure) to form a dual-arm nursing robot holding comfort prediction model, jointly searching for the optimal holding method in real time. The dual-arm nursing robot holding comfort prediction model was then used to analyze the comfort experience during holding and determine the optimal holding posture parameters.

[0058] The musculoskeletal biomechanics dataset was obtained through specific experiments. Imaging data were collected from ten subjects. SolidWorks was used to register the subjects' skeletal models with standard musculoskeletal models in Anybody software to obtain personalized musculoskeletal biomechanical models. The robot model was then simulated with the personalized musculoskeletal biomechanical models, and the input parameter was defined as the human-machine contact position. d 1, d 2) and human posture angle ( θ 1, θ 2, θ 3, θ 4, α The combination of these factors outputs muscle activation: the activation level of key muscle groups (such as sternocleidomastoid, external oblique, quadratus lumborum, gluteus maximus, rectus femoris, etc.).

[0059] Furthermore, a predictive model for the holding comfort of a dual-arm nursing robot is constructed by combining a human-machine contact mechanics model. The comfort experience during holding is analyzed, and the holding posture parameters are as follows: Figure 5 As shown, this includes human-machine contact position parameters (d1, d2) and human posture parameters (θ1, θ2, θ3, θ4, α). The human-machine contact position parameters (d1, d2) represent the contact positions between the robotic arm and the human thigh and back, respectively. The human posture parameters θ1 represent the angle between the human thigh and lower leg, θ2 represent the angle between the thigh and the horizontal plane, θ3 represent the angle between the back and the horizontal plane, θ4 represent the angle between the head and neck and the horizontal plane, and α represents the human tilt angle.

[0060] Based on the comfort prediction model of the dual-arm nursing robot, the holding posture is adjusted. The contact pressure under the optimal posture is lower than the safety threshold, and the muscle activation is in a low-load state, which provides a safe and comfortable motion reference for subsequent compliant execution.

[0061] The fourth step is to pick them up. Based on the optimal holding pose parameters obtained from the planning, and combined with the real-time 3D positions of the human body's key points in the robot's base coordinate system, the Cartesian coordinates of the target contact points of the robot's left forearm (corresponding to the back of the human thigh) and right forearm (corresponding to the back of the human body) are calculated. Since the human body may move slightly during the robot's approach, the system recalculates the target pose every 50ms to achieve dynamic tracking.

[0062] Regarding path planning and obstacle avoidance during approach, the Dynamic Window Approach (DWA) is a local path planning algorithm based on velocity sampling. Proposed by Fox et al. in 1997, it has been included as one of the default local planners in the ROS robot operating system. DWA, considering kinematic and dynamic constraints such as the robot's maximum / minimum velocity and acceleration, performs multiple sets of uniform sampling in the velocity space, mapping each sampled velocity to a trajectory over a specific time period. Each trajectory is then comprehensively scored, and the optimal velocity command is selected to drive the robot's movement. The comprehensive scoring considers factors such as azimuth deviation, obstacle distance, and movement speed, enabling the robot to quickly generate collision-free local paths in dynamic environments. This algorithm has advantages such as high computational efficiency, good real-time performance, and smooth trajectory generation. In the chassis navigation of the dual-arm nursing robot in this patent, DWA, as the obstacle avoidance planner of the local cost map, is responsible for responding to dynamic environmental changes in real time under the guidance of the global path, ensuring the smoothness and safety of the transfer process.

[0063] The lifting process is controlled in three stages: pre-tensioning (the arms slowly apply a small force to eliminate gaps and slightly press the robot's flexible layer, allowing the human body to fit against the robot's forearms), lifting (the end effector moves along the vertical direction (Z-axis of the world coordinate system) in an S-shaped velocity curve, with acceleration gradually increasing and then decreasing from 0 to avoid impact. The peak lifting speed does not exceed 0.1 m / s), and hovering (when the human's feet are about 5 cm off the ground, the lifting stops, and the posture is maintained for 1 second to wait for dynamic stabilization). During the lifting process, position and force are controlled simultaneously. The left arm mainly bears the vertical load and uses force control to maintain the set support force (approximately 60% of the human body weight). The right arm mainly plays a stabilizing and guiding role, using position control to track the trajectory and simultaneously adjusting the lateral force through impedance control to prevent the human body from sliding sideways. The force distribution between the two arms is coordinated through a task priority framework: the left arm prioritizes ensuring vertical support force, and the right arm prioritizes ensuring posture stability. During the lifting process, Openpose continuously monitors the posture angles (θ2, θ3) of the simplified four-link human body and compares them with the planned values. If the actual posture deviates from the planned posture by more than ±5°, the target posture of the arms at the ends will be readjusted through inverse kinematics to restore the body to the planned posture.

[0064] The fifth step is to find the target location and place it.

[0065] After successfully picking up and transporting the target object to its vicinity, the system activates the target detection (YOLOv5s algorithm) and pose estimation (Openpose) modules. The YOLOv5s algorithm is used to identify target carriers such as wheelchairs, sofas, and nursing beds in real time. For detected targets, the system uses the center pixel coordinates of the bounding box combined with Kinect depth images to convert 2D pixels into 3D points in the camera coordinate system using camera intrinsic parameters. These points are then transformed to the world coordinate system using an extrinsic calibration matrix to obtain the 3D position of the target carrier. Simultaneously, planar fitting is performed on the point cloud within the bounding box to estimate the carrier's frontal orientation angle ψ, providing a basis for the robot's approach direction during placement.

[0066] After obtaining the patient's (the person to be cared for) current pose, the robot chassis autonomously navigates from its current position to a preparatory pose approximately 0.5m ahead of the target using a combination of global path planning (A* algorithm) and local DWA obstacle avoidance. The placement trajectory consists of three phases: descent, release, and retraction. During the descent phase, the end effector moves along an S-shaped velocity curve in the vertical direction. When the end effector approaches the upper surface of the carrier contact surface, it switches to force control mode. The actual contact force is estimated using joint torque sensors, and the end effector height is adjusted using a PI force controller to set the desired contact force. Once the contact force reaches the set value and remains there for 0.5 seconds, it is considered "stable contact." During the release phase, the supporting force of both arms linearly decreases to 0 within 2 seconds. Simultaneously, the tactile sensors of both arms monitor the symmetry of the pressure distribution. If the pressure difference between the left and right arms exceeds 30%, the robot pauses and fine-tunes the chassis position. The robotic arms then slowly open outwards to separate the forearms from the human body, then rise upwards and retract to a safe folded posture. After retraction, the robot verifies successful placement by visually detecting the distance between key points on the human body and the end effector, and issues a voice prompt.

[0067] Throughout the placement process, Openpose continuously monitors the attitude angles (θ2, θ3) of the simplified four-bar linkage of the human body. If the deviation from the planned value exceeds ±8%, trajectory correction is triggered. At the same time, abnormal situations such as contact force exceeding 500N or sudden movement of the carrier will immediately stop and trigger an alarm.

[0068] Example 1 Offline Phase – Individualized Prior Trajectory Generation Based on CVAE In the original hugging process, the hugging posture planning mainly relies on LSTM networks to predict muscle activation and combines it with a human-machine contact mechanics model (a simulation model used to calculate the local pressure at the human-machine contact points) to perform a mesh search, obtaining the optimal hugging posture parameters (such as d1, d2, θ2, θ3, α) at ​​discrete time points. Although this method can optimize key contact points and posture angles, it generates static, point-like posture commands, rather than a continuous, smooth, human-like transfer trajectory (including joint angle sequences, end-effector posture sequences, velocities, etc.). More importantly, it does not explicitly utilize individual patient characteristics (height, weight, trunk size, etc.) to modulate the overall movement morphology—for example, taller patients require a wider hugging span, heavier patients require a slower lifting speed, and different sitting heights can cause vertical shifts in the back support point.

[0069] The patient's height, weight, body mass index (BMI), sitting height, and shoulder width are constructed into a patient condition vector. A conditional variational autoencoder (CVAE) is then used to learn from expert-taught trajectories acquired during teleoperation, generating a complete transfer trajectory (including sequences of joint angles and end-effector poses) adapted to the patient's individual characteristics, rather than just discrete holding pose points. The core idea is to enable the robot to automatically establish a mapping relationship between "patient individual parameters → complete transfer trajectory" by learning from expert teleoperation teaching data.

[0070] The specific process is as follows: 1. Teaching data acquisition and trajectory representation To obtain patient transfer movements with human-like coordination characteristics, a dual-arm telemanipulation device was used to collect expert teaching data. The dual-arm telemanipulation device maintains consistency with the dual-arm nursing robot in joint topology, degree-of-freedom configuration, and operation method, enabling the operator to achieve synchronous control of the robot's arms and torso through natural upper limb and waist coordinated movements. Compared to traditional methods that only record end-effector trajectories, this approach more completely preserves information on joint coordination, torso cooperation, and dual-arm coordination during the teaching process, thus providing a reliable data foundation for subsequent learning of human-like transfer movement patterns. The dual-arm telemanipulation device, such as... Figure 6 As shown, the system includes a support base, a control handle, a isomorphic teleoperated right arm, a hydraulic rod, an isomorphic teleoperated left arm, and a torsion spring. During each teaching session, the device synchronously records the joint angles, joint angular velocities, end-effector pose, and end-effector pose velocity of the robot's two arms and waist, constructing these as temporal trajectory samples. Let the nth teaching trajectory be... Represented as:

[0071] The state vector at a single time t is defined as:

[0072] in, For robot joint angle vectors, The joint angular velocity vector. These are the end-effector pose parameters. The end-effector pose velocity; Let T represent the state vector of the nth teaching trajectory at time T; T represents the length of the teaching trajectory.

[0073] This representation includes both the motion coordination in the joint space (such as arm synchronization and trunk coordination) and the execution characteristics in the task space (such as holding path and lifting trajectory).

[0074] All teaching trajectories are resampled to a uniform length T using cubic spline interpolation, and then standardized so that the mean of each dimension is 0 and the variance is 1, thus eliminating the differences in time length and speed scale between different teaching samples.

[0075] 2. Patient condition coding The patient condition vector is represented as:

[0076] Where h represents the patient's height, w represents their weight, and BMI represents their body mass index. Indicates sitting high, Let represent shoulder width, and 'c' represent the patient condition vector. These parameters are independent of each other and have clear physical meanings, collectively describing the patient's static geometric and mass characteristics.

[0077] Patient conditional vectors are mapped to high-dimensional embeddings through a conditional coding network (Multilayer Perceptron, MLP) to enhance expressive power. e c = f c (c) in, f c Indicates multilayer perceptron operation; e c This represents the patient's conditional embedding, i.e., the output of the conditional coding network.

[0078] 3. Construct a conditional variational autoencoder Traditional variational autoencoders (VAEs) primarily learn the probability distribution of trajectory samples themselves, but struggle to explicitly characterize the modulation effect of external conditions on trajectory morphology. For patient transfer tasks, different individual patient conditions lead to systematic changes in the robot's initial holding posture, arm extension range, lifting height, and trunk coordination. Therefore, relying solely on ordinary VAEs is insufficient for effective individualized trajectory generation. To address this, a conditional variational autoencoder (CVAE) is constructed, simultaneously incorporating patient condition vectors into the trajectory encoder, conditional prior network, and trajectory decoder to model the distribution of individualized transfer actions.

[0079] The Conditional Variational Autoencoder (CVAE) takes five individual parameters of the patient—height, weight, BMI, sitting height, and shoulder width—as conditional inputs. First, it maps these parameters to a high-dimensional conditional embedding through a Conditional Coding Network (MLP). Then, during offline training, the Transformer Encoder learns the motion distribution characteristics under different patient conditions from teleoperation teaching trajectories and, together with the conditional prior network, constrains the latent space, ensuring that the latent variable z can compactly represent individualized movement styles (e.g., taller individuals require a wider holding stride, and heavier individuals require a slower lifting speed). During online inference, simply inputting the conditional vector of a new patient into the conditional prior network allows sampling of the implicit motion representation adapted to that patient from the latent space. The Transformer Decoder then reconstructs a complete and continuous sequence of joint and end-effector poses step-by-step, ultimately generating an individualized prior trajectory that inherits the human-like coordination demonstrated by experts and precisely matches the current patient's body type in terms of spatial scale and movement rhythm.

[0080] For teaching trajectory Given the patient condition vector c, the approximate posterior distribution of the trajectory encoder can be written as:

[0081] The conditional generation distribution of the trajectory decoder is written as: ; in, These are latent variables used to represent implicit movement style information in the trajectory that is difficult to describe directly by explicit conditions, such as the operator's coordination habits, preference for bi-arm coordination, and smooth transition patterns.

[0082] Introducing a conditional prior distribution allows the latent variable distribution to adaptively adjust as patient conditions change. The conditional prior distribution is expressed as:

[0083] The training objective of a conditional variational autoencoder can be expressed as: ; in, For trajectory reconstruction loss, To balance the weighting coefficients of the reconstruction term and the distribution constraint term, This represents the Kullback–Leibler divergence.

[0084] The trajectory reconstruction loss is expressed in the form of mean square error, i.e. , in, For the trajectory decoder at time The generated state vector; This represents the state vector at time t.

[0085] The trajectory encoder is implemented using Transformer, and the specific processing procedure is as follows: 1) Convert the state vector x at each time step t Linear projection is tokene t =W x x t +b x W x and b x Linear projection parameters respectively.

[0086] 2) Embed patient conditions into e c Add to each token t The position code PE1(t) is superimposed on top: =tokene t +W c e c +PE1(t) in, Indicates the result after position encoding; W c Indicates learnable parameters; 3) After passing through an L-layer TransformerEncoder (multi-head self-attention + FFN), the temporal features are obtained. .

[0087] 4) Perform global pooling on the time-series features (or use a dedicated aggregation token) to obtain the global trajectory feature h. X .

[0088] 5) h X With e c After concatenation, the data is fed into a fully connected network, which outputs the mean μ of the posterior distribution. q and variance .

[0089] 4. Individualized Prior Trajectory Generation Latent variable z is obtained by sampling from the posterior distribution (during offline training) or from the prior distribution (during online inference), and z is compared with e. c Concatenate and map to a global conditional vector g.

[0090] The trajectory decoder is implemented using Transformer, and each query vector for: =W g g+PE2(t) Among them, W g This represents the query parameter; PE2(t) is the position code in the decoder; That is, all time steps share the same g, but the position encoding is different, thus generating a sequence with temporal structure.

[0091] After passing through multiple TransformerDecoders (with autoregressive masks or parallel decoding), the hidden representation at each step is obtained. Then linearly map to the reconstructed state vector To form a complete trajectory .

[0092] Example 2 Online trajectory deformation of tactile constraint physical model based on ZMP In the original patient-carrying process, the "lifting action" and "transfer" phases primarily rely on impedance control, force / position hybrid control, and tactile pressure distribution feedback for real-time adjustment. This method is essentially a passive or instantaneous error feedback control: the controller only responds when abnormal pressure or ZMP (zero torque point) deviation is detected. However, the patient transfer process is characterized by strong contact time-varying and non-minimum phase—early disturbances such as local pressure concentration or support point migration often manifest as obvious postural instability or discomfort only after a certain lag. Correcting errors solely based on the current moment's error often misses the optimal intervention opportunity.

[0093] This embodiment employs a ZMP-based tactile constraint physical model precisely to address this issue: it utilizes distributed tactile perception, robot motion state, and individual patient conditions to predict the evolution of human-machine contact and system equilibrium trends in the short time domain (e.g., 1-2 seconds), and accordingly makes forward-looking, continuous, and small-amplitude corrections to the prior trajectory, achieving proactive adjustments before instability occurs.

[0094] The specific process is as follows: 1. ZMP-based physical model of tactile constraints Transfer and transportation tasks are typical examples of close human-machine contact tasks. During the execution of actions, the contact position, support range, and force distribution between the robot's arms and the patient's torso continuously evolve with changes in the holding posture, patient micro-movements, and system inertial effects. If feedback adjustments are made solely based on the current state error, corrections are often only made passively after contact deviation or instability has occurred, making it difficult to promptly suppress balance deterioration caused by localized pressure concentration, support shifting, or sudden posture changes. Therefore, this embodiment introduces a tactile constraint physical model based on ZMP in trajectory deformation correction. This model provides a forward-looking estimate of the human-machine contact state and system balance trend within the short-term prediction domain, and uses ZMP as the core stability characterization quantity to construct a prediction and correction framework for transfer and transportation tasks. ZMP characterizes the equivalent point of application of the support reaction moment within the support plane under the action of the net external force, comprehensively reflecting the support domain distribution, load transfer characteristics, and overall system balance. In patient transfer scenarios, although the human-machine system is not a typical foot-supported configuration, if the effective contact area between the robot and the patient is considered equivalent to a dynamic support domain, the evolution law of ZMP can still serve as a key indicator for evaluating the system's support stability and load transfer trend. Therefore, ZMP is not used as a single geometric feature quantity alone, but rather it is incorporated into the state representation system of the tactile constraint physical model along with tactile derivative physical quantities such as contact resultant force, contact area, and pressure center. This enhances the model's accuracy and representational ability in describing the contact state evolution process in the short time domain. Figure 8 As shown, online trajectory deformation correction combines the generated trajectories from ZMP and CVAE to adjust the trajectory.

[0095] First, a comprehensive contact state vector s is constructed, which includes the robot's joint motion state, end-effector pose, tactile observation, and the patient's individual conditions. t Then, based on the contact force information acquired by the distributed tactile sensors... f i and its location r i Real-time calculation of contact resultant force F t Effective contact area A t Contact pressure center m t And the core stability indicator—zero torque point z t Key physical quantities; finally, the current integrated contact state vector. s t Recent historical status and planned action sequence A common input is a multi-step sequence prediction network based on the Transformer architecture. fψ ( This study utilizes a self-attention mechanism to model the cross-temporal coupling relationship between actions, contact states, and stability, thereby enabling the model to be applied in the future short-term domain. H Internally, the evolution trends of ZMP, contact resultant force, contact area, pressure center, and end-effector pose are jointly predicted. This provides a basis for decision-making regarding subsequent forward-looking online trajectory deformation.

[0096] The multi-step sequence prediction network based on the Transformer architecture borrows from the general sequence modeling architecture of Transformer and is specifically designed for nursing transfer tasks. It takes the current robot motion state, tactile sensor information, individual patient conditions, and the planned sequence of actions as inputs, and outputs a joint prediction of multiple physical quantities in the short-term time domain, including ZMP, contact force, effective contact area, contact pressure center, and end-effector pose. This sequence prediction network, which unifies the modeling of tactile contact state and motion planning actions to predict the short-term equilibrium evolution trend of the human-machine system, provides fundamental support for achieving subsequent forward-looking trajectory deformation.

[0097] 2. Construct a comprehensive contact state To simultaneously characterize the robot's motion state, contact state, and individual patient conditions, the time-space... The comprehensive contact state of the ZMP-based tactile constraint physical model is defined as follows:

[0098] in, and These represent the robot's current joint position and joint velocity, respectively. and These represent the end effector pose and its velocity, respectively. This represents the tactile observation at the current moment; For contact pressure center; Effective contact area; For contact resultant force; ZMP coordinates These represent the horizontal and vertical coordinates of ZMP on the support plane, respectively. This is the patient condition vector.

[0099] 3. ZMP Calculation Based on Haptic Feedback In patient transfer tasks, the support domain is not the soles of the feet, but rather multiple contact areas between the robot's arms and the patient's torso. Therefore, ZMP calculations must rely on the contact force distribution provided by distributed tactile sensors. Let the... The positions of the contact units in the robot's base coordinate system are: The corresponding contact force is denoted as Then the resultant force F of the system obtained by summing all contact elements and the resultant moment M about the origin O of the base coordinate system are...O They are respectively: ; in, This represents the number of effective contact units.

[0100] In the short-term prediction process of patient transfer, most of the system's adjustments occur during the relatively slow hold and support transfer phase. Therefore, a quasi-static approximation is adopted, considering the instantaneous contact force distribution as the main factor determining the current equilibrium trend. Under this assumption, the coordinates of ZMP on the support plane can be expressed as:

[0101] in, This is the vertical component of the system's resultant force. and They represent the resultant torque at... shaft and The component in the axial direction. This expression shows that the essence of ZMP is the equivalent mapping of the resultant moment induced by the contact force distribution on the support plane, and therefore it is highly sensitive to local support imbalance, load offset and force concentration.

[0102] In addition to ZMP, we further constructed tactile statistical features directly related to contact stability: First, the contact resultant force F t Defined as ; The resultant contact force is used to describe the overall contact load level at the current moment.

[0103] Secondly, the effective contact area A t Defined as:

[0104] In the formula, The contact detection threshold, For the first The physical area corresponding to each contact unit, and the indicator function. This is used to determine whether the contact unit (i.e., the sensing unit) is in an effective contact state. The effective contact area reflects the degree of coverage of the support area. A significant reduction in the effective contact area usually corresponds to the shrinkage of the support domain or the local concentration of contact load, which in turn increases the risk of system instability.

[0105] Based on this, the contact pressure center m t Defined as:

[0106] In the formula, To prevent the denominator from being an oddly small positive number.

[0107] The contact pressure center characterizes the weighted center location of the contact load in space and can describe the lateral and longitudinal offset behavior of the support point during holding. Rapid shift occurs and When the load decreases synchronously, it usually indicates that the support load is accumulating in a local area, which can be an important precursor to contact instability and patient posture deviation.

[0108] 4. Predicted output of the ZMP-based physical model for haptic constraints Set time The control action is Then, the tactile constraint physics model based on ZMP for the future The prediction of the step-state sequence can be written as:

[0109] in, The parameter is A ZMP-based physical model of tactile constraints. This represents the sequence of planned actions from the current moment to the end of the future prediction domain. The model's task is not merely to predict a single kinematic state, but to jointly predict multiple contact and equilibrium quantities closely related to stability in the short-term future, including: ; In the formula, Indicates the predicted ZMP coordinates. This indicates the predicted contact resultant force. Indicates the predicted effective contact area. Indicates the predicted center of pressure. This indicates the predicted end-effector pose.

[0110] By conducting joint time-series predictions on the aforementioned physical quantities, the model can not only deduce the future motion posture evolution of the robot, but also accurately predict the load transfer characteristics and system equilibrium state changes caused by the motion, thereby providing a forward-looking decision-making basis for subsequent online trajectory correction.

[0111] In terms of structural implementation, A multi-step sequence prediction network based on the Transformer architecture is employed. Its input consists of the current state. Recent historical status and planned action sequence a t:t+H-1Together, they form a model that utilizes a self-attention mechanism to model the cross-temporal coupling relationship between motion input, contact state, and stability indicators. Compared to models that only perform a one-step rolling extrapolation based on the current state, the tactile constraint physical model based on ZMP in this invention is implemented using a Transformer, which is more suitable for characterizing the short- to medium-term cumulative effects that exist during patient transfer, such as ZMP shift, contact area reduction, and local peak force increase caused by slight changes in holding position after several steps.

[0112] 5. Online trajectory deformation predicted based on tactile constraint physical model Once we obtain the predicted values ​​of ZMP, contact area, contact pressure center, peak force, etc. in the short time domain, we can take proactive measures instead of waiting for deviations to occur before making corrections.

[0113] Construct the following online optimization cost function to solve for the current optimal control action a. t : , in, For the desired ZMP, To predict peak contact force, and These are the desired contact area and the desired pressure center, respectively. These are the weighting coefficients for each item; This represents the control action at time t-1; This represents the optimal control action at time t; For prior control input; Indicates the prior end pose; Indicates the predicted ZMP coordinates. Indicates the predicted effective contact area. Indicates the predicted center of pressure. This indicates the predicted end-effector pose.

[0114] Each term in the formula has a clear physical meaning: the first term constrains the predicted ZMP to remain near the desired stable region, suppressing support offset from an overall balance perspective; the second term penalizes the predicted peak contact force to suppress excessive local stress; the third term constrains the contact area to be close to the desired value, avoiding excessive shrinkage of the support area; the fourth term adjusts the pressure center to reduce abnormal migration of support loads; and the fifth term constrains the predicted end pose to avoid deviating too far from the prior trajectory, preserving the human-like motion style in expert teaching. Furthermore, the last two terms are used to limit the current control input from deviating too much from the prior action and to suppress drastic changes between adjacent control moments, thereby improving the continuity and smoothness of the trajectory correction process.

[0115] Based on the above cost function, the online correction problem can be expressed as:

[0116] The formula shows that the current control input is not adjusted solely based on the instantaneous error, but rather comprehensively considers the future short-term contact and equilibrium evolution trends, and optimizes to obtain the optimal correction amount that balances stability, compliance, and human-like characteristics. .

[0117] If the control action is defined as the Cartesian space pose increment... Then it can be mapped to the joint space using the pseudo-inverse of the Jacobian matrix, that is:

[0118] in, This is the pseudo-inverse of the Jacobian matrix of the robot under the current configuration. The optimal end-effector pose increment is obtained through optimization. This allows for the updating of the robot joint states:

[0119] Considering the robot's own limits and human-robot contact safety constraints, the above optimization problem can be further written as:

[0120] in, and These represent the upper and lower limits of the joint angle, respectively. This indicates the maximum allowable joint change in a single step. This indicates the maximum permissible peak contact force.

[0121] The above constraints ensure that the online trajectory correction process meets both the limitations of robot kinematics and actuator capabilities, as well as human-robot contact safety requirements.

[0122] The cost function restructures the optimization objective from the traditional "precise reproduction of geometric trajectory" to "active management of physical contact and balance." Specifically, the error is no longer simply defined as the distance between the actual position and the desired position. Instead, it decomposes the error into a set of directly perceptible and predictable physical quantities, focusing on the safety requirements of the transfer task: The first term introduces ZMP error, which directly penalizes the deviation of the future balance point, prioritizing dynamic stability; the second to fourth terms introduce peak contact force, contact area, and contact pressure center error, explicitly elevating the tactile information, originally a feedback signal, to the optimization objective, enabling the controller to avoid risks such as excessive local pressure, insufficient support, or abnormal load migration in advance; the fifth term downgrades the prior trajectory from a hard tracking command to a soft constraint, retaining human-like movement style only under the premise of ensuring the aforementioned physical safety; the last two terms introduce a smoothness penalty on the control increment itself to prevent overly abrupt correction actions or violent jitter. The fundamental reason for this change is that the safety and stability of human-computer interaction in nursing transfers are far more important than the geometric accuracy of the movements. Therefore, the physical constraints were moved from the outside of the optimization problem to the core inside, and a controller was designed that can anticipate movement risks and actively seek a safe, smooth, and human-like compromise optimal solution.

[0123] In summary, the online trajectory deformation predicted by the ZMP-based tactile constraint physical model is no longer limited to the traditional passive compensation mode based on real-time errors. Instead, it predicts the changing trends of ZMP, contact area, pressure center, and peak contact force in the short-term time domain using the ZMP-based tactile constraint physical model, thereby achieving proactive correction for future stability trends. This method effectively improves the system's adaptability to contact disturbances, patient micro-movements, and local instability risks while preserving the human-like coordinated motion characteristics of the prior trajectory.

[0124] Any aspects not covered in this invention are applicable to existing technologies.

Claims

1. A control method for autonomous navigation, human-carrying, and transporting a dual-arm nursing robot, characterized in that, The process of the control method is as follows: The constructed musculoskeletal biomechanics dataset incorporates the activation levels of the sternocleidomastoid, external oblique, gluteus maximus, and rectus femoris muscles into an LSTM neural network. The LSTM neural network is then trained to predict the activation levels of human muscles. Then, a comfort prediction model for holding a dual-arm nursing robot is constructed by combining the human-machine contact mechanics model. The comfort experience in the holding state is analyzed, and the optimal holding posture parameters are determined, which are denoted as the planned posture. Based on the optimal holding posture parameters, and combined with the real-time three-dimensional position of the human body key points in the robot base coordinate system, the Cartesian coordinates of the target contact points of the robot's left forearm and right forearm are calculated. In the control of the lifting process, the lifting trajectory is divided into three stages: pre-tensioning stage, lifting stage, and hovering stage. During the lifting process, Openpose continuously monitors the posture angles (θ2, θ3) of the simplified four-bar linkage of the human body and compares them with the planned values. If the actual posture deviates from the planned value by more than ±5°, the target posture of the ends of the arms is readjusted through inverse kinematics to restore the human body to the planned posture. Target carriers are identified in real time using target detection algorithms; After obtaining the current position of the patient, the robot chassis autonomously navigates from its current position to a distance in front of the target carrier and maintains the preparatory position. In the navigation of the dual-arm nursing robot chassis, a combination of global path planning and local DWA obstacle avoidance is used for navigation. The placement process then proceeds in three stages: descent, release, and retraction. Once the retraction is complete, the successful placement is verified by visually detecting the distance between key points on the human body and the end effector of the robotic arm.

2. The control method according to claim 1, characterized in that, The process of determining the optimal holding posture parameters is as follows: the patient's height, weight, body mass index (BMI), sitting height, and shoulder width are constructed into a patient condition vector. The conditional variational autoencoder (CVAE) is used to learn the expert teaching trajectory collected by teleoperation to generate a complete transfer trajectory that matches the patient's individual characteristics. The robot learns from the expert teleoperation teaching data and automatically establishes a mapping relationship between "patient individual parameters → complete transfer trajectory", thereby obtaining a continuous, smooth, and human-like transfer trajectory.

3. The control method according to claim 1, characterized in that, The Conditional Variational Autoencoder (CVAE) includes a conditional coding network, a trajectory encoder, a conditional prior network, and a trajectory decoder. The patient's conditional vector is simultaneously introduced into the trajectory encoder, the conditional prior network, and the trajectory decoder to model the distribution of individualized transfer actions. Using five individual parameters of the patient—height, weight, BMI, sitting height, and shoulder width—as input, the parameters are first mapped to a high-dimensional conditional embedding through a conditional encoding network. Then, in the offline training phase, the trajectory encoder learns the motion distribution characteristics under different patient conditions from the teleoperation teaching trajectory and, together with the conditional prior network, constrains the latent space, so that the latent variable z can compactly represent the individualized motion style. In the online inference phase, only the patient condition vector of the new patient needs to be input into the conditional prior network to sample the implicit expression of the motion adapted to the patient from the latent space. Then, the trajectory decoder reconstructs the complete and continuous sequence of joint and end-effector poses step by step through the time step, and finally generates an individualized prior trajectory that inherits the humanoid coordination of expert teaching and precisely matches the current patient's body shape in terms of spatial scale and motion rhythm.

4. The control method according to claim 1, characterized in that, In the chassis navigation of a dual-arm nursing robot, the changing trends of ZMP, contact area, pressure center and peak contact force in the short time domain are predicted by a tactile constraint physical model, thereby enabling the active correction of the pose of the two arms to meet future stability trends.

5. The control method according to claim 4, characterized in that, The tactile constraint physical model is a ZMP-based tactile constraint physical model, and the active correction process is as follows: A comprehensive contact state vector is constructed based on the robot's joint motion state, end-effector pose, tactile observation, and individual patient conditions. Contact force information acquired by distributed tactile sensors f i and its location r i Real-time calculation of contact resultant force F t Effective contact area A t Contact pressure center m t and zero torque point z t ; Finally, the current state s t Recent historical status and planned action sequence a t:t+H-1 Common input multi-step sequence prediction network based on Transformer architecture fψ ( This study utilizes a self-attention mechanism to model the cross-temporal coupling relationship between actions, contact states, and stability, thereby enabling the model to be applied in the future short-term domain. H Internally, the evolution trends of ZMP, contact resultant force, contact area, contact pressure center, and end-effector pose are jointly predicted. This provides a basis for decision-making regarding subsequent forward-looking online trajectory deformation; After obtaining the predicted values ​​of ZMP, contact resultant force, contact area, contact pressure center, peak force, and end pose in the short-term time domain, a cost function is set to correct the trajectory deformation online. The cost function decomposes the error into a set of directly perceptible and predictable physical quantities: 1) Introducing ZMP error to directly penalize the deviation of the future equilibrium point, with dynamic stability as the highest priority; 2) Introducing peak contact force, contact area, and contact pressure center errors to explicitly elevate the tactile information, which was originally a feedback signal, to the optimization target, enabling the controller to avoid the risks of excessive local pressure, insufficient support, or abnormal load migration in advance; 3) Degrading the prior trajectory from a hard tracking command to a soft constraint, retaining the human-like movement style only under the premise of ensuring the aforementioned physical safety; 4) Introducing a smoothness penalty on the control increment itself to prevent the correction action from being too abrupt or violently jittering, thus obtaining a controller that can foresee action risks and actively seek a safe, smooth, and human-like compromise optimal solution.

6. The control method according to claim 5, characterized in that, The cost function J t for: ， in, For the desired ZMP, To predict peak contact force, and These are the desired contact area and the desired pressure center, respectively. These are the weighting coefficients for each item; This represents the control action at time t-1; This represents the optimal control action at time t; For prior control input; Indicates the prior end pose; Indicates the predicted ZMP coordinates. Indicates the predicted effective contact area. Indicates the predicted center of pressure. This indicates the predicted end-effector pose.

7. The control method according to claim 1, characterized in that, Control points for each stage of the lifting trajectory: The first stage is pre-tightening, where the arms slowly apply a small force to eliminate gaps and slightly press the flexible layer, so that the human body fits into the robot's forearms; The second stage is lifting, where the end effector moves along an S-shaped velocity curve in the vertical direction, with acceleration gradually increasing from 0 and then gradually decreasing, and the peak lifting speed not exceeding 0.1 m / s. The third stage is hovering, where the lifting stops when the human's feet are about 5 cm off the ground, maintaining the posture for 1 second to wait for dynamic stabilization. During the lifting process, position and force are controlled simultaneously. The left arm mainly bears the vertical load and uses force control to maintain the set support force, while the right arm mainly plays a stabilizing and guiding role, using position control to track the trajectory, and simultaneously adjusting the lateral force through impedance control to prevent the human from sliding sideways. The force distribution between the two arms is coordinated through a task priority framework: the left arm prioritizes ensuring vertical support force, and the right arm prioritizes ensuring posture stability.

8. The control method according to claim 1, characterized in that, During placement, in the descent phase, the end effector moves vertically along an S-shaped speed curve. When the end effector approaches the upper part of the carrier contact surface, it switches to force control mode. The actual contact force is estimated by the joint torque sensor, and the end effector height is adjusted by the PI force controller. The desired contact force is set. Once the contact force reaches the set value and lasts for 0.5 seconds, it is determined that "stable contact has been achieved". In the release phase, the support force of the two arms decreases linearly to 0 within 2 seconds. At the same time, the pressure distribution symmetry is monitored by the tactile sensors of the two arms. If the pressure difference between the left and right arms exceeds 30%, the system pauses and the chassis position is finely adjusted. In the retraction phase, the robotic arm slowly opens outward to separate the forearm from the human body, then lifts upward and retracts to a safe folded posture.

9. A dual-arm nursing robot, characterized in that, The robot performs the control method according to any one of claims 1-8.

Images