Upper limb rehabilitation robot human-robot collaborative trajectory tracking control method and system thereof

Abstract

The present application belongs to the technical field of rehabilitation robot control, and particularly relates to a man-machine collaborative trajectory tracking control method and system for an upper limb rehabilitation robot. The method comprises the following steps: generating an offline reference trajectory; constructing a man-machine coupling environment; training a man-machine collaborative trajectory tracking strategy by using a soft actor-critic algorithm; loading the strategy network to output joint control instructions to drive the rehabilitation robot to complete rehabilitation training. The present application adopts an adaptive reference point advancing mechanism jointly scheduled by a space channel, a time channel and a hysteresis detection, improves the reference trajectory index from a time open-loop function to a closed-loop feedback based on the real-time tracking state of an end effector, and solves the problems of reference point disconnection and tracking error divergence caused by open-loop advancing from the mechanism. At the same time, the residual muscle strength of a patient is modeled as a saturated spring force, which is used as a real input on the right side of a rigid body dynamics equation to participate in numerical integration by sub-step, instead of being an artificial term in a reward function, thereby avoiding reward leakage from the physical layer and fitting the real man-machine collaborative scene.

Description

Technical Field

[0001] This invention belongs to the field of rehabilitation robot control technology, specifically relating to a human-machine collaborative trajectory tracking control method and system for an upper limb rehabilitation robot. Background Technology

[0002] Stroke is one of the leading causes of limb motor dysfunction, characterized by high incidence and high disability rates. Modern neuroscience research shows that the central nervous system has plasticity, and damaged motor function can be reconstructed through high-intensity, high-repetition, task-oriented early rehabilitation training. Rehabilitation robots can provide patients with quantifiable, repeatable, long-term, and long-distance motor training, and are an important way to alleviate the shortage of rehabilitation therapists and improve the efficiency of rehabilitation training.

[0003] Rehabilitation robot-assisted training is usually divided into two stages: passive rehabilitation and active rehabilitation. In the passive stage, the affected limb is moved by tracking the desired trajectory to prevent muscle contracture and joint adhesion. As function recovers, the focus of training shifts to the active rehabilitation stage. At this time, the patient's participation directly affects the recovery effect of neuroplasticity. Therefore, how to provide "just the right" assistance according to the patient's real-time movement status, that is, the principle of on-demand assistance, is the core issue to improve the rehabilitation effect.

[0004] Existing active rehabilitation controllers have significant limitations: while reinforcement learning-based controllers can train strategies with task objectives in the absence of prior knowledge, their strategies only focus on "how to track a given reference," and the reference trajectory itself advances in an open-loop manner according to a time law, decoupling from the real-time tracking state of the strategy. In scenarios where patients have residual active assistive force but limited capabilities, when the end effector decelerates on geometrically challenging sections, the open-loop advancing reference point continues to move forward, and the tracking error monotonically accumulates and expands over time until it becomes uncontrollable; when the end effector stops, the fixed-rate reference point cannot provide a "waiting" margin, and the strategy cannot catch up. In addition, existing methods often write the patient's residual muscle strength into the reward function as an artificial reward or treat it as a random perturbation during simulation training. The former causes the patient's assistance to be incorrectly encoded as the "reward source" of the strategy, resulting in the learned behavior of the strategy not conforming to real physical causality; the latter severs the real physical coupling relationship between the patient's active force output and the robot's motion. Therefore, there is an urgent need for an upper limb rehabilitation robot control method and system that can schedule the advancement of the reference point in a closed loop based on the real-time tracking state of the end effector and characterize the patient's residual muscle strength at the physical level rather than the reward level. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies, such as reference trajectory open-loop advancement over time, decoupling from real-time tracking state of the strategy leading to reference disconnection, tracking error divergence and even task failure, reward leakage, poor physical consistency and low tracking accuracy caused by including the patient's residual muscle strength in the reward function during simulation training. The invention provides a human-machine collaborative trajectory tracking control method and system for upper limb rehabilitation robots that can advance reference points in a closed loop based on the real-time tracking state of the end position and characterize the patient's residual muscle strength at the physical level rather than the reward level.

[0006] The technical solution adopted by this invention to solve its technical problem is: a human-machine collaborative trajectory tracking control method for an upper limb rehabilitation robot, comprising:

[0007] S1. Offline reference trajectory generation: Build a robot joint space reference trajectory dataset and generate a waypoint sequence containing M waypoints;

[0008] S2. Constructing a human-machine coupled environment: Divide the patient into multiple motor function levels and configure a corresponding stiffness coefficient for each motor function level. With upper limit of force saturation The patient's residual muscle strength was modeled as a saturated spring force. ; the saturated spring force Mapped to joint space via geometric Jacobian matrix, and used for rigid body dynamics integration;

[0009] S3. Policy Network Training: Establish a Markov decision process and use the soft actor-critic algorithm to train the human-machine cooperative trajectory tracking strategy. Repeat the following steps in each control step until the trajectory is completed or the maximum number of steps is triggered:

[0010] S31. Observation and Assembly: Acquiring Joint Angle Vectors Joint angular velocity vector and the actual position of the end effector The value, using the currently maintained reference point index. Extract the target reference point position from the dataset. Calculate the tracking error vector ,Will , , Error vector magnitude , and the progress of normalized trajectory completion Concatenate to construct observation vectors ,in ;

[0011] S32, Strategy Reasoning and Execution: Observation Vectors Input a soft actor-critic policy network, output the mean and log-standard deviation of a Gaussian distribution, and obtain the action vector through reparameterization sampling. , action vector The first six dimensions are scaled and used as joint angle increments, which are then superimposed onto the current joint target.

[0012] S33, Physics Propulsion: In several physics sub-step cycles, a strategy torque is applied simultaneously within each sub-step. and the saturation spring force recalculated at each step The saturated spring force Within the current control step, the reference point remains unchanged, but the dynamic calculation of the real-time end displacement is carried out together with the rigid body dynamics integral to obtain the true joint state at the next moment.

[0013] S34. Adaptive Reference Point (ARP) Index Update: Based on the actual end-point position after physical propulsion. Calculate the rate coefficients under spatial channel, temporal channel, and sluggish detection. The reference point propulsion is obtained through joint scheduling of three signals: spatial channel, time channel, and jamming detection. ;

[0014] S35. Reward Calculation: Based on the aforementioned reference point advance. Calculate progress rewards End-stage bonus and milestone rewards Error based on physical propulsion and The tracking reward, smoothing penalty, and violation component are calculated and summed into a scalar reward value. ;

[0015] S36. Next-state assembly and storage: Using the new index Reread the new reference point Assemble the next observation The quadruple of this interaction ( , , , Store in the experience replay buffer;

[0016] S37. Gradient Update: When the amount of data in the experience replay buffer exceeds the preheating threshold, a batch of samples is randomly sampled, the loss function is calculated sequentially, the dual critic network, the policy network and the adaptive temperature coefficient are updated, and the network gradient update is performed.

[0017] S4. After completing the strategy training, load the trained strategy network and output joint control commands to drive the upper limb rehabilitation robot to complete human-machine collaborative trajectory tracking rehabilitation training.

[0018] Further, in step S1, the joint space reference trajectory dataset is formed by converting clinical daily living activity ability prescriptions, and is used for point-by-point indexing and advancement by online ARP. The dataset is processed as follows:

[0019] S11, Cartesian level:

[0020] The location of key waypoints is interpolated using cubic B-spline, and the attitude is interpolated using quaternion spherical linear interpolation. The trajectory arc length is reparameterized and scattered at equal intervals to form a waypoint sequence, so that the rehabilitation robot advances with equal end displacement each time.

[0021] S12, Joint Level: The inverse kinematics is solved point by point using the damped least squares method, and a hot start acceleration is performed; first, trapezoidal velocity planning is used, and then a fifth-order polynomial is used for smoothing, and a reference trajectory with timestamps is output.

[0022] S13. Data verification: Before data is entered into the database, accessibility, continuity, and dynamic feasibility are verified. If the data is qualified, it is included in the dataset to generate a waypoint sequence for online indexing. If the data is unqualified, local replanning is triggered, and steps S11-S13 are repeated.

[0023] Further, in step S2, the motor function level is classified according to the Fugl-Meyer Upper Limb Function Assessment Scale, and the saturation spring force is: , where the end deviation vector , This is the current reference point location. The current position of the robot's end effector. Let be the unit direction vector pointing to the reference point, when hour Take the zero vector. This is the spring stiffness coefficient. The upper limit of force saturation; the saturation spring force is passed through the geometric Jacobian matrix of the end effector. Mapped to joint torque The integral equation of the rigid body dynamics is: .

[0024] Further, in step S32, the action vector The joint angle increment .

[0025] Further, in step S34, the spatial channel ,in, , Preferably, ;when = At that time, the robot's end effector is in close contact with the reference point; when > hour, Immediately reset to zero, the reference point completely awaits the end effector; the time channel ,in , This represents the maximum number of control steps executed in a single task. This represents the time channel rate coefficient.

[0026] Further, in step S34, the stagnation detection is used to detect the coverage progress at the end point, and to reduce the time channel rate to extend the policy catch-up time when the policy stagnates. The time channel rate coefficient ,in The last time the end effector was used Control step time greater than zero, The stagnation threshold and .

[0027] Furthermore, , , The control frequency of the control step is 50Hz. One control step; preferably, , .

[0028] Further, in step S35, the progress reward ,in The tail segment reward ,in, The milestone reward Among them, when 80% of the nodes are reached, When the 100% node is reached, The tracking reward ,in , The smoothing penalty ,in, The aforementioned non-compliant component ,in , , The scalar reward value .

[0029] Furthermore, it also includes safety monitoring: real-time acquisition of the interaction force modulus between the end effector and the towed object. With interaction torque modulus ,when Exceeding the force safety threshold Or when Exceeding the torque safety threshold The system records all such occurrences, preferably... , .

[0030] A human-machine collaborative trajectory tracking control system for an upper limb rehabilitation robot, used to execute the control method described in claims 1-9, comprising:

[0031] The trajectory library generation module is used to interpolate and solve inverse kinematics for clinical daily living activity ability prescription task trajectories to generate a joint space reference trajectory dataset.

[0032] The patient ability grading module is configured to grade the patient's upper limb motor function based on the Fugl-Meyer Upper Limb Function Assessment Scale, and determine the patient's ability level and the corresponding saturation spring force parameter.

[0033] The human-machine coupled dynamics simulation module is configured to construct a coupled rigid body dynamics model that includes a robot and a patient with residual muscle strength. The saturated spring force is mapped to the joint space as a real physical external force through a geometric Jacobian matrix and participates in the dynamics integration.

[0034] The reinforcement learning training module is configured to train a cooperative tracking strategy based on the soft actor-critic algorithm within the Markov decision process framework. The cooperative tracking strategy takes the observation vector as input and the joint angle increment as the action output.

[0035] The adaptive reference scheduling module is configured to jointly calculate the index advance of the reference trajectory through three components—spatial channel, temporal channel, and lag detection—in each control step of executing the reinforcement learning strategy, dynamically update the current reference point position, and make the reference point advance rate adjusted by the real-time tracking status feedback of the end effector.

[0036] The reward calculation module is configured to calculate a scalar reward value based on the updated reference point index, which includes tracking reward, progress reward, tail reward, milestone reward, smoothing penalty and violation penalty, for parameter updates of the soft actor-critic algorithm;

[0037] The control module is used to load the trained policy network and output joint control commands to drive the upper limb rehabilitation robot to complete human-machine collaborative trajectory tracking rehabilitation training.

[0038] The beneficial effects of the human-machine collaborative trajectory tracking control method and system for upper limb rehabilitation robots of the present invention are as follows:

[0039] This invention designs an adaptive reference point advancement mechanism jointly scheduled by a spatial channel, a temporal channel, and lag detection. This transforms the advancement of the reference trajectory index from a traditional open-loop time function to a closed-loop feedback function where the end effector tracks the state in real time. The spatial channel employs a look-ahead, continuously satisfying criterion, advancing the index only when the end effector continuously falls into the waypoint neighborhood, ensuring the reference point waits entirely for the end effector. The temporal channel provides a fallback limit, and its maximum number of advanceable waypoints is structurally strictly less than the total number of waypoints, forcing the main advancement to rely on the spatial channel. Lag detection further reduces the temporal channel rate when the end effector stagnates for an extended period, prolonging the catch-up time. This joint scheduling of the three mechanisms enables the system to automatically degenerate into point-by-point advancement or even waiting in difficult road sections with geometric bends and dramatic curvature changes, structurally avoiding the reference point disconnection and monotonic divergence of tracking errors caused by open-loop advancement.

[0040] This invention categorizes patients into multiple motor function levels and models the patient's residual muscle strength as a saturated spring force pointing towards the current reference point. This force is then mapped to joint torque via a geometric Jacobian matrix and serves as the actual input term on the right-hand side of the rigid body dynamics equations for step-by-step numerical integration, rather than an artificial term in the reward function. This design avoids reward leakage from a physical perspective: the end-effector motion perceived by the strategy is the result of the combined action of the robot's active control torque and the patient's assistive force, perfectly consistent with real human-machine collaborative scenarios. Attached Figure Description

[0041] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0042] Figure 1 This is a schematic diagram of the overall process of the human-machine collaborative trajectory tracking control method according to an embodiment of the present invention;

[0043] Figure 2 This is an overall architecture diagram of the human-machine collaborative trajectory tracking control system according to an embodiment of the present invention;

[0044] Figure 3 This is a saturated spring force model curve of residual active force under different motion function levels in an embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram of the force distribution on the human-machine coupling system according to an embodiment of the present invention;

[0046] Figure 5 This is a schematic diagram of the policy network training process according to an embodiment of the present invention. Detailed Implementation

[0047] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0048] like Figures 1-5 The embodiment shown is a human-machine collaborative trajectory tracking control method for an upper limb rehabilitation robot according to the present invention. The end effector of the upper limb rehabilitation robot pulls the object being pulled along a reference trajectory. During the movement, the end effector is subjected to external forces from the object being pulled. The method includes the following steps: S1, constructing a robot joint space reference trajectory dataset, sampling the reference trajectory within the dataset at equal intervals along the arc length to generate a waypoint sequence containing M waypoints, and having the end effector pull the affected limb along this sequence. M is the total number of waypoints, and the reference point index is... Values ​​are taken from In step S1, the joint space reference trajectory dataset is generated from clinical Activities of Daily Living (ADL) prescriptions and is used for point-by-point indexing and advancement by the online ARP. The dataset is processed as follows:

[0049] S11, Cartesian level:

[0050] The location of key waypoints is ensured by cubic B-spline interpolation to maintain continuous acceleration in Cartesian space, and the attitude is ensured by quaternion spherical linear interpolation to maintain smooth rotation. The arc length along the trajectory is reparameterized and equidistantly distributed to form a waypoint sequence, so that each advancement of the reference point corresponds to an approximately equal end-effector displacement. This ensures that each index advancement of the rehabilitation robot results in an equal end-effector displacement, which serves as the spatial distance threshold. Provides a consistent geometric reference along the entire trajectory;

[0051] S12, Joint Level: The inverse kinematics is solved point by point using the damped least squares method, mapping Cartesian space points to joint space angles. Adaptive damping is used to avoid singular positions and hot start is performed to accelerate convergence. The solution of the first path points is used as the initial value for hot start. Trapezoidal velocity planning is used first, and then a fifth-order polynomial is used for smoothing. The reference trajectory with timestamp is output.

[0052] S13. Data verification: Before data is entered into the database, accessibility, continuity, and dynamic feasibility are verified. If the data is qualified, it is included in the dataset to generate a waypoint sequence for online indexing. If the data is unqualified, local replanning is triggered, and steps S11-S13 are repeated.

[0053] In this embodiment, the dataset contains 100 everyday activity trajectories, such as raising a cup, reaching forward, washing one's face, and combing one's hair. Each trajectory is a complete task sequence rather than a point-to-point small-amplitude movement. Each original trajectory contains 1000 sampling points. During training, each trajectory is downsampled at 20 intervals to 51 waypoints (including the endpoint), resulting in a total of M=51 waypoints and 5100 pathpoints across the 100 trajectories. The arc-length equal-distance dispersion has the following effect on this invention: it ensures that each advancement of the reference point index corresponds to approximately equal end-point displacement, thus ensuring that the spatial distance threshold corresponds to the same "proximity" semantics across any segment of the trajectory, providing a consistent geometric benchmark for threshold setting throughout the entire trajectory. Trajectory feasibility verification: On the human side, the upper limb is simplified as a link with the shoulder joint as the origin. The lengths of each segment are based on adult anthropometric standards: 0.30m for the upper arm, 0.25m for the forearm, and 0.10m for the hand. The theoretical maximum radius when fully extended is 0.65m. For each of the 5100 path points, the distance to the shoulder joint was calculated. All of them fell within the reachable radius, achieving a 100% reachability rate. The maximum distance was 0.633m, the minimum distance was less than 0.65m, and the average distance was 0.261m. The robot-side reachability was guaranteed by the point-by-point inverse kinematics solution during the generation phase. The landing point was entirely within the robotic arm's sampling workspace: X∈[−0.47,+0.11]m, Y∈[+0.15,+0.72]m, and Z∈[+0.92,+1.79]m, leaving ample margin. It should be noted that the human reachability radius criterion is based on full extension, is somewhat optimistic, and is only a necessary condition. Its purpose is to confirm that the trajectory does not exceed the normal range of human movement.

[0054] S2. Constructing a human-machine coupling environment: Patients are divided into several motor function levels according to the Fugl-Meyer Upper Limb Function Assessment Scale, and a corresponding stiffness coefficient is configured for each motor function level. With upper limit of force saturation ,and and All increase with the improvement of motor function level; Modeling the patient's residual active assistive force: Stroke hemiplegic patients have varying degrees of residual motor ability during active rehabilitation training of the upper limbs. Under the end-effector traction rehabilitation paradigm, after sensing the guidance direction of the robot's end effector, the patient actively applies an assistive force pointing towards the current reference point to the end effector. The magnitude of this assistive force is positively correlated with the end-effector position deviation: when the deviation is small, the force increases linearly with the deviation; after the deviation exceeds a certain range, it tends to saturate due to the patient's maximum muscle strength limitation. The patient's residual muscle strength is modeled as a saturated spring force. ; saturate spring force By mapping the geometric Jacobian matrix to the joint space, the data participates in the rigid body dynamics integral, rather than being included in the artificial terms of the reward function, thus avoiding reward leakage from the physical channel. In this embodiment, the simulation environment is built based on the MuJoCo physics engine. The robotic arm is a 6-DOF serial robotic arm with an end effector mass of 0.8 kg, a control frequency of 50 Hz, a control step size of 0.02 s, and each control step contains 10 physical substeps. The joint angles and joint angular velocities of the robotic arm are limited by the hard constraints of the mechanical structure. Inputs exceeding the limits are automatically truncated by the simulator. Each random seed training takes about 11 hours, with 8M steps and 4 parallel environments.

[0055] This embodiment employs an end-effector traction rehabilitation paradigm, where a robotic end effector tractions the patient's affected limb to complete daily living activities, with the patient contributing their residual active force. Patient ability grading and force parameter mapping: Clinically, the Fugl-Meyer Motor Function Assessment Scale is used to quantify and grade the upper limb motor function of stroke-induced hemiplegic patients. The upper limb portion is scored continuously from 0 to 66 points based on items such as reflexes, synergistic movements, dissociative movements, and coordination; higher scores indicate better retention of active motor function. This embodiment divides patients into three levels—severe, moderate, and mild—according to approximately equal intervals, corresponding to score ranges of 0-20, 21-40, and 41-60 points respectively, to cover the range of clinically relevant abilities from severe motor impairment to near-normal levels requiring active rehabilitation training. The maximum end-effector active assistive force amplitude that can be applied during rehabilitation training is designed in three levels: severely ill patients have almost no active force output, with an equivalent end-effector assistive force of less than approximately 2N, corresponding to a saturation distance. Moderately ill patients possess some ability to separate movements, and can be subjected to approximately 2–5 N, corresponding to a saturation distance. Mild cases retain more residual muscle strength, with an assistive force of approximately 5–8 N at the distal end, corresponding to a saturation distance. Based on this, the patient force parameters are configured as shown in the table below:

[0056]

[0057] The saturation distances for the three levels mentioned above are all within the working scale of the end-of-life activity trajectory, ranging from 0.4 to 0.667 m. In normal tracking scenarios, the end-of-life tracking deviation is approximately 11 mm, far smaller than the saturation distances. Therefore, the patient's force remains within the linear response zone during normal tracking. The saturation mechanism only takes effect when a significant abnormal deviation occurs at the end-of-life stage, preventing the assistive force from exceeding the physiological upper limit. The design of parameters increasing with ability level aligns with the clinical rehabilitation principles of the Assist-as-Needed (AAN) control strategy: patients with greater abilities can exert larger active assistive forces independently, correspondingly reducing the guiding force required by the robot. It should be noted that the stiffness K here describes the growth rate of the patient's net assistive force with end-of-life deviation, not the end-of-life impedance of the human upper limb; the two are not comparable in magnitude. The above mapping is an approximate correspondence established to distinguish patients with different abilities within the simulation domain, not a clinical diagnostic standard. In actual deployment, the force amplitude thresholds for each level still need to be re-determined based on specific patient assessments.

[0058] In step S2, the saturation spring force is: , where the end deviation vector , This is the current reference point location. The current position of the robot's end effector. This is the unit direction vector pointing towards the reference point. When the deviation is small ( The deviation increases linearly, and the deviation exceeds the saturation distance. After remaining Unchanged. When hour Use the zero vector to avoid singularities. The spring stiffness coefficient (unit: N / m) characterizes the linear growth rate of the patient's net assist force with end-effector deviation; The upper limit of force saturation (unit: N) reflects the maximum end-effector force that a patient at this ability level can provide. Its physical basis is that after the patient perceives the robot's guidance direction, they will actively apply an assistive force pointing towards the current reference point. The magnitude of this assistive force is positively correlated with the end-effector deviation. When the deviation is small, the assistive force is nearly linear with the end-effector deviation, with the slope of the linear segment being K. Exceeding the saturation distance... The upper limit of muscle strength is then maintained. The values ​​remain unchanged. The calculated saturation distances for the three levels are 0.667m, 0.5m, and 0.4m, respectively, all of which are on the same order of magnitude as the end-of-life activity trajectory. In contrast, the end-of-life tracking deviation under normal training scenarios is on the order of 11mm, which is much smaller than the saturation distance. Therefore, the patient's force remains in the linear response zone during normal tracking, and the saturation mechanism only takes effect when there is a significant abnormal deviation at the end, preventing the assistive force from exceeding the physiological upper limit.

[0059] The patient's residual active assist force acts on the end effector, transmitting the saturation spring force through the geometric Jacobian matrix of the end effector. Mapped to joint torque And as the integral equation of rigid body dynamics for human-machine coupled systems. The actual input on the right, and the active control torque generated by the reinforcement learning policy network via the joint controller. They jointly participate in the step-by-step dynamics numerical integration, jointly determine the joint acceleration, and do not include the saturation spring force in the reward function. The inertia matrix, The Coriolis and centrifugal force matrix, For gravity, The active control torque generated by the joint controller is the strategy. The patient-assisted force is equivalent to a torque in joint space. This patient force serves as the link between the patient's subjective initiative and the robot's physical motion; it is neither an external disturbance nor a controllable quantity of the robot, but rather an objective component of the human-robot collaborative physical process. This coupled dynamics is numerically integrated step-by-step by the physics engine, with the patient force recalculated and applied in each sub-step. Within each physical sub-step, the patient force is recalculated and applied based on the current reference point position and the real-time end-effector position, which remain unchanged within the control step, ensuring its direction tracks the end-effector state in real time. Modeling the residual active patient-assisted force as a true input to the dynamic equations, rather than writing it into the reward function, has two core advantages: First, it ensures physical consistency; the end-effector motion perceived by the strategy is the result of the combined action of the robot's active control torque and the patient-assisted force, consistent with real human-robot collaborative scenarios. Second, it avoids reward leakage; if the patient force is written into the reward function, the patient assistance will be incorrectly encoded as a "reward source" for the strategy, causing the learned behavior to be inconsistent with real physical causality, thus affecting tracking accuracy and generalization ability.

[0060] S3. Policy Network Training: A Markov Decision Process (MDP) is established, and the Soft Actor-Critic (SAC) algorithm is used to train the human-machine cooperative trajectory tracking strategy. In the online phase, within the MDP framework, the cooperative tracking strategy is trained using the SAC algorithm, and the indexing rate of the reference trajectory is dynamically scheduled through the Adaptive Reference Point Advancement (ARP) mechanism. The auxiliary trajectory tracking task is modeled as a finite-time Markov Decision Process, and a quintuple is defined. The following steps are repeated in each control step until the trajectory is completed or the maximum number of steps is triggered, i.e., ... As the current reference point index for the next control step, until... Reaching M or reaching termination:

[0061] S31. Observation and Assembly: Acquiring Joint Angle Vectors Joint angular velocity vector and the actual position of the end effector The value, using the currently maintained reference point index. Extract the target reference point position from the dataset. Calculate the tracking error vector ,Will , , Error vector magnitude , and the progress of normalized trajectory completion Concatenate them to form the observation vector, i.e., the state space: observation vector Observation vector ,in, For normalized joint angle vectors, For normalized joint angular velocity vector, The Cartesian error vector of the end effector relative to the current reference path point. For scalar error modulus, Index for the current reference point To normalize the completion progress, reference point index With normalization progress As two components of the state vector, the policy perceives its own position in the trajectory, while the reference point position is transmitted through the error vector. Its magnitude and its indirect entry into observation enable the strategy to perceive "how far away from the target", and the two together constitute the basis for the strategy's local decision-making.

[0062] S32, Strategy Reasoning and Execution: Observation Vectors The input is the soft actor-critic (SAC) algorithm policy network, which is the actor network in the soft actor-critic algorithm based on the maximum entropy reinforcement learning framework. It uses a two-layer fully connected network with 256 neurons each. The output is a Gaussian distribution with mean and log-standard deviation, which is then compressed using tanh to obtain bounded actions. The action vector is obtained through reparameterized sampling. , action vector The first six dimensions of the model are scaled and used as joint angle increments, which are then superimposed on the current joint target to drive the end effector movement, thus obtaining the end effector position after execution. Action space: In step S32, the policy outputs a seven-dimensional action at each control step, resulting in an action vector. The first six dimensions are scaled and then added to the current joint target as joint angle increments: The seventh dimension is reserved as the gripper channel, which is fixed at 0 in this task; min and maxThese represent the lower and upper limits of the physical constraints of the joint, respectively, indicating the joint limit or safe hard stop of the robotic arm.

[0063] Policy optimization algorithm: The soft actor-commentator (SAC) algorithm is adopted to simultaneously maximize cumulative reward and policy entropy within the maximum entropy framework. The entropy term ensures that the policy continues to explore while completing the task, preventing premature entrapment in local optima, which is particularly important for non-stationary environments with patient force perturbations. The policy network outputs the mean and log-standard deviation of a Gaussian distribution. The policy network is a fully connected network with two hidden layers, each with 256 neurons and the ReLU activation function. The value network adopts a dual-commentator structure, identical to the policy network structure, to alleviate the Q-value overestimation problem. Reparameterization techniques are used to allow the policy gradient to backpropagate through Monte Carlo estimation. Tanh compression ensures bounded actions, and the log probability is adjusted by incorporating a log-Jacobi correction term from the tanh transformation. In practice, a numerically stable equivalent form is used. The temperature coefficient is automatically adjusted to maintain the target entropy. Hyperparameters: Batch size 512, learning rate 3×10⁻ 4 Discount factor 0.99, experience replay buffer capacity 10 6 .

[0064] S33, Physics Propulsion: In several MuJoCo physics substep cycles, policy torques are applied simultaneously within each substep. and the saturated spring force recalculated according to step S2 saturated spring force The direction is determined by the current reference point position, which remains unchanged within the current control step, and the real-time end effector position, but the saturated spring force is dynamically calculated with the real-time end effector displacement. They jointly participate in the rigid body dynamics integral to obtain the true joint state at the next moment;

[0065] S34. Adaptive Reference Point (ARP) Index Update: Based on the actual end-point position after physical propulsion. Calculate the rate coefficients under spatial channel, temporal channel, and sluggish detection. The reference point propulsion is obtained through joint scheduling of three signals: spatial channel, temporal channel, and sluggishness detection. And update the reference point index accordingly to obtain a new reference point index; Spatial channel: determine the propulsion feasibility based on the proximity of the end point to the reference point, i.e. ,in, The spatial distance threshold, The upper limit of the number of path points for a single control step and , , Preferably, ;when = When the maximum value is 3, the robot's end effector is in close contact with the current reference point; when > hour, Immediately reset to zero, the reference point completely awaits the end effector; the spatial channel is the main propulsion channel, employing a look-ahead, continuously satisfying criterion—based on Based on, since The process involves looking ahead at each waypoint, and the maximum number of waypoints consecutively satisfying the "endpoint is approaching" condition is used as the advance amount for the spatial channel. A maximum of three waypoints are advanced per step. The index is only advanced according to this consecutive waypoint count when several consecutive waypoints from the first look-ahead waypoint fall within a neighborhood centered on the current position of the end effector and with a spatial distance threshold as the radius. This automatically degrades the index to point-by-point advancement or even complete waiting in difficult road sections with geometric bends and dramatic curvature changes, preventing the index from jumping over road sections that the end effector has not actually traversed. The spatial channel uses the maximum number of consecutive waypoints falling within the distance threshold as the main advance amount. This criterion is a look-ahead continuous satisfaction criterion, rather than determining the reference point based on nearest point projection or single-point neighborhood determination. The effect is that the index will only cross over when the end effector has actually "crossed" the neighborhood of several consecutive waypoints, thus automatically degrading the index to point-by-point advancement or even complete waiting in difficult road sections with geometric bends and dramatic curvature changes, preventing the index from jumping over road sections that have not actually traversed.

[0066] The time channel serves as a fallback, providing a lower bound for the index that increases linearly with time based on the round duration to prevent the strategy from permanently stalling on difficult sections. Its rate coefficient is less than 1, ensuring that the maximum number of waypoints it can advance within the maximum control steps is strictly less than the total number of waypoints. Structurally, it cannot independently cover the entire trajectory. ,in For the time budget of each route point, This represents the maximum number of control steps executed in a single task. In this embodiment, the time channel rate coefficient is... =1000 control steps, M=51, therefore =1000 / 51≈19.6 control steps;

[0067] Stuck detection is used to detect coverage progress at the end point of coverage. When a strategy stalls, the advance rate of the time channel is reduced. Stuck detection identifies strategy stalls on a longer time scale and is a rate adjustment mechanism for the time channel. When the end point continuously exceeds a stall threshold without achieving a spatial channel gain greater than zero, the time channel rate coefficient is reduced, thereby decreasing the advance rate of the time channel and giving the end point more time to catch up. Stuck detection lowers the time channel rate coefficient when the end point stalls beyond a threshold. ,in The last time the end effector covered the new waypoint, i.e., the last time it was used Control step time greater than zero, The stagnation threshold and ,satisfy The key design feature of this invention lies in the fact that the time channel is intentionally designed to be underdriven: because the rate coefficient is less than 1, the time channel... Theoretical upper limit of the number of waypoints that can be advanced internally. The number of waypoints, approximately 55% of the complete trajectory, is strictly less than the total number of waypoints M. Therefore, the time channel cannot structurally cover the entire reference trajectory independently, and the main propulsion of the reference point index must be supplemented by the spatial channel. This mechanism ensures that the main propulsion is driven by the actual tracking state at the end point, and the time channel only plays a backup role, rather than degenerating into an open-loop time law with a lower limit. , In this embodiment, the control frequency of the control step is 50Hz. Each control step is a stagnation threshold; when the end continuously exceeds... When the control step does not cover any waypoints, Depend on Reduce to This gives the end-of-pipe team more time to catch up; preferably, , This limits the number of waypoints that can be advanced within the time tunnel to approximately 55% of the total number of waypoints. The maximum value between the spatial tunnel and the time tunnel is taken as the reference point advancement amount. ,Right now ; Set the current index Advance from reference point Summing and truncating the sum with the minimum value of the total number of waypoints M, i.e. The reference point index is updated after the dynamical numerical integration and before the reward calculation, to increase the propulsion. With tracking error vector All calculations are based on the actual position of the end effector after this control step. An adaptive reference point advancement mechanism indexes the reference point. The update mechanism has been transformed from an open-loop function of time to a closed-loop feedback function of the terminal state. The propulsion rate is jointly scheduled by three components: spatial channel, temporal channel, and lag detection. This solves the problem of tracking error divergence caused by reference point disconnection. The average trajectory compliance rate across random seeds is 98%, the average tracking error is 11.2 mm, and there are zero safety violations. The spatial channel reflects the instantaneous deviation between the terminal state and the reference point, while the temporal channel reflects the cumulative progress over the round time dimension. The former characterizes "whether the current position is close to the reference point," and the latter characterizes "whether the overall system is keeping up with the pace." Their functions are complementary rather than redundant.

[0068] S35. Reward Calculation: Based on the aforementioned propulsion amount Calculate progress rewards End-stage bonus and milestone rewards Both progress rewards and tail-end rewards use the reference point advance as the independent variable, allowing the advance rate of the adaptive reference point advance mechanism to directly modulate the temporal distribution of the reward signal—during terminal deceleration, the reference point waits and the advance decreases, preventing the strategy from receiving misleading rewards in an invalid state where the reference point has already moved far away; after terminal tracking is restored, the advance returns to positive, and the reward is realigned to the currently reachable reference point. This is based on the error after physical advance. and The tracking reward, smoothing penalty, and violation component are calculated and summed into a scalar reward value. In step S35, progress reward ,in End-stage bonus ,in, Milestone Awards Among them, when 80% of the nodes are reached, When the 100% node is reached, Tracking rewards ,in , Smooth penalty ,in, ; illegal quantity ,in , , Scalar reward value The reward function consists of six components, designed to balance trajectory accuracy, progress, and motion smoothness. Milestone rewards and tail-end rewards are triggered concurrently at the 80% and 100% nodes, respectively providing sparse positive rewards and dense decaying rewards, jointly suppressing the reward sparsity problem in the later stages of long-range trajectories. Both progress rewards and tail-end rewards are based on... Therefore, the adaptive propulsion rate of this mechanism directly modulates the temporal distribution of the reward signal. In difficult sections where deceleration occurs at the end, the reference point enters a waiting state, and the time channel provides a safety net to complete the slow propulsion. This significantly reduces the amount of rewards, which complements the "continuous small incentive" provided by the exponential decay of tracking rewards, preventing the strategy from receiving misleading rewards in an ineffective state where the reference point has long since disappeared; after resuming tracking at the end, Returning to a positive value, the progress reward is realigned to the currently reachable reference point, guiding the strategy to continue. This reward interface is a key difference between this invention and the virtual target point velocity adaptation approach in conventional path following control: in conventional approaches, the reference point advancement law only applies to the controller's error calculation, while in this invention, the advancement amount... At the same time, it is also the independent variable of the reinforcement learning reward function, forming an additional closed-loop coupling between the scheduler and the learning process.

[0069] The above design constitutes three feedback pathways across two time scales: at the control step scale, an increase in tracking error in the physical pathway leads to a linear increase in patient force, pushing the endpoint towards the reference point; in the scheduling pathway, if the tracking error exceeds a threshold, the spatial channel resets to zero, and the reference point waits for the endpoint; at the round scale, the reward signal in the learning pathway drives the evolution of network parameters via the playback buffer. The physical and scheduling pathways are triggered in parallel at the control step scale and act in the same direction, creating a two-way "push and wait" squeeze on the current error, forming synergistic feedback rather than mutual cancellation, which is the structural guarantee of closed-loop stability.

[0070] S36. Next-state assembly and storage: Using the new index Reread the new reference point and calculate the reference point. Assemble the next observation The quadruple of this interaction ( , , , Store in the experience replay buffer;

[0071] S37. Gradient Update: When the amount of data in the experience replay buffer exceeds the warm-up threshold, a batch of samples is randomly sampled, the loss function is calculated sequentially, the dual critic network, the policy network, and the adaptive temperature coefficient are updated, and the network gradient update is performed; the parameters of the policy network and the critic network are sampled and updated from the experience replay buffer until convergence, and the converged policy network parameters are deployed to the controller of the upper limb rehabilitation robot.

[0072] S4. After completing the strategy training, load the trained strategy network and output joint control commands to drive the upper limb rehabilitation robot to complete human-machine collaborative trajectory tracking rehabilitation training.

[0073] It also includes safety monitoring steps: real-time acquisition of the interaction force modulus between the end effector and the towed object. With interaction torque modulus ,when Exceeding the force safety threshold Or when Exceeding the torque safety threshold The system records all such occurrences, preferably... , .

[0074] This embodiment verifies the control method through simulation experiments. The evaluation indicators used in the simulation include: coverage rate (completeness of each trajectory); PR (percentage of trajectories with coverage ≥ 80%, directly reflecting system availability); CR (average coverage of all trajectories); TE (tracking error, only counting the tracking error at the end position of compliant trajectories); IF (average interaction force, the average contact force between the end of all trajectories and the environment, in Newtons (N); and SV (number of safety violations, counted during simulation when the auxiliary force is > 10 N or the torque is > 10 N). The number of events. It should be noted that the auxiliary force safety threshold in this embodiment... and torque safety threshold The settings are based on engineering experience values ​​for upper limb rehabilitation robots and are auxiliary rather than therapeutic thresholds. The contact force thresholds for clinical rehabilitation equipment are typically lower, generally 3N-5N. This embodiment uses 10N as the safety boundary within the simulation domain to ensure that the strategy can be effectively flagged and identified when anomalies occur. In actual clinical deployment, the above thresholds must be reset based on the specific patient's rehabilitation assessment results, individual tolerance, and relevant medical device regulatory requirements; this embodiment does not impose such limitations.

[0075] The simulation experiment used three random seeds (42 / 123 / 456) for complete training and testing respectively. Each seed tested 100 trajectories. The main experimental results are shown in the table below.

[0076]

[0077] As shown in the table above, all three random seeds achieved initial convergence within approximately 1M steps. The cross-seed success rate was 98.0 ± 1.6%, indicating that the framework can reliably converge under different training initializations. The standard deviation of coverage across 100 trajectories within a single seed ranged from 0 to 9%, which, compared with the cross-seed standard deviation (approximately 1.6%), suggests that the main performance bottleneck lies in the geometric difficulty of the trajectories rather than the randomness of training. Furthermore, there were zero security violations in 300 tests, and the average interaction force was 0.186N ± 0.008N, far below the saturation threshold of 5N.

[0078] To further verify the effectiveness of each design component, this embodiment, under the condition of a fixed random seed Seed=42, removed five design components for ablation experiments. Each group underwent a complete 8M-step training run, and a control group with "patient force disabled during testing" was added. The ablation baseline was the result of a single training run with the complete model Seed=42, achieving a compliance rate of 97%. The difference from the 98% in the main results table stemmed from the initial random numbers used in different test runs; both were within the standard deviation across seeds (±1.6%) and did not affect the validity of the conclusions. The ablation experimental results are shown in the table below.

[0079]

[0080] The specific analysis of the ablation experiment is as follows: (1) When the adaptive reference point propulsion mechanism was removed, all 100 trajectories failed to meet the standard, with a coverage rate of only 44.4% ± 9.1%. The reference disconnection failure mainly occurred at the coverage rate of 25% to 75%. The compliance rate dropped sharply from 97% to 0, a decrease of 97 percentage points, which was the most significant among the five groups. This directly confirms the analysis of the fixed-rate propulsion-induced reference disconnection failure in this invention. Under this configuration, the reference point propulsion method was replaced by an open-loop propulsion driven only by the bottom-line time channel and the jam detection, losing the ability to "wait" based on the end arrival situation. When the end decelerates in the geometrically difficult road section, the reference point continues to advance, and the tracking error monotonically increases with time until it gets out of control. It is worth noting that if a rough calculation is performed based on all trajectories (including those that did not meet the standards), the average error is approximately 7.0 mm, which is lower than the 12.8 mm of the complete model. This is not a performance improvement, but rather a distortion of the indicator caused by the definition of the reference window. That is, when the strategy stagnates and the reference point stops advancing, there are still early path points relatively close to the endpoint within the window, and the error is calculated based on these nearby points, resulting in a lower error. This phenomenon reveals that tracking error must be considered in conjunction with coverage to accurately reflect tracking quality.

[0081] (2) Patient force removal vs. "Patient force off during testing": The patient force removal group (no patient force during both training and testing) achieved a target success rate of 72.0%, but the tracking error increased to 31.8 mm, approximately 2.5 times that of the complete model. The coverage standard deviation increased from ±5.2% to ±20.7%. This result indicates that the strategy can independently converge to a basically usable tracking capability in a training environment with no patient force, but accuracy and stability are significantly reduced. The "patient force off during testing" control group used training weights identical to those of the complete model, i.e., K=10 during the training phase. =5N, with patient force set to zero only during testing, the target achievement rate plummeted from 97% to 0%. Both groups had the same testing conditions (no patient force) but different training conditions. The former achieved 72% of the target, while the latter completely failed. This difference indicates that the performance loss in the latter did not come from the "no patient force" itself, but from the inconsistency between the distribution of training and testing phases.

[0082] This allows for a more precise explanation of the behavior of the complete model: the persistent patient force during training causes the strategy to converge to a tracking mode that coordinates with the patient force, rather than autonomous tracking. The average interaction force of the complete model is only 0.196N, far below the safety threshold. This phenomenon has often been interpreted as "the strategy does not depend on the patient force," but this control shows that this interpretation is invalid—the low interaction force is a mathematical result of "small end-effector bias → small patient force," rather than causal evidence that "the strategy does not require patient force." This coordinated mode is consistent with the clinical paradigm of "robot and patient jointly completing actions" advocated by on-demand rehabilitation robots, and can be seen as an expected product of modeling patient force as a physical environment term. In clinical deployment scenarios, residual muscle force of the patient is always present; there is no "complete shutdown." Therefore, this control reveals the essential characteristics of the strategy's behavior, rather than a limitation on its actual usability.

[0083] (3) Other components: After removing the tail reward, the achievement rate dropped from 97% to 94%, and the newly added failed trajectories all stopped at about 90% coverage, which is consistent with the design motivation that "the cumulative error in the later stage of the trajectory and the configuration constraints increase the difficulty of coverage". After removing the course sampling, the final value achievement rate remained basically the same, reaching 98%, and the standard deviation of the coverage rate increased from ±5.2% of the complete model to ±11.3%, indicating that the role of course sampling is not to improve the final value, but to reduce the exploration cost of difficult trajectories and improve the consistency between trajectories. Under the condition of 8M steps of sufficient training, uniform sampling can also cover difficult trajectories with sufficient sampling times, so the difference in the final value index is not significant. This mechanism should make a more significant contribution to scenarios with stricter evaluation thresholds or shorter training steps. After removing the milestone reward, the various indicators are very close to the complete model. Since the progress reward and the tail reward have provided sufficiently dense gradient signals, the incremental contribution of the milestone reward is partially absorbed.

[0084] Based on the combined results of the five ablation groups, the five design components are divided into three tiers according to their contribution levels: The first tier is the adaptive reference point advancement mechanism, which makes a decisive contribution. Its removal reduces the target achievement rate to zero and is a necessary condition for the feasibility of the framework; The second tier is the physical layer coupling modeling of patient force, which makes an important contribution. Its removal reduces the tracking error to approximately 2.5 times that of the complete model and decreases the target achievement rate by 25 percentage points. It is a key support for accuracy and stability; The third tier consists of tail-end rewards, course sampling, and milestone rewards, which make supplementary contributions. The three components work together to optimize the tail-end coverage capability and training consistency.

[0085] A human-machine collaborative trajectory tracking control system for an upper limb rehabilitation robot, used to execute control methods, including:

[0086] The trajectory library generation module is used to interpolate and solve inverse kinematics for clinical activities of daily living (ADL) prescription task trajectories to generate a joint space reference trajectory dataset.

[0087] The patient ability grading module is configured to grade the patient's upper limb motor function based on the Fugl-Meyer Upper Limb Function Assessment Scale, and determine the patient's ability level and the corresponding saturation spring force parameter.

[0088] The human-machine coupled dynamics simulation module is configured to construct a coupled rigid body dynamics model that includes a robot and a patient with residual muscle strength. The saturated spring force is used as a real physical external force and mapped to the joint space through a geometric Jacobian matrix to participate in the dynamics integration.

[0089] The reinforcement learning training module is configured to train a cooperative tracking strategy based on the SAC algorithm within the Markov decision process framework. The cooperative tracking strategy takes the observation vector as input and the joint angle increment as the action output.

[0090] The adaptive reference scheduling module is configured to perform each control step of the reinforcement learning strategy, including independent spatial channel units, temporal channel units, lag detection units, and index synthesis modules. The spatial channel units are used to perform spatial channel operations, the temporal channel units are used to perform temporal channel operations, the lag detection units are used to perform lag detection operations and provide the output temporal channel rate coefficients to the temporal channel units, and the index synthesis module feeds back the new reference point index to the observation assembly and the spatial channel units. The index advance of the reference trajectory is calculated jointly by the three components of spatial channel, temporal channel, and lag detection, and the current reference point position is dynamically updated so that the reference point advance rate is adjusted by the real-time tracking status feedback of the end effector.

[0091] The reward calculation module is configured to calculate a scalar reward value based on the updated reference point index, which includes tracking reward, progress reward, tail reward, milestone reward, smoothing penalty and violation penalty, for parameter updates of the SAC algorithm.

[0092] The control module is used to load the trained policy network and output joint control commands to drive the upper limb rehabilitation robot to complete human-machine collaborative trajectory tracking rehabilitation training.

[0093] It should be understood that the above embodiments are only used to explain the present invention and only illustrate several implementation methods of the present invention. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. Those skilled in the art can make several modifications and improvements without departing from the concept of the present invention: the upper limb rehabilitation robot is not limited to a six-degree-of-freedom serial robotic arm, but can be a serial robotic arm with other degrees of freedom, a parallel mechanism, or an exoskeleton mechanism; the training algorithm of the policy network is not limited to the soft actor-critic algorithm, but can be other deep reinforcement learning algorithms based on policy gradients; the patient's ability level is not limited to three levels, and the clinical scale is not limited to the Fugl-Meyer Upper Limb Function Assessment Scale, but can be other rating scales capable of quantifying the patient's residual motor function; the proximity condition of the spatial channel is not limited to Euclidean distance threshold determination, but can adopt weighted distance, ellipsoidal neighborhood, or a composite criterion considering posture deviation; the reference trajectory is not limited to daily living activity trajectories, but can be applied to other forms of rehabilitation training trajectories; these all fall within the protection scope of the present invention. Obvious variations or modifications derived from the spirit of the present invention are still within the protection scope of the present invention.

Claims

1. A human-machine collaborative trajectory tracking control method for an upper limb rehabilitation robot, characterized in that, include: S1. Offline reference trajectory generation: Build a robot joint space reference trajectory dataset and generate a waypoint sequence containing M waypoints; S2. Constructing a human-machine coupled environment: Divide the patient into multiple motor function levels and configure a corresponding stiffness coefficient for each motor function level. With upper limit of force saturation The patient's residual muscle strength was modeled as a saturated spring force. ; the saturated spring force Mapped to joint space via geometric Jacobian matrix, and used for rigid body dynamics integration; S3. Policy Network Training: Establish a Markov decision process and use the soft actor-critic algorithm to train the human-machine cooperative trajectory tracking strategy. Repeat the following steps in each control step until the trajectory is completed or the maximum number of steps is triggered: S31. Observation and Assembly: Acquiring Joint Angle Vectors Joint angular velocity vector and the actual position of the end effector The value, using the currently maintained reference point index. Extract the target reference point position from the dataset. Calculate the tracking error vector ,Will , , Error vector magnitude , and the progress of normalized trajectory completion Concatenate to construct observation vectors ,in ; S32, Strategy Reasoning and Execution: Observation Vectors Input a soft actor-critic policy network, output the mean and log-standard deviation of a Gaussian distribution, and obtain the action vector through reparameterization sampling. , action vector The first six dimensions are scaled and used as joint angle increments, which are then superimposed onto the current joint target. S33, Physics Propulsion: In several physics sub-step cycles, a strategy torque is applied simultaneously within each sub-step. and the saturation spring force recalculated at each step The saturated spring force Within the current control step, the reference point remains unchanged, but is dynamically calculated along with the real-time end displacement, and participates in the rigid body dynamics integral to obtain the true joint state at the next moment; S34. Adaptive Reference Point Index Update: Based on the actual end-point position after physical propulsion. Calculate the rate coefficients under spatial channel, temporal channel, and sluggish detection. The reference point propulsion is obtained through joint scheduling of three signals: spatial channel, temporal channel, and sluggishness detection. And update the reference point index accordingly to obtain the new reference point index; S35. Reward Calculation: Based on the aforementioned reference point advance. Calculate progress rewards End-stage bonus and milestone rewards Error based on physical propulsion and The tracking reward, smoothing penalty, and violation component are calculated and summed into a scalar reward value. ; S36. Next-state assembly and storage: Using the new index Reread the new reference point Assemble the next observation The quadruple of this interaction ( , , , Store in the experience replay buffer; S37. Gradient Update: When the amount of data in the experience replay buffer exceeds the preheating threshold, a batch of samples is randomly sampled, the loss function is calculated sequentially, the dual critic network, the policy network and the adaptive temperature coefficient are updated, and the network gradient update is performed. S4. After completing the strategy training, load the trained strategy network and output joint control commands to drive the upper limb rehabilitation robot to complete human-machine collaborative trajectory tracking rehabilitation training.

2. The human-machine collaborative trajectory tracking control method for an upper limb rehabilitation robot according to claim 1, characterized in that, In step S1, the joint space reference trajectory dataset is generated from clinical daily living activity ability prescriptions and is used for point-by-point indexing and advancement by online ARP. The dataset is processed as follows: S11, Cartesian level: The positions of key waypoints are interpolated using cubic B-spline, and the attitude is interpolated using quaternion spherical linear interpolation. The trajectory arc length is reparameterized and scattered at equal intervals to form a waypoint sequence, so that the rehabilitation robot advances with equal end displacement each time. S12, Joint Level: The inverse kinematics are solved point by point using the damped least squares method, and a hot start acceleration is performed; First, use trapezoidal velocity planning, then use a fifth-order polynomial for smoothing, and output a reference trajectory with timestamps; S13. Data verification: Before data is entered into the database, accessibility, continuity, and dynamic feasibility are verified. If the data is qualified, it is included in the dataset to generate a waypoint sequence for online indexing. If the data is unqualified, local replanning is triggered, and steps S11-S13 are repeated.

3. The human-machine collaborative trajectory tracking control method for an upper limb rehabilitation robot according to claim 1, characterized in that, In step S2, the motor function level is classified according to the Fugl-Meyer Upper Limb Function Assessment Scale, and the saturation spring force is: , where the end deviation vector , This is the current reference point location. The current position of the robot's end effector. Let be the unit direction vector pointing to the reference point, when hour Take the zero vector. This is the spring stiffness coefficient. This represents the upper limit of force saturation. The saturation spring force is applied through the geometric Jacobian matrix of the end effector. Mapped to joint torque The integral equation of rigid body dynamics is: .

4. The human-machine collaborative trajectory tracking control method for an upper limb rehabilitation robot according to claim 1, characterized in that, In step S32, the action vector The joint angle increment .

5. The human-machine collaborative track for an upper limb rehabilitation robot according to claim 1 The trace tracking control method is characterized by, In step S34, the spatial channel ,in, , Preferably, ;when = At that time, the robot's end effector is in close contact with the reference point; when > hour, Immediately reset to zero, the reference point completely awaits the end effector; the time channel ,in , This represents the maximum number of control steps executed in a single task. This represents the time channel rate coefficient.

6. The human-machine collaborative trajectory tracking control method for an upper limb rehabilitation robot according to claim 1, characterized in that, In step S34, the stagnation detection is used to detect coverage progress at the end point, and to reduce the time channel rate to extend the policy catch-up time when the policy stagnates. The time channel rate coefficient ,in The last time the end effector was used Control step time greater than zero, The stagnation threshold and .

7. The human-machine collaborative trajectory tracking control method for an upper limb rehabilitation robot according to claim 6, characterized in that, , , The control frequency of the control step is 50Hz. One control step; preferably, , .

8. The human-machine collaborative trajectory tracking control method for an upper limb rehabilitation robot according to claim 1, characterized in that, In step S35, the progress reward ,in The tail segment reward ,in, The milestone reward Among them, when 80% of the nodes are reached, When the 100% node is reached, The tracking reward ,in , The smoothing penalty ,in, The aforementioned non-compliant component ,in , , The scalar reward value .

9. The human-machine collaborative trajectory tracking control method for an upper limb rehabilitation robot according to claim 1, characterized in that, It also includes safety monitoring, and real-time acquisition of the interaction force modulus between the end effector and the towed object. With interaction torque modulus ,when Exceeding the force safety threshold Or when Exceeding the torque safety threshold The system records all such occurrences, preferably... , .

10. A human-machine collaborative trajectory tracking control system for an upper limb rehabilitation robot, characterized in that, For performing the control method according to any one of claims 1-9, comprising: The trajectory library generation module is used to interpolate and solve inverse kinematics for clinical daily living activity ability prescription task trajectories, and generate a joint space reference trajectory dataset. The patient ability grading module is configured to grade the patient's upper limb motor function based on the Fugl-Meyer Upper Limb Function Assessment Scale, and determine the patient's ability level and the corresponding saturation spring force parameter. The human-machine coupled dynamics simulation module is configured to construct a coupled rigid body dynamics model that includes a robot and a patient with residual muscle strength. The saturated spring force is mapped to the joint space as a real physical external force through a geometric Jacobian matrix and participates in the dynamics integration. The reinforcement learning training module is configured to train a cooperative tracking strategy based on the soft actor-critic algorithm within the Markov decision process framework. The cooperative tracking strategy takes the observation vector as input and the joint angle increment as the action output. The adaptive reference scheduling module is configured to jointly calculate the index advance of the reference trajectory through three components—spatial channel, temporal channel, and lag detection—in each control step of executing the reinforcement learning strategy, dynamically update the current reference point position, and make the reference point advance rate adjusted by the real-time tracking status feedback of the end effector. The reward calculation module is configured to calculate a scalar reward value based on the updated reference point index, which includes tracking reward, progress reward, tail reward, milestone reward, smoothing penalty and violation penalty, for parameter updates of the soft actor-critic algorithm; The control module is used to load the trained policy network and output joint control commands to drive the upper limb rehabilitation robot to complete human-machine collaborative trajectory tracking rehabilitation training.

Images