A mechanical arm trajectory tracking control method and system based on a double-layer controller

Abstract

The application discloses a kind of mechanical arm trajectory tracking control method and system based on double-layer controller, comprising the following steps: S1: multiple Actor networks and multiple Critic networks are constructed in control strategy optimization layer.S2: the real-time environment state of mechanical arm is input to each Actor network to generate multiple different trajectory action information, and the optimal trajectory action A is obtained by screening using multiple Critic networks, and the optimal trajectory action instruction is transmitted to trajectory tracking control layer.S3: trajectory tracking control layer calculates total control torque, and drives mechanical arm movement.S4: the new joint state after mechanical arm movement is fed back to control strategy optimization layer, and the next trajectory action instruction is calculated.The application generates mechanical arm control strategy in upper control strategy optimization layer first, the global trajectory control instruction generated is tracked by lower trajectory tracking control layer, and the joint state of mechanical arm is fed back in real time, so as to gradually optimize movement trajectory, and the motion precision is higher, and the system calculation convergence speed is faster.

Description

Technical Field

[0001] This invention relates to the field of robotic arm trajectory tracking control technology, and in particular to a robotic arm trajectory tracking control method and system based on a dual-layer controller. Background Technology

[0002] In fields such as industrial automation and intelligent robotics, trajectory tracking control is a core component for achieving precise operations. Proportional-integral-derivative (PID) control, as a classic feedback control method, has long been widely used in the motion control of equipment such as robotic arms, CNC machine tools, and industrial machinery due to its advantages of simple structure, ease of implementation, and high reliability. However, as intelligent manufacturing increasingly demands higher control precision, response speed, and anti-interference capabilities from equipment, the inherent shortcomings of traditional PID control are becoming increasingly apparent.

[0003] Reinforcement learning, as an intelligent control technique based on trial and error, has gradually become a research hotspot in the field of complex system control due to its powerful policy adaptation capabilities and global optimization potential. Through continuous interaction between the agent and the environment, reinforcement learning optimizes the control strategy with the goal of maximizing cumulative rewards. It does not rely on precise system models and can adaptively handle uncertainties and nonlinear problems in complex scenarios, improving the robustness of control systems in scenarios such as robotic arms. However, existing methods often employ simple concatenation or single optimization logic, making it difficult to achieve deep synergy between global optimization and local tracking in robotic arm control. Their generalization ability under different complexity scenarios still needs improvement, and their suppression effect on dynamic disturbances is not ideal.

[0004] For example, patent document CN120217577A discloses a dynamic simulation method for trajectory planning and tracking of a rigid-flexible coupled manipulator based on intelligent algorithms. The scheme first uses the second type of Lagrange equations to dynamically model the two-link rigid-flexible coupled manipulator; then, for point-to-point motion tasks, using the residual vibration energy of the flexible link as the optimization objective, it employs a particle swarm optimization algorithm for trajectory planning, searching for an ideal joint trajectory that can effectively suppress residual vibration; finally, to drive the manipulator to accurately follow the planned high-quality trajectory, a deep reinforcement learning-based method is used to track the ideal joint trajectory in real time. The trajectory planning and tracking performance of this method heavily relies on the accuracy of the initial dynamic modeling and is easily affected by unknown external disturbances, leading to slow system control convergence speed and poor convergence performance.

[0005] For example, patent document CN120663327A discloses a PID control method for a hydraulic robotic arm based on reinforcement learning. Its steps include: acquiring hydraulic cylinder displacement data through a displacement sensor; designing a PID controller based on the deviation between the displacement signal and the command signal; and automatically adjusting the time-varying gain of the controller in real time using an evaluation neural network-execution neural network architecture to correct the PID parameters. Finally, based on the optimized gain parameters, high-precision tracking control of the hydraulic cylinder is achieved. However, control strategies trained for specific hydraulic robotic arms and operating conditions can only achieve certain results in some hydraulic robotic arm controls. When faced with new motion trajectories, their generalization ability is insufficient, requiring retraining for the new motion trajectory.

[0006] In summary, designing a control method that can accurately track both the local and global aspects of a robotic arm's operating cycle is a technical problem that needs to be solved. Summary of the Invention

[0007] To address the technical problems of response lag and insufficient generalization in existing control algorithms for robotic arm trajectory planning and control, this invention provides a robotic arm trajectory tracking control method and system based on a two-layer controller to solve the above problems.

[0008] This invention proposes a robotic arm trajectory tracking control method based on a two-layer controller, comprising the following steps: S1: Construct a control strategy optimization layer and a trajectory tracking control layer. In the control strategy optimization layer, construct multiple Actor networks and multiple Critic networks.

[0009] S2: Real-time environmental status of the robotic arm Each Actor network generates multiple different trajectory action information. Multiple Critic networks are used to evaluate the value of multiple trajectory action information, and the optimal trajectory action A is selected. The optimal trajectory action command is then passed to the trajectory tracking control layer.

[0010] S3: The trajectory tracking control layer calculates the total control torque based on the optimal trajectory action command. And drive the robotic arm to move based on the calculation results.

[0011] S4: Feed back the new joint state after the robotic arm moves to the control strategy optimization layer, and continue to calculate the next trajectory action command until the robotic arm reaches the target position.

[0012] In an optional embodiment of the present invention, the real-time environmental status of the robotic arm Including the joint status of the robotic arm External environmental condition E, target position of robotic arm movement .

[0013] The joint state of the robotic arm Including actual joint angles Actual joint angular velocity Joint angle error and joint angular velocity error ,in, , , For the desired joint angle; The desired joint angular velocity.

[0014] The external environmental state E includes external contact force. System dynamic disturbances and sensor noise .

[0015] The target position of the robotic arm movement Including expected joint angles Desired joint angular velocity Desired joint acceleration And the target pose T of the robotic arm.

[0016] In an optional embodiment of the present invention, the Actor network includes an Actor1 network and an Actor2 network. The two Actor networks extract candidate trajectories A1 and A2 through a reward function, wherein the reward function is: ,in, A reward for the robotic arm reaching the target position. The penalty to be given for each step of execution. As a penalty for collision, and These are the weight coefficients in the Actor1 network. Less than In the Actor2 network Greater than .

[0017] In an optional embodiment of the present invention, the Critic network includes a Critic1 network, a Critic2 network, and a Critic3 network, and each Critic network calculates the combined value of A1 and A2 based on a weighted fusion rule. The optimal trajectory action is selected through a comprehensive value screening rule, where i is the Actor network ordinal number, i=1,2.

[0018] In an optional embodiment of the present invention, the comprehensive value The calculation method is as follows: Based on the trainable parameters of each Critic network The candidate trajectories A1 and A2 are evaluated for value, and six raw Q values ​​are output: , , , , , Where j is the Critic network ordinal number, j=1, 2, 3.

[0019] Calculate the overall value: .

[0020] Filtering the optimal trajectory action: ,in, , These are the policy functions for Actor1 and Actor2, respectively. , These are the network parameters for Actor1 and Actor2, respectively.

[0021] In an optional embodiment of the present invention, the total control torque The calculation method includes the following steps: S31: Calculate the feedforward compensation torque based on the optimal trajectory motion command information. This counteracts the dynamic interference of the robotic arm's inertial force, gravity, and centrifugal force.

[0022] S32: Design the fitness function J to balance the parameters to be optimized in the PID controller.

[0023] S33: Calculate the feedback control quantity based on the optimized PID parameters in step S32. .

[0024] S34: Feedforward compensation torque With feedback control quantity Superimposed to form the total control torque .

[0025] In an optional embodiment of the present invention, the feedforward compensation torque The calculation formula is: .

[0026] Where M is the inertia matrix, N is the Coriolis force and centrifugal force matrix, and G is the gravity vector.

[0027] In an optional embodiment of the present invention, the formula for the fitness function J is: .

[0028] .

[0029] in, To control energy consumption, Adjustment time required for system stability , , , These are the weighting coefficients; This is the proportionality coefficient. The integral coefficient is... is the differential coefficient.

[0030] Feedback control quantity The calculation formula is: .

[0031] In an optional embodiment of the present invention, the control strategy optimization layer further establishes a training data temporary storage area, the working process of which is as follows: After executing the optimal trajectory action A, the environment is fed back with a new state. Task reward value Rv forms interaction samples ( A, Rv, Store the training data in the temporary storage area.

[0032] Randomly sample a small batch of samples dr from the training data temporary storage area, and update the parameters using a cross-update function: first optimize the parameters of the Critic network based on the actions and states of the Actor1 network, and then adjust the parameters of the Actor2 network based on the updated Critic network feedback; then reverse the process, first optimizing the parameters of the Critic network based on the actions and states of the Actor2 network, and then adjusting the parameters of the Actor1 network.

[0033] The present invention also proposes a system that operates using the above-described robotic arm trajectory tracking control method based on a two-layer controller. The system includes a control strategy optimization layer and a trajectory tracking control layer. The control strategy optimization layer includes multiple Actor networks for generating candidate actions, multiple Critic networks for value evaluation, and a trajectory command standardization output unit. The trajectory tracking control layer includes a total control torque calculation and command execution and real-time status feedback unit.

[0034] The beneficial effects of this invention are: (1) In this invention, the control strategy of the robotic arm is generated in the upper control strategy optimization layer. The generated global trajectory control command is tracked by the lower trajectory tracking control layer and the joint status of the robotic arm is fed back in real time, thereby gradually optimizing the motion trajectory. Compared with the traditional control method, the robotic arm motion accuracy is higher and the system calculation convergence speed is faster in this invention.

[0035] (2) The present invention uses a dual Actor network to generate candidate trajectory actions in parallel, and uses a three-Critic network to quantify and evaluate the trajectory value to select the optimal trajectory action. Compared with the traditional model training method, the dual Actor network expands the model strategy selection and avoids local optima caused by single preference. The three-Critic network provides a more stable Q value, enabling the model to handle various extreme scenarios and enhance the model's generalization. Attached Figure Description

[0036] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0037] Fig. 1 This is a flowchart of the robotic arm trajectory tracking control method based on a dual-layer controller as described in this invention; Fig. 2 This is a schematic diagram of the hierarchical structure of the control strategy optimization layer and the trajectory tracking control layer in this invention. Detailed Implementation

[0038] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0039] like Figs. 1-2 As shown, a robotic arm trajectory tracking control method based on a two-layer controller includes the following steps: S1: System initialization, constructing the control system.

[0040] Specifically, this involves constructing a control strategy optimization layer and a trajectory tracking control layer. The control strategy optimization layer builds multiple Actor networks and multiple Critic networks. The control strategy optimization layer is used for motion trajectory planning and selection, while the trajectory tracking control layer calculates the driving force based on the motion trajectory determined by the control strategy optimization layer and executes the corresponding action. The Actor networks generate candidate actions, and the Critic networks evaluate the value of these candidate actions, providing a basis for selecting the optimal action.

[0041] S2: Select the optimal trajectory A.

[0042] Real-time environmental status of the robotic arm Each Actor network generates multiple different trajectory action information. Multiple Critic networks are used to evaluate the value of multiple trajectory action information, and the optimal trajectory action A is selected. The optimal trajectory action command is then passed to the trajectory tracking control layer.

[0043] Real-time environmental status of robotic arm This refers to the joint state of the robotic arm. External environmental condition E, target position of robotic arm movement Among them, the joint status of the robotic arm The parameters included are: actual joint angles. Actual joint angular velocity Joint angle error and joint angular velocity error , , , For the desired joint angle; The desired joint angular velocity. The external environmental state E includes parameters such as: external contact force. (For example, the contact force on the Z-axis of the robotic arm), system dynamic disturbances (e.g., sudden changes in inertial force, fluctuations in joint friction) and sensor noise The target position of the robotic arm movement. The parameters included are: desired joint angles. Desired joint angular velocity Desired joint acceleration And the target pose T of the robotic arm.

[0044] In this invention, multiple Actor networks and Critic networks are independently set up. Traditional single-Actor networks or dual-Actor networks have problems such as large Q-value estimation and difficulty in model training convergence. This invention uses multiple Actor networks to generate candidate trajectories with different focuses, and then uses multiple independent Critic networks for value evaluation. The problem of inaccurate Q-value is solved by weighted fusion, which enables the model to handle various extreme scenarios and enhances the model's generalization ability.

[0045] However, an excessive number of Actor and Critic networks can lead to increased training costs and slow model response. Therefore, this invention preferably employs a combination of two Actor networks and three Critic networks. Specifically, the Actor network includes Actor1 and Actor2, where Actor1 focuses on minimizing obstacle avoidance distance during training, while Actor2 focuses on smoother motion. Actor1 generates candidate action A1, and Actor2 generates candidate action A2. The Critic network comprises three independent Critic1, Critic2, and Critic3 networks.

[0046] The emphasis of the two Actor networks during training is reflected by the reward function Rv, which is formulated as follows: (1) In the formula, The closer the robotic arm is to the target position, the greater the reward. The larger; when the target position is fully reached, Take the maximum value. This is a penalty for each step, meaning that if the trajectory task is not completed, the reward will be deducted for each time step. The more steps and the longer the time taken, the lower the total reward. Its purpose is to force the system to speed up the movement and shorten the trajectory completion time, avoiding sluggishness and hesitation. A value less than 0 is essentially a negative reward. Collision penalty, which is a penalty for touching an obstacle or being too close to an obstacle, serves to constrain the movement trajectory, guide the robotic arm to actively avoid obstacles, and tend to choose the shorter detour path. and These are the weight coefficients. Since the Actor1 network focuses on "shortest obstacle avoidance distance," the weights in the Actor1 network are... Less than ,For example It can be taken as 0.1~0.5. The values ​​can be 1 to 3. Similarly, since the Actor2 network focuses on "smoother motion" during training, the values ​​in the Actor2 network... Greater than .

[0047] The evaluation method for candidate trajectories A1 and A2 by the three Critic networks is to first consider the real-time environmental state of the robotic arm. Candidate trajectories A1 and A2 are simultaneously input into three independent Critic networks, and the comprehensive value of each candidate trajectory is then calculated. Finally, the optimal trajectory A is selected through comprehensive value comparison. Here, i is the Actor network ordinal number, i=1,2.

[0048] Comprehensive value The calculation steps are as follows: First, based on the trainable parameters of each Critic network The candidate trajectories A1 and A2 are evaluated for value, and six raw Q values ​​are output: , , , , , Where j is the Critic network ordinal number, j=1,2,3. The trainable parameters of the Critic network. It refers to the weights w and biases b of all neurons in the network, which are a set of multidimensional vectors.

[0049] Subsequently, the combined value of candidate trajectories A1 and A2 is calculated based on the weighted fusion rules: (2) Where i=1,2 corresponds to A1 and A2. The above weighted fusion rule calculation formula can alleviate the estimation bias of Q-value in traditional reinforcement learning.

[0050] The optimal trajectory A can be obtained by using an action screener, which employs the following formula: (3) In the formula, , These are the policy functions for Actor1 and Actor2, respectively. , These are the network parameters for Actor1 and Actor2, respectively. The optimal action is selected as the core aggregation prototype to ensure trajectory robustness.

[0051] S3: Standardize the optimal trajectory A and calculate the total control torque. .

[0052] The standardization process refers to generating a model that includes the desired joint angles. Desired joint angular velocity Desired joint angular acceleration The trajectory control commands are transmitted to the trajectory tracking control layer.

[0053] Total control torque The calculation method includes the following steps: S31: Calculate the feedforward compensation torque based on the optimal trajectory motion command information. This counteracts the dynamic interference of the robotic arm's inertial force, gravity, and centrifugal force.

[0054] Feedforward compensation torque The calculation formula is: (4) Where M is the inertia matrix, N is the Coriolis force and centrifugal force matrix, and G is the gravity vector.

[0055] S32: Design the fitness function J to balance the parameters to be optimized in the PID controller.

[0056] The formula for the fitness function J is: (5) (6) in, To control energy consumption, The settling time required for the system to stabilize, i.e., the time it takes for the error to enter the steady-state band. , , , These are weighting coefficients used to balance performance metrics across multiple dimensions; This is the proportionality coefficient. The integral coefficient is... is the differential coefficient.

[0057] S33: Calculate the feedback control quantity based on the optimized PID parameters in step S32. .

[0058] Feedback control quantity The calculation formula is: (7) S34: Feedforward compensation torque With feedback control quantity Superimposed to form the total control torque That is, total control torque The following formula is used for calculation: (8) Finally, the total control torque will be... The output is sent to the robotic arm actuator, and each joint follows the trajectory command A and control torque. sports.

[0059] S4: Feed back the new joint state after the robotic arm moves to the control strategy optimization layer, and continue to calculate the next trajectory action command until the robotic arm reaches the target position.

[0060] That is, after the robotic arm executes the motion command, the sensors collect the actual joint status in real time. This information is fed back to the control strategy optimization layer for updating the environment state in the next time step.

[0061] In a further design, the present invention also establishes a training data temporary storage area in the control strategy optimization layer. When the system is training offline, the training data temporary storage area is activated to store and update the experience and parameters obtained from reinforcement learning training.

[0062] The working process of the training data temporary storage area is as follows: After executing the optimal trajectory action A, the environment is fed back with a new state. Task reward value Rv forms interaction samples ( A, Rv, The data is stored in the training data temporary storage area. To ensure the diversity of training samples, the training is set to contain 200 time steps per round, for a total of 180 independent training rounds.

[0063] A small batch of samples (dr) is randomly sampled from the training data temporary storage area, and the parameters are updated using a cross-update function: first, the Critic network parameters are optimized based on the actions and states of the Actor1 network, and then the Actor2 network parameters are adjusted based on the updated Critic network feedback; then the process is reversed, first optimizing the Critic network parameters based on the actions and states of the Actor2 network, and then adjusting the Actor1 network parameters. Specifically, this means using the samples from this batch of samples corresponding to Actor1 (… The TD loss is calculated using the reward value Rv (A1), and backpropagation is used to update the Critic network parameters to make the evaluation criteria more accurate. Next, the updated Critic network parameters are fixed, and the Actor2 network parameters are updated using its output value and action gradient signals. This process is then repeated in reverse, using the same batch of sampled samples, and taking the corresponding (A1, Rv) values ​​from the Actor2 network. (A2), recalculate the loss and continue iterating the Critic network parameters to allow the evaluation criteria to adapt to both action strategies simultaneously. Fix the latest Critic network parameters and update the Actor1 network parameters with the help of its gradient feedback.

[0064] Based on the above-described robotic arm trajectory tracking control method using a dual-layer controller, the system of this invention includes a control strategy optimization layer and a trajectory tracking control layer (e.g., ...). Fig. 2 As shown, the control strategy optimization layer includes multiple Actor networks for generating candidate actions, multiple Critic networks for value evaluation, and a trajectory instruction standardization output unit; the trajectory tracking control layer includes a total control torque calculation and instruction execution and real-time status feedback unit.

[0065] The control strategy optimization layer mainly performs the following five tasks: generating candidate trajectory actions in parallel using two actors and quantifying the trajectory value through a three-critie network; optimizing action selection; storing and updating experience; and outputting standardized trajectory commands. Based on the real-time environmental state of the robotic arm, iterative optimization through reinforcement learning strategies generates optimal robust trajectory commands, providing high-quality control commands for lower-level tracking.

[0066] The trajectory tracking control layer mainly performs the following tasks: calculating the feedforward compensation torque of the robotic arm using an inverse kinematics model; optimizing the PID feedback control quantity based on the current environment to obtain the feedback control quantity; fusing the feedforward and feedback control quantities and providing real-time feedback on the system status, thereby suppressing external dynamic disturbances and response lag issues.

[0067] In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0068] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

[0069] Symbol explanation: Real-time environmental status of the robotic arm.

[0070] , the joint status of the robotic arm.

[0071] E, External environment status.

[0072] The target position of the robotic arm movement.

[0073] , Joint angle error.

[0074] , Joint angular velocity error.

[0075] , actual joint angle.

[0076] , desired joint angle.

[0077] , desired joint angular velocity.

[0078] , actual joint angular velocity.

[0079] External contact force.

[0080] Dynamic disturbances in the system.

[0081] Sensor noise.

[0082] , expected joint acceleration.

[0083] T represents the target pose of the robotic arm.

[0084] The reward for the robotic arm reaching the target position.

[0085] The penalty that will be given for each step of execution.

[0086] Collision penalty.

[0087] and The weighting coefficients in the reward function.

[0088] The comprehensive value corresponding to the i-th Actor network.

[0089] , where is all the trainable parameters of the j-th Critic network.

[0090] The Q-value of the Actor1 network was evaluated using the Critic1 network.

[0091] The Q-value of the Actor2 network was evaluated using the Critic1 network.

[0092] The Q-value of the Actor1 network was evaluated using the Critic2 network.

[0093] The Q-value of the Actor2 network was evaluated using the Critic2 network.

[0094] The Q-value of the Actor1 network was evaluated using the Critic3 network.

[0095] The Q-value of the Actor2 network was evaluated using the Critic3 network.

[0096] The strategy function of Actor1.

[0097] The strategy function of Actor2.

[0098] The network parameters of Actor1.

[0099] The network parameters of Actor2.

[0100] A1 and A2, candidate trajectories.

[0101] A, optimal trajectory action.

[0102] Total control torque.

[0103] Feedforward compensation torque.

[0104] J, fitness function.

[0105] Feedback control quantity.

[0106] M is the inertia matrix.

[0107] N represents the matrix of Coriolis force and centrifugal force.

[0108] G is the gravity vector.

[0109] Control energy consumption.

[0110] The adjustment time required for system stability.

[0111] , , , The weight coefficients in the fitness function J formula.

[0112] , proportionality coefficient.

[0113] , Integral coefficient.

[0114] Differential coefficients.

Claims

1. A robotic arm trajectory tracking control method based on a two-layer controller, characterized in that, Includes the following steps: S1: Construct a control strategy optimization layer and a trajectory tracking control layer. In the control strategy optimization layer, construct multiple Actor networks and multiple Critic networks. S2: Real-time environmental status of the robotic arm Each Actor network generates multiple different trajectory action information. Multiple Critic networks are used to evaluate the value of multiple trajectory action information, and the optimal trajectory action A is selected. The optimal trajectory action command is then passed to the trajectory tracking control layer. S3: The trajectory tracking control layer calculates the total control torque based on the optimal trajectory action command. And drive the robotic arm to move according to the calculation results; S4: Feed back the new joint state after the robotic arm moves to the control strategy optimization layer, and continue to calculate the next trajectory action command until the robotic arm reaches the target position.

2. The robotic arm trajectory tracking control method based on a dual-layer controller according to claim 1, characterized in that: The real-time environmental status of the robotic arm Including the joint status of the robotic arm External environmental condition E, target position of robotic arm movement ; The joint state of the robotic arm Including actual joint angles Actual joint angular velocity Joint angle error and joint angular velocity error ,in, , , For the desired joint angle; The desired joint angular velocity; The external environmental state E includes external contact force. System dynamic disturbances and sensor noise ; The target position of the robotic arm movement Including expected joint angles Desired joint angular velocity Desired joint acceleration And the target pose T of the robotic arm.

3. The robotic arm trajectory tracking control method based on a dual-layer controller according to claim 1, characterized in that: The Actor network includes Actor1 network and Actor2 network. The two Actor networks extract candidate trajectories A1 and A2 through a reward function, which is: ,in, A reward for the robotic arm reaching the target position. The penalty to be given for each step of execution. As a penalty for collision, and These are the weight coefficients in the Actor1 network. Less than In the Actor2 network Greater than .

4. The robotic arm trajectory tracking control method based on a dual-layer controller according to claim 3, characterized in that: The Critic network includes Critic1, Critic2, and Critic3. Each Critic network calculates the combined value of A1 and A2 based on a weighted fusion rule. The optimal trajectory action is selected through a comprehensive value screening rule, where i is the Actor network ordinal number, i=1,2.

5. The robotic arm trajectory tracking control method based on a dual-layer controller according to claim 4, characterized in that, The overall value The calculation method is as follows: Based on the trainable parameters of each Critic network The candidate trajectories A1 and A2 are evaluated for value, and six raw Q values ​​are output: , , , , , Where j is the Critic network ordinal number, j=1, 2, 3; Calculate the overall value: ； Filtering the optimal trajectory action: ,in, , These are the policy functions for Actor1 and Actor2, respectively. , These are the network parameters for Actor1 and Actor2, respectively.

6. The robotic arm trajectory tracking control method based on a dual-layer controller according to claim 1, characterized in that, The total control torque The calculation method includes the following steps: S31: Calculate the feedforward compensation torque based on the optimal trajectory motion command information. This counteracts the dynamic interference of the robotic arm's inertial force, gravity, and centrifugal force. S32: Design the fitness function J to balance the parameters to be optimized in the PID controller; S33: Calculate the feedback control quantity based on the optimized PID parameters in step S32. ; S34: Feedforward compensation torque With feedback control quantity Superimposed to form the total control torque .

7. The robotic arm trajectory tracking control method based on a dual-layer controller according to claim 6, characterized in that: The feedforward compensation torque The calculation formula is: ； Where M is the inertia matrix, N is the Coriolis force and centrifugal force matrix, and G is the gravity vector.

8. The robotic arm trajectory tracking control method based on a dual-layer controller according to claim 6, characterized in that: The formula for the fitness function J is: ， ， in, To control energy consumption, Adjustment time required for system stability , , , These are the weighting coefficients; This is the proportionality coefficient. The integral coefficient is... These are the differential coefficients; Feedback control quantity The calculation formula is: 。 9. The robotic arm trajectory tracking control method based on a dual-layer controller according to claim 1, characterized in that: The control strategy optimization layer also establishes a training data temporary storage area, and the working process of the training data temporary storage area is as follows: After executing the optimal trajectory action A, the environment is fed back with a new state. Task reward value Rv forms interaction samples ( A, Rv, Store the data in the training data temporary storage area. Randomly sample a small batch of samples dr from the training data temporary storage area, and update the parameters using a cross-update function: first optimize the parameters of the Critic network based on the actions and states of the Actor1 network, and then adjust the parameters of the Actor2 network based on the updated Critic network feedback; then reverse the process, first optimizing the parameters of the Critic network based on the actions and states of the Actor2 network, and then adjusting the parameters of the Actor1 network.

10. A system, characterized in that, The system operates using the robotic arm trajectory tracking control method based on a dual-layer controller as described in any one of claims 1-9. The system includes a control strategy optimization layer and a trajectory tracking control layer. The control strategy optimization layer includes multiple Actor networks for generating candidate actions, multiple Critic networks for value evaluation, and a trajectory instruction standardization output unit. The trajectory tracking control layer includes a total control torque calculation and command execution and real-time status feedback unit.

Images