Model-free control method and system for dual-link robotic arm under cloud-edge collaboration

Abstract

The present invention discloses a model-free control method and system for a double-link robotic arm under cloud-edge-end collaboration; the terminal collects real-time data of the double-link robotic arm and sends it to the edge end; the edge end pre-processes the data and inputs it into a multivariable model-free controller to calculate control instructions; the terminal controls the movement of the double-link robotic arm according to the control instructions, collects real-time data of the double-link robotic arm and sends it to the edge end; the cloud end optimizes and updates the parameters of the multivariable model-free controller according to the real-time data fed back by the edge end until the control task is completed; the control method and system provided by the present invention are not only suitable for double-link robotic arms, reduce their modeling difficulty, improve control accuracy, and enhance computing efficiency, but also have the prospect of being extended to other multi-input and multi-output systems.

Description

Technical Field

[0001] The present invention belongs to the field of robotic arm control, and in particular relates to a model-free control method and system for a double-link robotic arm under cloud-edge collaboration. Background Art

[0002] With the advancement of science and technology and the accelerated development of industrial automation, robotic arms have become an indispensable part of the automation field, undertaking various important tasks such as assembly, handling, and welding. The two-link robotic arm has been widely used in industrial production, medical rehabilitation, and service robots. However, the strong time-varying, strong nonlinearity, and strong coupling brought by its double-joint structure have brought great challenges to the research of its decoupling control method. The control method of the two-link robotic arm is divided into model-based control and data-driven control. The model-based control method requires the establishment of an accurate dynamic model. However, the complex internal factors and external environment make modeling difficult, which affects its in-depth research and application. Data-driven control directly uses rich input and output data to directly design the controller, which has received widespread attention from scholars.

[0003] In 2013, Hou Zhongsheng and Zhu Yuanming first proposed a direct single-variable model-free adaptive control method for single-input single-output systems in their paper "Controller-dynamic-linearization-basedmodel free adaptive control for discrete-time nonlinear systems." IEEE Transactions on Industrial Informatics, 2013, 9(4): 2301-2309. After 7 years, Yu Xian from Beijing Jiaotong University developed a direct multivariable control method for multi-input multi-output systems in his doctoral dissertation "Data-driven iterative learning control and application based on controller dynamic linearization [D]", Beijing Jiaotong University, 2020. It has now become a typical data-driven control method, attracting scholars to explore its potential in combination with neural networks to improve the control performance of multi-input multi-output controlled systems. With the development of cloud computing, edge computing, and direct multivariable model-free adaptive control methods, new ideas have been provided for the control of dual-link robotic arms; the powerful computing and storage capabilities of cloud computing platforms support complex model training and optimization, and edge computing devices realize real-time data processing and rapid response to control instructions. The direct multivariable model-free adaptive control method combined with a network with stronger learning ability can further improve the performance of model-free adaptive control. Therefore, in order to improve the control performance of the dual-link robotic arm and combine the advantages of the cloud-edge end, the present invention deeply integrates direct multivariable model-free adaptive control and reinforcement learning, and proposes a dual-link robotic arm model-free control method and system based on cloud-edge-end collaboration with stronger learning ability. Summary of the invention

[0004] In order to solve the problems existing in the background technology, the purpose of the present invention is to provide a model-free control method for a double-link robotic arm in collaboration with a cloud-edge end, wherein the cloud-edge end includes a cloud end, an edge end, and a terminal, and includes the following steps:

[0005] Step (1): The terminal collects real-time data of the double-link robot arm, wherein the real-time data includes the real-time expected joint angle of the first link, the real-time operating joint angle of the first link, the real-time expected joint angle of the second link, and the real-time operating joint angle of the second link, and sends the real-time data to the edge end;

[0006] Step (2): the edge end preprocesses the real-time data, inputs the preprocessed data into a multivariable decoupled model-free controller based on an online Actor-Critic reinforcement learning network for adaptive learning, calculates control instructions for the double-link robotic arm, the control instructions include control instructions for the first link and control instructions for the second link, sends the control instructions to the terminal, and the terminal controls the movement of the double-link robotic arm according to the control instructions for the first link and the control instructions for the second link; at the same time, the edge end sends the preprocessed data to the cloud;

[0007] Step (3): The cloud optimizes and updates the parameters of the multivariable decoupling model-free controller according to the data fed back by the edge end, and transmits the optimized and updated parameters of the multivariable decoupling model-free controller to the multivariable decoupling model-free controller at the edge end;

[0008] Repeat the above steps (1) to (3) until the control task is completed.

[0009] According to the model-free control method of a double-link robotic arm in cloud-edge collaboration provided by the present invention, in step (2), the edge end preprocesses the real-time data, inputs the preprocessed data into a multivariable decoupled model-free controller based on an online Actor-Critic reinforcement learning network for adaptive learning, and calculates control instructions for the double-link robotic arm, wherein the control instructions include control instructions for the first link and control instructions for the second link, and includes the following steps:

[0010] Step (2.1): k At the sampling time, the mathematical formula of the multivariable decoupling model-free controller is:

[0011] ;

[0012] in, k is the sampling moment, the double-link robotic arm ; for k The control command of the first link of the double-link robot at the sampling time, for k Control instructions for the second link of the double-link robot arm at the sampling time; for k The sampling time s +2 time-varying block matrices, For the said s +2 time-varying block matrices x 1st row x 2 columns of elements, , ; said s+2 main diagonal elements of the time-varying block matrix express k Sampling time double link manipulator x 1. The control command of the connecting rod k Sampling time double link manipulator x 1. The mapping relationship between the trajectory tracking error of the connecting rod; s +2 off-diagonal elements of the time-varying block matrix express k Sampling time double link manipulator x 1. The control command of the connecting rod k Sampling time double link manipulator x The mapping relationship between the trajectory tracking errors of the 2-link, ; , , ; L is the controller pseudo order, L is a positive integer; for k Sampling time double link manipulator The trajectory tracking error of the connecting rod, , for k Sampling time double link manipulator The real-time desired joint angle of the link, for k Sampling time double link manipulator The real-time operating joint angle of the connecting rod; for k Sampling time double link manipulator Tracking error of connecting rod The first-order backward difference of ;

[0013] Step (2.2): Construct an adaptive learning mechanism based on an online Actor-Critic reinforcement learning network;

[0014] The online Actor-Critic reinforcement learning network contains Online Actor-Critic reinforcement learning subnetwork, two-link robotic arm ;No. The input of the online Actor-Critic reinforcement learning subnetwork contains k Sampling time double link manipulator Tracking error of connecting rod , k Sampling time double link manipulator First-order backward difference of link trajectory tracking error , k Sampling time double link manipulator Any one or any combination of the high-order backward differences of the link trajectory tracking error will be The value of traverses the positive integer interval All values ​​in All inputs of the online Actor-Critic reinforcement learning subnetwork; the The output of the online Actor-Critic reinforcement learning subnetwork includes an Actor output part and a Critic output part, wherein the Actor output part includes a time-varying block matrix No. The Critic output part contains the value function; The value of traverses the positive integer interval All values ​​in An online Actor-Critic reinforcement learning subnetwork.

[0015] According to the model-free control method of a dual-link robotic arm in cloud-edge collaboration provided by the present invention, in step (3), the cloud optimizes and updates the parameters of the multivariable decoupling model-free controller based on the data fed back by the edge, and the optimization and updating process includes the following steps:

[0016] Step (3.1): Construct a real-time dynamic linearization model of a two-link robotic arm with coupling characteristics:

[0017] ;

[0018] in, k is the sampling moment, the double-link robotic arm ; for The real-time operating joint angle of the first link of the two-link robot at the sampling time, for The real-time operating joint angle of the second link of the two-link robot arm at the sampling moment; for The control command of the first link of the double-link robot at the sampling time, for Control instructions for the second link of the double-link robot arm at the sampling time; , , ;definition , for A Jacobian matrix of the two-link manipulator at the sampling time, wherein the non-diagonal elements of the Jacobian matrix represent the coupling characteristics between different links; is the Jacobian matrix No. x 3rd row x 4 columns of elements, , , express and The mapping relationship between them;

[0019] Jacobian matrix of the two-link manipulator The iterative learning law is:

[0020] ;

[0021] in, is the step size factor, is the penalty factor; , ; represents the two-norm;

[0022] Step (3.2): In the optimization update process, a one-step forward error is used; based on the real-time dynamic linearization model of the two-link manipulator with coupling characteristics described in step (3.1), the mathematical formula for calculating the one-step forward error is:

[0023] ;

[0024] Among them, the double-link robotic arm ; for k The trajectory tracking error of the first link of the double-link manipulator at the +1 sampling time, for k +1 The trajectory tracking error of the second link of the double-link manipulator at sampling time; for k +1 The expected joint angle of the first link of the two-link robot at sampling time, for k +1 The expected joint angle of the second link of the two-link robot at sampling time;

[0025] Step (3.3): In the optimization update process, the Jacobian matrix of the two-link manipulator is used; based on the real-time dynamic linearization model of the two-link manipulator with coupling characteristics described in step (3.1), the mathematical formula for calculating the partial derivative matrix of the two-link manipulator is:

[0026] ;

[0027] in, for against The derivative of , ;

[0028] Step (3.4): In the optimization and updating process, the dual-link robot arm is used Connecting rod control instructions ,in , respectively for the time-varying block matrix The partial derivative matrix of all elements in L is the controller pseudo order, L is a positive integer, and the mathematical formula for calculating the partial derivative matrix is:

[0029] ;

[0030] Step (3.5): Define the time difference function ,in is the value function, is the discount factor; to minimize the system performance index function As the goal, the gradient descent method is used to optimize and update the (Dual-link robotic arm ) The weights of the online Actor-Critic reinforcement learning subnetwork, which include Actor network weights and Critic network weights , h is the number of hidden layer nodes of the reinforcement learning network, The value of traverses the positive integer interval All values ​​in The weights of an online Actor-Critic reinforcement learning subnetwork;

[0031] Optimization Update The sampling time Actor network weights for online Actor-Critic reinforcement learning subnetworks :

[0032] If the controller is pseudo-order ,but

[0033] ;

[0034] If the controller is pseudo-order ,but

[0035] ;

[0036] in, For the The learning rate of the Actor network, , ; for k Sampling time double link manipulator The trajectory tracking error of the connecting rod, The mathematical formula for the one-step forward error described in step (3.2) is used for calculation, ; is a value with a mean of 0 and a variance of The normal distribution function of For the The hidden layer of the online Actor-Critic reinforcement learning subnetwork i The output of each node;

[0037] Optimization Update The sampling time Critic network weights of online Actor-Critic reinforcement learning subnetworks :

[0038] ;

[0039] in, For the The learning rate of the Critic network;

[0040] Step (3.6): Optimize updates The time-varying block matrix of the multivariable decoupled model-free controller at the sampling time:

[0041] If the controller is pseudo-order ,but

[0042] ;

[0043] Will The value of traverses the positive integer interval All values ​​within, obtain the time-varying block matrix;

[0044] If the controller is pseudo-order ,but

[0045] ;

[0046] in, ,Will The value of traverses the positive integer interval All values ​​within , get the time-varying block matrix.

[0047] According to the model-free control method of a dual-link robotic arm under cloud-edge collaboration provided by the present invention, the Actor-Critic reinforcement learning network adopts a radial basis function network; the radial basis function network adopts a structure with a single hidden layer, that is, a three-layer network structure consisting of an input layer, a single hidden layer, and an output layer.

[0048] According to the model-free control method of a double-link robotic arm in cloud-edge collaboration provided by the present invention, the edge end in step (2) preprocesses the real-time data, including filtering and denoising the real-time data.

[0049] According to the model-free control method of the double-link robotic arm under cloud-edge-end collaboration provided by the present invention, in step (1), the terminal collects real-time data of the double-link robotic arm through a sensor signal acquisition device.

[0050] The present invention also provides a model-free control system for a double-link robotic arm in cloud-edge collaboration, comprising:

[0051] The terminal includes a double-link mechanical arm running in real time, a data acquisition module, a first data output module, and a first data input module;

[0052] A data acquisition module, used to collect real-time data of the double-link mechanical arm through a sensor signal acquisition device, wherein the real-time data includes a real-time expected joint angle of the first link of the double-link mechanical arm, a real-time operating joint angle of the first link, a real-time expected joint angle of the second link, and a real-time operating joint angle of the second link;

[0053] A first data output module, used to send the real-time data to an edge end;

[0054] A first data input module, used to receive a control instruction issued by the edge end, wherein the control instruction includes a control instruction of the first connecting rod and a control instruction of the second connecting rod;

[0055] The edge end includes a second data input module, a data preprocessing module, a multivariable decoupling model-free controller calculation module, and a second data output module;

[0056] A second data input module, used for receiving real-time data of the double-link robotic arm uploaded by a terminal;

[0057] A data preprocessing module, used for preprocessing the real-time data and then inputting it into a multivariable decoupling model-free controller calculation module, and sending it to the cloud;

[0058] A multivariable decoupled model-free controller calculation module; used for calculating the control instructions of the double-link manipulator through a multivariable decoupled model-free controller that performs adaptive learning based on an online Actor-Critic reinforcement learning network, wherein the control instructions include the control instructions of the first link and the control instructions of the second link;

[0059] A second data output module, used for sending the control instruction to the terminal;

[0060] The cloud comprises a third data input module, a data storage module, a multivariable decoupling model-free controller optimization and updating module, and a third data output module;

[0061] The third data input module is used to receive the pre-processed real-time data uploaded by the edge end;

[0062] A data storage module, used to store the pre-processed real-time data uploaded by the edge end;

[0063] A multivariable decoupling model-free controller optimization and update module is used to optimize and update the parameters of the multivariable decoupling model-free controller;

[0064] The third data output module is used to transmit the optimized and updated multivariable decoupled model-free controller parameters to the edge end.

[0065] Furthermore, the present invention adopts the following technical solutions:

[0066] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the model-free control method of a double-link robotic arm in cloud-edge collaboration as described above.

[0067] Furthermore, the present invention adopts the following technical solutions:

[0068] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the model-free control method of a double-link robotic arm under cloud-edge collaboration as described above is implemented.

[0069] The beneficial technical effects of the present invention are:

[0070] For multi-input and multi-output controlled systems, existing research has not yet involved combining direct multivariable model-free adaptive control with reinforcement learning networks, and combining the advantages of cloud-edge terminals to solve the control algorithm calculation and control command rapid response means. The model-free control method and system of a double-link robotic arm under cloud-edge-terminal collaboration provided by the present invention at least include the following beneficial technical effects:

[0071] (1) Reduce the difficulty of modeling: The present invention adopts a data-driven control method, which does not require the establishment of an accurate dynamic model, thus reducing the difficulty of modeling. This method has more obvious advantages when dealing with multivariable systems with strong time-varying, strong nonlinearity, and strong coupling, and is easier to apply in practice;

[0072] (2) Improve control accuracy: By deeply integrating multivariable model-free adaptive control and reinforcement learning, this method can achieve stronger learning ability, thereby improving the control accuracy of the dual-link robotic arm and meeting higher precision application requirements;

[0073] (3) Improve computing efficiency: Use the powerful computing power of the cloud computing platform for model training and data analysis, and use edge computing devices to achieve real-time data processing and rapid response to control instructions, effectively improving computing efficiency;

[0074] (4) Broadening the scope of application: The control method and system proposed in the present invention are not only applicable to dual-link robotic arms, but can also be extended to other multi-input and multi-output control systems, and have broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 It is a structural schematic diagram of a double-link robotic arm;

[0076] Figure 2 A schematic flow chart of a model-free control method for a double-link robotic arm in cloud-edge collaboration provided by the present invention;

[0077] Figure 3 A schematic diagram of the connection modules of the model-free control system of the double-link robotic arm in cloud-edge collaboration provided by the present invention. DETAILED DESCRIPTION

[0078] The present invention discloses a model-free control method and system for a double-link robotic arm under cloud-edge collaboration; the terminal collects real-time data of the double-link robotic arm and sends it to the edge; the edge preprocesses the data and inputs it into a multivariable model-free controller to calculate control instructions; the terminal controls the movement of the double-link robotic arm according to the control instructions, collects real-time data of the double-link robotic arm and sends it to the edge; the cloud optimizes and updates the parameters of the multivariable model-free controller according to the real-time data fed back by the edge until the control task is completed. The control method and system provided by the present invention are not only applicable to double-link robotic arms, reduce their modeling difficulty, improve control accuracy, and enhance calculation efficiency, but also have the application prospect of being extended to other multi-input and multi-output systems.

[0079] The following is a further clear and complete description of the model-free control method and system of the double-link robotic arm under cloud-edge-end collaboration provided by the present invention in conjunction with the accompanying drawings:

[0080] Figure 1The structural diagram of the double-link robotic arm is given. Figure 2 A flow chart of a model-free control method for a double-link robotic arm in cloud-edge-end collaboration provided by the present invention is given; the present invention provides a model-free control method for a double-link robotic arm in cloud-edge-end collaboration, wherein the cloud-edge-end includes a cloud, an edge, and a terminal, and includes the following steps:

[0081] Step (1): the terminal collects real-time data of the double-link robot arm through a sensor signal acquisition device, wherein the real-time data includes a real-time expected joint angle of the first link, a real-time operating joint angle of the first link, a real-time expected joint angle of the second link, and a real-time operating joint angle of the second link, and sends the real-time data to the edge end;

[0082] Step (2): the edge end preprocesses the real-time data, inputs the preprocessed data into a multivariable decoupled model-free controller based on an online Actor-Critic reinforcement learning network for adaptive learning, calculates control instructions for the double-link robotic arm, the control instructions include control instructions for the first link and control instructions for the second link, sends the control instructions to the terminal, and the terminal controls the movement of the double-link robotic arm according to the control instructions for the first link and the control instructions for the second link; at the same time, the edge end sends the preprocessed data to the cloud;

[0083] Step (3): The cloud optimizes and updates the parameters of the multivariable decoupling model-free controller according to the data fed back by the edge end, and transmits the optimized and updated parameters of the multivariable decoupling model-free controller to the multivariable decoupling model-free controller at the edge end;

[0084] Repeat the above steps (1) to (3) until the control task is completed.

[0085] According to the model-free control method of a double-link robotic arm in cloud-edge collaboration provided by the present invention, in step (2), the edge end preprocesses the real-time data, inputs the preprocessed data into a multivariable decoupled model-free controller based on an online Actor-Critic reinforcement learning network for adaptive learning, and calculates control instructions for the double-link robotic arm, wherein the control instructions include control instructions for the first link and control instructions for the second link, and includes the following steps:

[0086] Step (2.1): k At the sampling time, the mathematical formula of the multivariable decoupling model-free controller is:

[0087] ;

[0088] in, k is the sampling moment, the double-link robotic arm ; for k The control command of the first link of the double-link robot at the sampling time, for k Control instructions for the second link of the double-link robot arm at the sampling time; for k The sampling time s +2 time-varying block matrices, For the said s +2 time-varying block matrices x 1st row x 2 columns of elements, , ; said s +2 main diagonal elements of the time-varying block matrix express k Sampling time double link manipulator x 1. The control command of the connecting rod k Sampling time double link manipulator x 1. The mapping relationship between the trajectory tracking error of the connecting rod; s +2 off-diagonal elements of the time-varying block matrix express k Sampling time double link manipulator x 1. The control command of the connecting rod k Sampling time double link manipulator x The mapping relationship between the trajectory tracking errors of the 2-link, ; , , ; L is the controller pseudo order, L is a positive integer; for k Sampling time double link manipulator The trajectory tracking error of the connecting rod, , for k Sampling time double link manipulator The real-time desired joint angle of the link, for k Sampling time double link manipulator The real-time operating joint angle of the connecting rod; for k Sampling time double link manipulator Tracking error of connecting rod The first-order backward difference of ;

[0089] Step (2.2): Construct an adaptive learning mechanism based on an online Actor-Critic reinforcement learning network;

[0090] The online Actor-Critic reinforcement learning network contains Online Actor-Critic reinforcement learning subnetwork, two-link robotic arm ;No. The input of the online Actor-Critic reinforcement learning subnetwork contains k Sampling time double link manipulator Tracking error of connecting rod , k Sampling time double link manipulator First-order backward difference of link trajectory tracking error , k Sampling time double link manipulator Any one or any combination of the high-order backward differences of the link trajectory tracking error will be The value of traverses the positive integer interval All values ​​within the All inputs of the online Actor-Critic reinforcement learning subnetwork; the The output of the online Actor-Critic reinforcement learning subnetwork includes an Actor output part and a Critic output part, wherein the Actor output part includes a time-varying block matrix No. The Critic output part contains the value function; The value of traverses the positive integer interval All values ​​in An online Actor-Critic reinforcement learning subnetwork.

[0091] According to the model-free control method of a dual-link robotic arm in cloud-edge collaboration provided by the present invention, in step (3), the cloud optimizes and updates the parameters of the multivariable decoupling model-free controller based on the data fed back by the edge, and the optimization and updating process includes the following steps:

[0092] Step (3.1): Construct a real-time dynamic linearization model of a two-link robotic arm with coupling characteristics:

[0093] ;

[0094] in, k is the sampling moment, the double-link robotic arm ; for The real-time operating joint angle of the first link of the two-link robot at the sampling time, for The real-time operating joint angle of the second link of the two-link robot arm at the sampling moment; for The control command of the first link of the double-link robot at the sampling time, for Control instructions for the second link of the double-link robot arm at the sampling time; , , ;definition , for A Jacobian matrix of the two-link manipulator at the sampling time, wherein the non-diagonal elements of the Jacobian matrix represent the coupling characteristics between different links; is the Jacobian matrix No. x 3rd row x 4 columns of elements, , , express and The mapping relationship between them;

[0095] The Jacobian matrix of the two-link manipulator The iterative learning law is:

[0096] ;

[0097] in, is the step size factor, is the penalty factor; , ; represents the two-norm;

[0098] Step (3.2): In the optimization update process, a one-step forward error is used; based on the real-time dynamic linearization model of the two-link manipulator with coupling characteristics described in step (3.1), the mathematical formula for calculating the one-step forward error is:

[0099] ;

[0100] Among them, the double-link robotic arm ; for k The trajectory tracking error of the first link of the double-link manipulator at the +1 sampling time, for k +1 The trajectory tracking error of the second link of the double-link manipulator at sampling time; for k+1 The expected joint angle of the first link of the two-link robot at sampling time, for k +1 The expected joint angle of the second link of the two-link robot at sampling time;

[0101] Step (3.3): In the optimization update process, the Jacobian matrix of the two-link manipulator is used; based on the real-time dynamic linearization model of the two-link manipulator with coupling characteristics described in step (3.1), the mathematical formula for calculating the partial derivative matrix of the two-link manipulator is:

[0102] ;

[0103] in, for against The derivative of , ;

[0104] Step (3.4): In the optimization and updating process, the dual-link robot arm is used Connecting rod control instructions ,in , respectively for the time-varying block matrix The partial derivative matrix of all elements in L is the controller pseudo order, L is a positive integer, and the mathematical formula for calculating the partial derivative matrix is:

[0105] ;

[0106] Step (3.5): Define the time difference function ,in is the value function, is the discount factor; to minimize the system performance index function As the goal, the gradient descent method is used to optimize and update the (Dual-link robotic arm ) The weights of the online Actor-Critic reinforcement learning subnetwork, which include Actor network weights and Critic network weights , h is the number of hidden layer nodes of the reinforcement learning network, The value of traverses the positive integer interval All values ​​in The weights of an online Actor-Critic reinforcement learning subnetwork;

[0107] Optimization Update The sampling time Actor network weights for online Actor-Critic reinforcement learning subnetworks :

[0108] If the controller is pseudo-order ,but

[0109] ;

[0110] If the controller is pseudo-order ,but

[0111] ;

[0112] in, For the said The learning rate of the Actor network, , ; for k Sampling time double link manipulator The trajectory tracking error of the connecting rod, The mathematical formula for the one-step forward error described in step (3.2) is used for calculation, ; is a value with a mean of 0 and a variance of The normal distribution function of For the The hidden layer of the online Actor-Critic reinforcement learning subnetwork i The output of each node;

[0113] Optimization Update The sampling time Critic network weights of online Actor-Critic reinforcement learning subnetworks :

[0114] ;

[0115] in, For the said The learning rate of the Critic network;

[0116] Step (3.6): Optimize updates The time-varying block matrix of the multivariable decoupled model-free controller at the sampling time:

[0117] If the controller is pseudo-order ,but

[0118] ;

[0119] Will The value of traverses the positive integer interval All values ​​within, obtain the time-varying block matrix;

[0120] If the controller is pseudo-order ,but

[0121] ;

[0122] in, ,Will The value of traverses the positive integer interval All values ​​within , get the time-varying block matrix.

[0123] According to the model-free control method of a dual-link robotic arm under cloud-edge collaboration provided by the present invention, the Actor-Critic reinforcement learning network adopts a radial basis function network; the radial basis function network adopts a structure with a single hidden layer, that is, a three-layer network structure consisting of an input layer, a single hidden layer, and an output layer.

[0124] According to the model-free control method of a double-link robotic arm in cloud-edge collaboration provided by the present invention, the edge end in step (2) preprocesses the real-time data, including filtering and denoising the real-time data.

[0125] Figure 3 A schematic diagram of the connection modules of the model-free control system of the double-link robotic arm in cloud-edge collaboration provided by the present invention is given. The present invention also provides a model-free control system of the double-link robotic arm in cloud-edge collaboration, including:

[0126] The terminal includes a double-link mechanical arm running in real time, a data acquisition module, a first data output module, and a first data input module;

[0127] A data acquisition module, used to collect real-time data of the double-link mechanical arm through a sensor signal acquisition device, wherein the real-time data includes a real-time expected joint angle of the first link of the double-link mechanical arm, a real-time operating joint angle of the first link, a real-time expected joint angle of the second link, and a real-time operating joint angle of the second link;

[0128] A first data output module, used to send the real-time data to an edge end;

[0129] A first data input module, used to receive a control instruction issued by the edge end, wherein the control instruction includes a control instruction of the first connecting rod and a control instruction of the second connecting rod;

[0130] The edge end includes a second data input module, a data preprocessing module, a multivariable decoupling model-free controller calculation module, and a second data output module;

[0131] A second data input module, used for receiving real-time data of the double-link robotic arm uploaded by a terminal;

[0132] A data preprocessing module, used for preprocessing the real-time data and then inputting it into a multivariable decoupling model-free controller calculation module, and sending it to the cloud;

[0133] A multivariable decoupled model-free controller calculation module; used for calculating the control instructions of the double-link manipulator through a multivariable decoupled model-free controller that performs adaptive learning based on an online Actor-Critic reinforcement learning network, wherein the control instructions include the control instructions of the first link and the control instructions of the second link;

[0134] A second data output module, used for sending the control instruction to the terminal;

[0135] The cloud comprises a third data input module, a data storage module, a multivariable decoupling model-free controller optimization and updating module, and a third data output module;

[0136] The third data input module is used to receive the pre-processed real-time data uploaded by the edge end;

[0137] A data storage module, used to store the pre-processed real-time data uploaded by the edge end;

[0138] A multivariable decoupling model-free controller optimization and update module is used to optimize and update the parameters of the multivariable decoupling model-free controller;

[0139] The third data output module is used to transmit the optimized and updated multivariable decoupled model-free controller parameters to the edge end.

[0140] Furthermore, the present invention adopts the following technical solutions:

[0141] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the model-free control method of a double-link robotic arm in cloud-edge collaboration as described above.

[0142] Furthermore, the present invention adopts the following technical solutions:

[0143] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the model-free control method of a double-link robotic arm under cloud-edge collaboration as described above is implemented.

[0144] Through the description of the above embodiments, it can be clearly understood by those skilled in the art that the facilities of the present invention can be implemented by means of software plus the necessary general hardware platform. The embodiments of the present invention can be implemented using an existing processor, or by a dedicated processor used for this purpose or other purposes for an appropriate system, or by a hard-wired system. The embodiments of the present invention also include non-transitory computer-readable storage media, which include machine-readable media for carrying or having machine-executable instructions or data structures stored thereon; such machine-readable media can be any available medium that can be accessed by a general or special-purpose computer or other machine with a processor. For example, such machine-readable media can include RAM, ROM, EPROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store the required program code in the form of machine-executable instructions or data structures, and can be accessed by a general or special-purpose computer or other machine with a processor. When information is transmitted or provided to a machine via a network or other communication connection (hard-wired, or wireless, or a combination of hard-wired and wireless), the connection is also considered a machine-readable medium.

[0145] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A model-free control method for a double-link robotic arm in cloud-edge collaboration, wherein the cloud-edge includes a cloud, an edge, and a terminal, and is characterized in that: The following steps are involved: Step (1): The terminal collects real-time data of the double-link robot arm, wherein the real-time data includes the real-time expected joint angle of the first link, the real-time operating joint angle of the first link, the real-time expected joint angle of the second link, and the real-time operating joint angle of the second link, and sends the real-time data to the edge end; Step (2): the edge end preprocesses the real-time data, inputs the preprocessed data into a multivariable decoupled model-free controller based on an online Actor-Critic reinforcement learning network for adaptive learning, calculates control instructions for the double-link robotic arm, the control instructions include control instructions for the first link and control instructions for the second link, sends the control instructions to the terminal, and the terminal controls the movement of the double-link robotic arm according to the control instructions for the first link and the control instructions for the second link; at the same time, the edge end sends the preprocessed data to the cloud; Step (3): The cloud optimizes and updates the parameters of the multivariable decoupling model-free controller according to the data fed back by the edge end, and transmits the optimized and updated parameters of the multivariable decoupling model-free controller to the multivariable decoupling model-free controller at the edge end; Repeat the above steps (1) to (3) until the control task is completed.

2. The model-free control method of the double-link robotic arm in cloud-edge collaboration according to claim 1 is characterized in that: In step (2), the edge end preprocesses the real-time data, inputs the preprocessed data into a multivariable decoupled model-free controller based on an online Actor-Critic reinforcement learning network for adaptive learning, and calculates control instructions for the dual-link robotic arm, wherein the control instructions include control instructions for the first link and control instructions for the second link, including the following steps: Step (2.1): k At the sampling time, the mathematical formula of the multivariable decoupling model-free controller is: ； in, k is the sampling moment, the double-link robotic arm ; for k The control command of the first link of the double-link robot at the sampling time, for k Control instructions for the second link of the double-link robot arm at the sampling time; for k The sampling time s +2 time-varying block matrices, For the s +2 time-varying block matrices x 1st row x 2 columns of elements, , ; said s +2 main diagonal elements of the time-varying block matrix express k Sampling time double link manipulator x 1. The control command of the connecting rod k Sampling time double link manipulator x 1. The mapping relationship between the trajectory tracking error of the connecting rod; s +2 off-diagonal elements of the time-varying block matrix express k Sampling time double link manipulator x 1. The control command of the connecting rod k Sampling time double link manipulator x The mapping relationship between the trajectory tracking errors of the 2-link, ; , , ; L is the controller pseudo order, L is a positive integer; for k Sampling time double link manipulator The trajectory tracking error of the connecting rod, , for k Sampling time double link manipulator The real-time desired joint angle of the link, for k Sampling time double link manipulator The real-time operating joint angle of the connecting rod; for k Sampling time double link manipulator Tracking error of connecting rod The first-order backward difference of ; Step (2.2): Construct an adaptive learning mechanism based on an online Actor-Critic reinforcement learning network; The online Actor-Critic reinforcement learning network contains Online Actor-Critic reinforcement learning subnetwork, two-link robotic arm ;No. The input of the online Actor-Critic reinforcement learning subnetwork contains k Sampling time double link manipulator Tracking error of connecting rod , k Sampling time double link manipulator First-order backward difference of link trajectory tracking error , k Sampling time double link manipulator Any one or any combination of the high-order backward differences of the link trajectory tracking error will be The value of traverses the positive integer interval All values ​​in All inputs of the online Actor-Critic reinforcement learning subnetwork; The output of the online Actor-Critic reinforcement learning subnetwork includes an Actor output part and a Critic output part, wherein the Actor output part includes a time-varying block matrix No. The Critic output part contains the value function; The value of traverses the positive integer interval All values ​​in An online Actor-Critic reinforcement learning subnetwork.

3. The model-free control method of the double-link robotic arm in cloud-edge collaboration according to claim 2 is characterized in that: In step (3), the cloud optimizes and updates the parameters of the multivariable decoupling model-free controller based on the data fed back by the edge end. The optimization and updating process includes the following steps: Step (3.1): Construct a real-time dynamic linearization model of a two-link robotic arm with coupling characteristics: ； in, k is the sampling moment, the double-link robotic arm ; for The real-time operating joint angle of the first link of the two-link robot at the sampling time, for The real-time operating joint angle of the second link of the two-link robot arm at the sampling moment; for The control command of the first link of the double-link robot at the sampling time, for Control instructions for the second link of the double-link robot arm at the sampling time; , , ;definition , for A Jacobian matrix of the two-link manipulator at the sampling time, wherein the non-diagonal elements of the Jacobian matrix represent the coupling characteristics between different links; is the Jacobian matrix No. x 3rd row x 4 columns of elements, , , express and The mapping relationship between them; The Jacobian matrix of the two-link manipulator The iterative learning law is: ； in, is the step size factor, is the penalty factor; , ; represents the two-norm; Step (3.2): In the optimization update process, a one-step forward error is used; based on the real-time dynamic linearization model of the two-link manipulator with coupling characteristics described in step (3.1), the mathematical formula for calculating the one-step forward error is: ； Among them, the double-link robotic arm ; for k The trajectory tracking error of the first link of the double-link manipulator at the +1 sampling time, for k +1 The trajectory tracking error of the second link of the double-link manipulator at sampling time; for k +1 The expected joint angle of the first link of the two-link robot at sampling time, for k +1 The expected joint angle of the second link of the two-link robot at sampling time; Step (3.3): In the optimization update process, the Jacobian matrix of the two-link manipulator is used; based on the real-time dynamic linearization model of the two-link manipulator with coupling characteristics described in step (3.1), the mathematical formula for calculating the partial derivative matrix of the two-link manipulator is: ； in, for against The derivative of , ; Step (3.4): In the optimization and updating process, the dual-link robot arm is used Connecting rod control instructions ,in , respectively for the time-varying block matrix The partial derivative matrix of all elements in L is the controller pseudo order, L is a positive integer, and the mathematical formula for calculating the partial derivative matrix is: ； Step (3.5): Define the time difference function ,in is the value function, is the discount factor; to minimize the system performance index function As the goal, the gradient descent method is used to optimize the update The weights of the online Actor-Critic reinforcement learning subnetwork, which contains the Actor network weights and Critic network weights , h is the number of hidden layer nodes of the reinforcement learning network, The value of traverses the positive integer interval All values ​​in The weights of an online Actor-Critic reinforcement learning subnetwork; Optimization Update The sampling time Actor network weights for online Actor-Critic reinforcement learning subnetworks : If the controller is pseudo-order ,but ； If the controller is pseudo-order ,but ； in, For the The learning rate of the Actor network, , ; for k Sampling time double link manipulator The trajectory tracking error of the connecting rod, The mathematical formula for the one-step forward error described in step (3.2) is used for calculation, ; is a value with a mean of 0 and a variance of The normal distribution function of For the The hidden layer of the online Actor-Critic reinforcement learning subnetwork i The output of each node; Optimization Update The sampling time Critic network weights of online Actor-Critic reinforcement learning subnetworks : ； in, For the The learning rate of the Critic network; Step (3.6): Optimize update The time-varying block matrix of the multivariable decoupled model-free controller at the sampling time: If the controller is pseudo-order ,but ； Will The value of traverses the positive integer interval All values ​​within, obtain the time-varying block matrix; If the controller is pseudo-order ,but ； in, ,Will The value of traverses the positive integer interval All values ​​within , get the time-varying block matrix.

4. The model-free control method of the double-link robotic arm in cloud-edge collaboration according to claim 1 is characterized in that: The Actor-Critic reinforcement learning network adopts a radial basis function network; the radial basis function network adopts a structure with a single hidden layer.

5. The model-free control method of the double-link robotic arm in cloud-edge collaboration according to claim 1 is characterized in that: In step (2), the edge end preprocesses the real-time data, including filtering and denoising the real-time data.

6. The model-free control method of the cloud-edge-end collaborative double-link robotic arm according to claim 1 is characterized in that: In the step (1), the terminal collects real-time data of the double-link robotic arm through a sensor signal acquisition device.

7. The model-free control system of the double-link robotic arm under cloud-edge collaboration is characterized by: include: The terminal includes a double-link mechanical arm running in real time, a data acquisition module, a first data output module, and a first data input module; A data acquisition module, used to collect real-time data of the double-link robotic arm, wherein the real-time data includes a real-time expected joint angle of the first link of the double-link, a real-time operating joint angle of the first link, a real-time expected joint angle of the second link, and a real-time operating joint angle of the second link; A first data output module, used to send the real-time data to an edge end; A first data input module, used to receive a control instruction issued by the edge end, wherein the control instruction includes a control instruction of the first connecting rod and a control instruction of the second connecting rod; The edge end includes a second data input module, a data preprocessing module, a multivariable decoupling model-free controller calculation module, and a second data output module; A second data input module, used for receiving real-time data of the double-link robotic arm uploaded by a terminal; A data preprocessing module, used for preprocessing the real-time data and then inputting it into a multivariable decoupling model-free controller calculation module, and sending it to the cloud; A multivariable decoupled model-free controller calculation module; used for calculating the control instructions of the double-link manipulator through a multivariable decoupled model-free controller that performs adaptive learning based on an online Actor-Critic reinforcement learning network, wherein the control instructions include the control instructions of the first link and the control instructions of the second link; A second data output module, used for sending the control instruction to the terminal; The cloud comprises a third data input module, a data storage module, a multivariable decoupling model-free controller optimization and updating module, and a third data output module; The third data input module is used to receive the pre-processed real-time data uploaded by the edge end; A data storage module, used to store the pre-processed real-time data uploaded by the edge end; A multivariable decoupling model-free controller optimization and update module is used to optimize and update the parameters of the multivariable decoupling model-free controller; The third data output module is used to transmit the optimized and updated multivariable decoupled model-free controller parameters to the edge end.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the model-free control method of a double-link robotic arm under cloud-edge collaboration as described in any one of claims 1 to 6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the model-free control method of the double-link robotic arm under cloud-edge collaboration as described in any one of claims 1 to 6.

Images